Tuesday, 2021-04-13

*** mlavalle has quit IRC00:12
*** mailingsam has quit IRC00:19
*** jralbert has quit IRC00:23
*** lbragstad_ has joined #opendev00:43
*** lbragstad has quit IRC00:45
ianwok, the unreachable mirror node has made the base job fail :/01:04
kevinzianw: Just saw your email, I will take a look at this now01:08
ianwthanks :)01:13
*** hamalq has quit IRC01:19
openstackgerritIan Wienand proposed opendev/system-config master: Add planet.openstack.org redirect to static  https://review.opendev.org/c/opendev/system-config/+/78599301:19
openstackgerritIan Wienand proposed opendev/system-config master: Add planet.openstack.org redirect to static  https://review.opendev.org/c/opendev/system-config/+/78599301:21
kevinzianw: recovered,01:24
kevinzLooks the l3_agent is not recovered well after restart.  Now it is back01:24
ianwkevinz: excellent, thanks; it can ping ipv4 again01:25
ianwand afs is back too01:26
kevinzianw: Np :-)01:27
openstackgerritIan Wienand proposed opendev/system-config master: Stop managing planet01.openstack.org  https://review.opendev.org/c/opendev/system-config/+/78599401:29
openstackgerritIan Wienand proposed opendev/system-config master: all-clouds: correct typo in OSU configuration  https://review.opendev.org/c/opendev/system-config/+/78599701:36
openstackgerritMerged opendev/system-config master: all-clouds: correct typo in OSU configuration  https://review.opendev.org/c/opendev/system-config/+/78599702:39
openstackgerritIan Wienand proposed opendev/system-config master: all-clouds: add OSU OSL project_id as well  https://review.opendev.org/c/opendev/system-config/+/78600203:34
openstackgerritIan Wienand proposed opendev/system-config master: Add planet.openstack.org redirect to static  https://review.opendev.org/c/opendev/system-config/+/78599304:15
openstackgerritIan Wienand proposed opendev/system-config master: Stop managing planet01.openstack.org  https://review.opendev.org/c/opendev/system-config/+/78599404:15
openstackgerritIan Wienand proposed opendev/zone-opendev.org master: Add OSUOSL mirror node  https://review.opendev.org/c/opendev/zone-opendev.org/+/78600304:22
openstackgerritIan Wienand proposed opendev/system-config master: Add OSUOSL mirror  https://review.opendev.org/c/opendev/system-config/+/78600404:23
*** d34dh0r53 has quit IRC04:30
openstackgerritMerged opendev/zone-opendev.org master: Add OSUOSL mirror node  https://review.opendev.org/c/opendev/zone-opendev.org/+/78600304:39
ianwi've rebooted review02 with the netplan pin04:53
ianwkevinz: it still seems like we're getting a lot of node_failure results04:58
ianw2021-04-13 04:38:56,102 ERROR nodepool.NodeLauncher: [e: bc285eb9006841dcad6f0f9a512c2032] [node_request: 300-0013667983] [node: 0023969930] Detailed node error: No valid host was found. There are not enough hosts available.05:00
ianwsame issue as yesterday; i'll send you a mail with some id's05:01
*** akahat has quit IRC05:02
*** marios has joined #opendev05:03
*** fbo has quit IRC05:03
openstackgerritMerged opendev/system-config master: all-clouds: add OSU OSL project_id as well  https://review.opendev.org/c/opendev/system-config/+/78600205:04
*** fbo has joined #opendev05:07
*** akahat has joined #opendev05:14
openstackgerritIan Wienand proposed opendev/system-config master: Add planet.openstack.org redirect to static  https://review.opendev.org/c/opendev/system-config/+/78599305:18
openstackgerritIan Wienand proposed opendev/system-config master: Stop managing planet01.openstack.org  https://review.opendev.org/c/opendev/system-config/+/78599405:18
*** ralonsoh has joined #opendev05:29
*** ralonsoh has quit IRC05:29
*** ralonsoh has joined #opendev05:30
*** slaweq has joined #opendev05:52
*** sboyron has joined #opendev06:08
openstackgerritMerged opendev/system-config master: Add OSUOSL mirror  https://review.opendev.org/c/opendev/system-config/+/78600406:15
openstackgerritIan Wienand proposed opendev/system-config master: Add planet.openstack.org redirect to static  https://review.opendev.org/c/opendev/system-config/+/78599306:17
openstackgerritIan Wienand proposed opendev/system-config master: Stop managing planet01.openstack.org  https://review.opendev.org/c/opendev/system-config/+/78599406:17
*** amoralej|off is now known as amoralej06:23
ianwthe osu osl cloud mirror should deploy now; i'll check on it later.  i'm off for now06:32
*** dpawlik has joined #opendev06:33
*** dpawlik has quit IRC06:37
*** dpawlik has joined #opendev06:40
*** dpawlik has quit IRC06:42
*** eolivare has joined #opendev06:46
*** fressi has joined #opendev06:51
*** dpawlik has joined #opendev06:54
*** dpawlik has quit IRC06:55
*** dpawlik has joined #opendev07:12
*** dpawlik has joined #opendev07:12
*** dpawlik has quit IRC07:13
*** dpawlik has joined #opendev07:14
*** andrewbonney has joined #opendev07:14
kevinzianw: I wil check, so just arrived home07:23
openstackgerritBernard Cafarelli proposed openstack/project-config master: Update neutron stable grafana dashboards  https://review.opendev.org/c/openstack/project-config/+/78600807:23
openstackgerritIan Wienand proposed opendev/system-config master: Fix OSU OSL mirror host variable file typo  https://review.opendev.org/c/opendev/system-config/+/78600907:23
kevinzianw: sorry just arrived home,  I will check it now07:23
ianwkevinz: no problems; it seems *mostly* ok, but a few failures popping up07:24
kevinzianw: I suppose it should be the resource is used up07:24
ianwi don't think it's near capacity ...07:25
ianwat least, we're not at our capacity ... but someone else in the cloud?07:25
kevinzianw: It looks that it did not reach the capacity07:26
*** jpena|off is now known as jpena07:34
openstackgerritIan Wienand proposed opendev/system-config master: host_vars : add .yaml extension  https://review.opendev.org/c/opendev/system-config/+/78601107:35
*** sboyron has quit IRC07:37
*** sboyron has joined #opendev07:38
*** sboyron has quit IRC07:38
*** sboyron has joined #opendev07:39
*** rpittau|afk is now known as rpittau07:43
kevinzianw: Just double check the total scheduler policy and capacity,  I calculate the totally memories quota for os-jobs is 732G,  and cpus is 320 * 2(cpu_overcommit) = 64007:45
openstackgerritMerged openstack/project-config master: Update neutron stable grafana dashboards  https://review.opendev.org/c/openstack/project-config/+/78600807:46
hrwnice, new aarch64 nodes ;)07:47
kevinzhrw: morning!07:48
*** tosky has joined #opendev07:49
kevinzianw: it looks that the os-jobs is not reaching the limit anyway.  And I think we can also use some small vms to run UT for nova/neutron/cinder,  rather than the os.large with 8C8G vms.07:49
openstackgerritMerged opendev/system-config master: Fix OSU OSL mirror host variable file typo  https://review.opendev.org/c/opendev/system-config/+/78600907:57
*** hberaud has left #opendev08:25
*** dtantsur|afk is now known as dtantsur08:31
*** janders8 is now known as janders08:32
*** hrw has quit IRC08:47
*** hrw has joined #opendev08:49
*** dtantsur is now known as dtantsur|brb08:57
*** snapdeal has joined #opendev09:07
*** dpawlik has quit IRC09:28
*** akahat has quit IRC09:28
*** fbo has quit IRC09:29
*** kopecmartin has joined #opendev09:33
*** snapdeal has quit IRC09:43
ianwkevinz: we've been pretty consistent in keeping our nodes at 8gb environments.  we could have smaller variants, but it's something we'd want to consider i guess09:47
*** fbo has joined #opendev09:48
*** dpawlik has joined #opendev09:52
*** akahat has joined #opendev09:53
*** rpittau is now known as rpittau|bbl09:54
*** sboyron has quit IRC09:54
*** sboyron has joined #opendev09:55
ianwkevinz: at our current limit of 40 nodes i wouldn't think we'd be having issues with 732g ram and 640-odd cpus09:58
*** dtantsur|brb is now known as dtantsur10:00
openstackgerritIan Wienand proposed opendev/system-config master: OSU OSL: fix typo in certificate name  https://review.opendev.org/c/opendev/system-config/+/78603110:09
ianwosuosl and osusol are hard to tell apart!10:11
openstackgerritIan Wienand proposed opendev/system-config master: OSU OSL: fix typo in certificate name  https://review.opendev.org/c/opendev/system-config/+/78603110:18
kevinzianw: Np, I will check if there is bug in our nova-scheduler10:18
*** dpawlik has quit IRC10:45
*** dpawlik9 has joined #opendev10:52
openstackgerritMerged opendev/system-config master: OSU OSL: fix typo in certificate name  https://review.opendev.org/c/opendev/system-config/+/78603111:00
*** tkajinam has quit IRC11:13
*** Alex_Gaynor has joined #opendev11:15
Alex_Gaynor👋 is there an issue with the arm64 provider? Those jobs don't appear to be starting.11:15
ianwAlex_Gaynor: probably, we've been having issues recently :/11:21
ianwwe've got a lot of active nodes currently11:22
kevinzianw: I see several new vm are launched11:22
ianwkevinz: yeah, i might be being a bit pessimistic11:23
Alex_GaynorOk, looks like they are starting. Just a bit slow this morning. I empathasize, I too have not had my tea yet.11:23
ianwwe've got 24 nodes active, 13 building11:23
kevinzianw: But launching is slow due to parallelly creation11:23
ianwso we're close to capacity11:23
kevinzianw: yes,  I see11:24
ianwAlex_Gaynor: working on getting about an extra 15 nodes from OSU online soon11:24
Alex_Gaynoroooh, that's exciting!11:24
ianwof course, interest in using it tends to expand with capacity as well :)11:25
kevinzI advice we do not need to run some architecture non-related UT jobs11:25
kevinzianw: I saw neutron ut is requesting quite a lot times..11:26
ianwyeah, in one way, that's probably not likely to break on arm64; but it also has a lot of binary dependencies going on under the hood which are good to expose to alternative platforms11:27
kevinzianw: OK,  that's a good point.11:29
ianwit is a fair point though, i mean given the resources we always have to consider the usage/reward pay-off11:30
ianwhttps://us.linaro.cloud:8774/v2.1/servers, Quota exceeded for instances: Requested 1, but already used 41 of 40 instances11:31
ianwi wonder why we're exceeding the quota11:31
*** jpena is now known as jpena|lunch11:31
ianwok, i'm out for now, ttyl11:32
*** artom has joined #opendev12:15
*** amoralej is now known as amoralej|lunch12:29
*** rpittau|bbl is now known as rpittau12:31
*** jpena|lunch is now known as jpena12:37
*** klonn has joined #opendev12:42
*** d34dh0r53 has joined #opendev12:53
*** mailingsam has joined #opendev13:15
*** tosky has quit IRC13:19
*** amoralej|lunch is now known as amoralej13:22
*** tosky has joined #opendev13:25
*** tosky has quit IRC13:29
*** tosky has joined #opendev13:29
*** mlavalle has joined #opendev13:40
*** lpetrut has joined #opendev13:56
*** klonn has quit IRC14:02
*** lpetrut has quit IRC14:20
*** lbragstad_ is now known as lbragstad14:31
*** klonn has joined #opendev14:52
*** amoralej is now known as amoralej|off15:05
*** DSpider has joined #opendev15:36
*** klonn has quit IRC15:54
*** marios is now known as marios|out16:09
clarkbfungi: I'm going to put together PTG info today and send it to openstack-discuss with [tact-sig][dev][infra][qa] subject prefix and a separate email to service-discuss. I'm going to position it as office hours/help/debug/etc time and will ask people to throw their topics on an etherpad (to be made) as well to help scheduling16:13
clarkbfungi: do you think that is sufficient for getting this info in front of the necessary individuals?16:13
fungiyes, that sounds perfect. thanks!16:15
*** hamalq has joined #opendev16:20
*** rpittau is now known as rpittau|afk16:24
*** marios|out has quit IRC16:38
*** jpena is now known as jpena|off16:43
openstackgerritMerged opendev/base-jobs master: Revert "Temporarily disable log uploads to OVH BHS Swift"  https://review.opendev.org/c/opendev/base-jobs/+/78501916:44
clarkbemails sent to service-discuss@lists.opendev.org and openstack-discuss lists.16:52
JayFwhat is a TaCT sig?17:04
*** DSpider has quit IRC17:05
clarkbJayF: https://governance.openstack.org/sigs/tact-sig.html basically the group of people within the openstack project that tie in closely with opendev making use of zuul, maintaining configs for zuul, etc17:05
JayFah, makes sense. Thanks!17:06
*** DSpider has joined #opendev17:06
*** dtantsur is now known as dtantsur|afk17:11
openstackgerritMerged opendev/system-config master: Add zuul keystore password  https://review.opendev.org/c/opendev/system-config/+/78598017:15
*** cenne is now known as cenne-away17:16
*** klonn has joined #opendev17:16
fungiJayF: when the opendev collaboratory became independent of the openstack project, the tact sig is more or less the openstack-specific bits the old infra team took care of17:19
JayFWhen you have an "infra IRC peanut gallery"-sig, I'll attend ;)17:20
fungiso that opendev wouldn't be burdened with openstack-specific responsibilities17:20
JayFThat makes a lot of sense17:20
fungi(ultimately it's a lot of the same people, just helps us draw clearer boundaries between what's opendev and what's openstack)17:20
*** sshnaidm has joined #opendev17:28
*** sshnaidm is now known as sshnaidm|pto17:28
*** eolivare has quit IRC18:00
openstackgerritJeremy Stanley proposed openstack/project-config master: Add an empty project for an OpenStack base ACL  https://review.opendev.org/c/openstack/project-config/+/78283018:33
openstackgerritJeremy Stanley proposed openstack/project-config master: Allow delete permissions in Gerrit ACLs  https://review.opendev.org/c/openstack/project-config/+/78608818:33
*** mailingsam has quit IRC18:51
*** andrewbonney has quit IRC18:54
*** auristor has quit IRC19:01
*** auristor has joined #opendev19:04
*** auristor has quit IRC19:05
ianwanyone ever seen something like19:07
ianwBadRequestException: 400: Client Error for url: http://arm-openstack.osuosl.org:9696/v2.0/security-group-rules, Unrecognized attribute(s) 'remote_address_group_id'19:07
ianwout of the cloud launcher?19:07
funginope, maybe some neutron folks can read those tea leaves?19:08
ianwhttps://storyboard.openstack.org/#!/story/2008577 ... looks like i'm not alone :/19:12
*** ralonsoh has quit IRC19:13
ianwit seems the field was added with https://review.opendev.org/c/openstack/openstacksdk/+/755644 but unconditionally19:20
*** beef_parotta has joined #opendev19:23
fungioh neat19:24
fungiand the version of neutron deployed there lacks support for it?19:24
ianwi guess so, that's my working theory.  always nice to find my yak nice and early in the day :)19:25
*** auristor has joined #opendev19:25
fungigives you time to become friends before the shavnig begins19:26
*** beef_parotta has quit IRC19:30
JayFI gave hangyang a heads up about that bug, since he committed the blamed code (he is a downstream co-worker)19:30
ianw JayF: thanks :)19:34
openstackgerritJeremy Stanley proposed opendev/jeepyb master: Update Gerrit hook command-line parameters  https://review.opendev.org/c/opendev/jeepyb/+/78609519:53
fungiit's also become apparent that we haven't restarted gerrit since the earlier jeepyb fix to stop trying to assign bugs merged, so we should get that ^ landed too and then plan for a restart sometime when it won't be too disruptive19:54
fungimelwitt: yoctozepto: ^ progress!19:54
*** avass has quit IRC19:55
fungii keep forgetting we need gerrit restarts for jeepyb updates now, since it's baked into our container images19:57
*** klonn has quit IRC20:02
ianwfungi: does that need to be backwards compatible?20:02
*** avass has joined #opendev20:02
clarkbfungi: note I don't think that is true for jeepyb commands outside of the hooks (as they do docker run but gerrit execs the hooks directly so they are special)20:02
clarkbianw: fungi: it is probably friendliest to be backward compat but I'm not sure if anyone is using jeepyb other than ourselves at this point20:03
fungiianw: i didn't keep backward compatibility in the previous change to stop assigning bugs20:03
fungimy expectation is that if anyone is continuously deploying jeepyb they're also upgrading gerrit20:04
ianwok, lgtm then20:04
fungialso i should have commented or included a link to the hooks documentation, but that's where i got the full set of parameters from20:05
clarkbianw I abandoned my 3.2.8 change to avoid any confusion and put pointers to your plugin fix20:06
fungii've hand-patched those two jeepyb changes (the merged one and the latest one) into the container and am continuing to watch my ad hoc debug log20:09
fungii need to switch to cooking dinner, but will work on a test change/bug afterward if i don't spot one in the log20:09
clarkbshould we land ianw's plugin fix, then the jeepyb fix? that way we won't end up with a broken image should we need to restart?20:09
clarkbbut then we can sit on a restart for a bit20:09
fungiyeah, that sounds fine20:10
clarkbI've +2'd both of them if fungi wants to be second reviewers on the plugin fix and ianw on the jeepyb change I think we can +a them in that order20:11
clarkbbut also eat dinner :)20:11
openstackgerritJeremy Stanley proposed opendev/jeepyb master: Correct set_in_progress parameters  https://review.opendev.org/c/opendev/jeepyb/+/78612020:15
fungiclarkb: ianw: also ^ (if it's not tested it's broken!)20:15
clarkbfungi: ianw: another one on the cleanup list should probably be firehose? I also think that one may be able to just go away?20:27
ianwyeah, seems likely20:28
clarkbcool, I can probably work on that since ya'll have others you are looking at20:29
clarkbI'm double checking what injects into mqtt and it seems that we stopped doing ansible at some point, germqtt is on firehose and lpmqtt is one firehose. This means i need to disable mqtt on the subunit and logstash workers20:38
clarkbwe should also likely cleanup the inbox that lpmqtt uses? fungi do you know what it uses?20:38
clarkblogstash workers already disable mqtt20:39
mtreinishI'll have to ping andreaf, I think he was still playing with https://github.com/mtreinish/ciml a little bit which depends on firehose20:44
*** sboyron has quit IRC20:46
clarkbmtreinish: ok, the data can be retrieved through other channels too if necessary (like the gerrit event stream for gerrit events etc)20:47
mtreinishbut it's not critical, there are alternatives (the firehose bit was just for demoing real time reporting)20:47
fungiclarkb: there is a cyrus-imap service running on the firehose server, we simply subscribed its address to some projects on lp20:47
clarkbfungi: oh so we need to remove email addrs from the lp side20:47
clarkbmtreinish: at this point all the config management needs rewriting and it was never really used in production for much so it is easier to turn if off if you need to pass along the reasoning20:48
fungialso we can't upgrade the underlying operating system because the existing configuration management for it would break20:48
mtreinishoh, I don't think it really needs a reason. It was a fun experiment, but it never really caught on20:49
fungiyeah, it was a worthwhile endeavor20:50
fungii certainly learned a lot from that project20:50
mtreinishyeah, me too20:51
mtreinishheh, and it was the first time I wrote haskell (for the docs) :P20:52
openstackgerritClark Boylan proposed opendev/system-config master: Stop publishing subunit worker data to mqtt  https://review.opendev.org/c/opendev/system-config/+/78612520:52
openstackgerritClark Boylan proposed opendev/system-config master: Remove firehose.openstack.org  https://review.opendev.org/c/opendev/system-config/+/78612620:52
clarkbthe first change there should be pretty safe to land (though maybe that is what andreaf was using? I don't mind waiting a bit before landing any of that)20:52
clarkbfungi: any objections to approving the git-review --no-thin change https://review.opendev.org/c/opendev/git-review/+/785723 ?20:53
clarkbianw reviewed it too but needs a +A20:53
ianwi'm happy for +A after our discussion yesterday20:54
clarkbcool, want to do it or should I?20:54
ianwdone :)20:54
clarkbnow to figure out what lp account sends email to firehose20:55
mtreinishclarkb: nah, there's no real need to wait. I pinged him about it and I don't think he's actively using that component of ciml for anything important20:56
clarkbmtreinish: thanks!20:56
mtreinishit was listening to the subunit publish events though to trigger running predictions against the trained model based on new subunit2sql records, but it was never for anything more than a proof of concept20:57
mtreinishbut, the real meat of that project was building the ML models and getting it to predict things based on the logs (never got it to predict why a failure occured, but we could identify which cloud vendor a job ran in from it's dstat data)20:58
clarkbI can see what email address is being delivered to in the exim logs, but I don't appear to be able to reverse lookup on lp for the user that way21:00
mtreinishIIRC we put the lp creds in the secrets store with the other shared root creds (I mainly just remember being chastized for sharing the password out loud at a midcycle when we first set it up)21:02
clarkbmtreinish: ah thanks I can look there next21:03
clarkbyup found it, now to see if I can disable email alerts21:05
clarkbianw: I notice that the change to the zuul results plugin implies that the enablement is intended to go in refs/meta/config on a per project basis. Do you know if getFromProjectConfigWithInheritance(project.getNameKey(), pluginName) checks the global config like you set in your change? I assume it must since we use selenium to check this stuff now21:06
ianwclarkb: yeah, it gets defaulted on by the All-Projects update21:07
ianwas you say, the screenshots show it enabled21:07
clarkbianw: all-projects doesn't help us in prod though21:08
clarkbhttps://review.opendev.org/c/opendev/system-config/+/778116/9/playbooks/roles/gerrit/templates/gerrit.config.j2 is what we should be applying in prod and in test, if that isn't sufficient then I think we'll need to manually update all-projects too?21:08
clarkbalso if that isn't sufficient should we just remove it from gerrit.config.j2?21:08
ianwhrrrmm, now you've made me think :)21:10
clarkb(I'm also not sure I like changes like the upstream one. If you installed the plugin them maybe it should work by default rather than need to be enabled from there. I can see disabling it though)21:11
ianwactually I think yes it all comes from the global config.  i'm thinking we should actually remove it from All-Projects21:11
clarkbianw: cool, lets try that and see if it works and if so \o/21:11
ianwi.e. remove setting it in https://review.opendev.org/c/opendev/system-config/+/778116/9/playbooks/test-review.yaml21:11
clarkbfungi: mtreinish: I'm in the lp account and it doesn't appear to be subscribed to any mailing lists and if I got to bugs it says there are no subscribed packages or bugs21:12
clarkbany idea what bit of lp configuration causes it to get emails?21:12
mtreinishit was a global thing for all of openstack. Maybe a group membership or something21:13
openstackgerritIan Wienand proposed opendev/system-config master: Handle zuul-summary-results as .jar / per-project config  https://review.opendev.org/c/opendev/system-config/+/77811621:13
mtreinishit's been too long since I was last using lp regularly I don't remember the details of how to do that21:14
clarkbmtreinish: same here, but that helps thanks21:14
ianwclarkb: ^^ will keep an eye on21:14
mtreinishprobably something related to: https://launchpad.net/openstack21:14
mtreinishclarkb: is just deleting the lp account an option?21:15
mtreinishthat might be easier :)21:15
clarkbyes you can delete an account too21:16
clarkbI was thinking I would do that after firehose went away, but disable the email alerts  before21:16
clarkbmaybe it is ok for lp to fail to deliver email for a short time and delete after without stopping the emails first21:17
*** tristanC_ is now known as tristanC21:19
clarkbI'll punt on this for now. I think fungi groks this stuff pretty well and probably knows the anwer or how to find it21:21
fungii'll catch up on all this after i'm done eating21:36
fungibut yeah, should just be able to auth as that account and unsubscribe it from https://bugs.launchpad.net/openstack21:39
clarkbhrm when I logged in and went to the susbcribed bug list page it said there were no subscriptions, but maybe going to that page directly would work22:02
clarkband no rush I think I'll pick this back up tomorrow22:02
fungior i'm happy to work on it once wallaby release jobs are in the clear22:07
clarkbif you want to do the unsub I won't complain :)22:07
clarkbonce that happens and https://review.opendev.org/c/opendev/system-config/+/786125 lands and is applied my next plan was to stop services on the server and just double check eeverything remains happy22:07
clarkbthen we can land the cleanup change22:08
fungiokay, 782603+786095+786120 have all been hand-patched into the jeepyb inside the gerrit container on review.o.o for the past hour, and i've seen no new backtraces from any of the hooks which call update-bug22:11
fungiso that's a good sign at least22:11
clarkbhrm I don't remember reviewing 3, just 2 /me looks22:12
fungi782603 merged last week22:12
fungiwe just haven't restarted gerrit22:12
clarkbah ya and I reviewed it, got it22:13
fungiso i had to patch it into the volume tree for the currently running container, which is not the latest one pulled on the server22:13
clarkbfungi: I think we want to restart with https://review.opendev.org/c/opendev/system-config/+/778116/ in place too, once testing confirms the global config is sufficient22:13
fungiyeah, i figure we can plan to do a restart at some quiet time later this week22:14
clarkbyup sounds like friday is looking good for it22:15
fungiin theory, update-bug is fixed now with the hand-patched jeepyb in the container, i just haven't confirmed we've gotten any actual bug updates yet22:17
*** tkajinam has joined #opendev22:23
clarkbianw: looks like it says "zuul integration is not enabled"22:40
clarkbNot sure I'm a fan of the upstream change :/ would really have preferred that installing the plugin implied on by default, disable if you want to rather than off by default if plugin is present22:40
clarkbthat said updating our All-Projects config isn't terribly difficult, we would just need to plan to do it after we updated our image22:41
ianwclarkb: hrm, i feel like it should be able to read from the config file too22:41
ianwit probably needs more java22:42
clarkbthat would be another good option, refer to the system config in addition to project configs22:43
corvusianw: er, regarding your comment on https://gerrit-review.googlesource.com/c/plugins/zuul-results-summary/+/298465/ i can confirm there are more users than just opendev22:47
corvusianw, clarkb: do you think we could switch the default real quick? :)22:48
clarkbcorvus: that would probably be my preference. Since installing the plugin implies you want the functionality22:49
clarkbthen if you don't want the functionality on a per project basis you can disable them individually22:49
corvusi also agree it seems counter-intuitive to have it off by default; it's just as easy for someone to set it to 'off' in all-projects then enable it per-project if they want the other behavior22:49
clarkbbut I don't know what the motivation behind that changw as in the first place22:49
corvusclarkb: i think it's because luca has some gerritforge projects using zuul but not all22:50
corvusso it's definitely a (friendly) behavior preference conflict between those of us in an all-zuul world, and those in a semi-zuul world22:50
ianwi should have thought more about it not reading the value from the global config file22:51
ianwi'm willing to take advice, we can revert, or propose basically making  https://gerrit-review.googlesource.com/c/plugins/zuul-results-summary/+/298465/4/java/com/googlesource/gerrit/plugins/zuulresultssummary/GetConfig.java#44 true i guess22:52
corvusi was about to propose a change to do the latter22:53
clarkbya I think making the default true there is a simple fix and gives behavior that makes sense to most users (I assume most users want the functionality to exist if they install it)22:53
corvusianw, clarkb: remote:   https://gerrit-review.googlesource.com/c/plugins/zuul-results-summary/+/303262 Enable the plugin by default [NEW]23:01
ianwas i've now found out, a single +2 in gerrit's workflow is indication anyone can submit.  so i think i should probably +1 this and just check with luca/david they are ok with it23:02
ianwbut lgtm23:04
clarkbI too just left a +1 based on ^23:04
clarkband double checking with them seems like a good idea23:04
corvusi added luca and david as reviewers (as that does seem to be the practice in the gerrit community)23:04
ianwwe can make a retrospective tag before this change, if that would help23:05
clarkbmaybe? I don't know what typical practice for plugin installs is. We have the flexibility to do branches, tags, refs, or shas I think23:07
ianwyeah, it would depend on what the other users are deploying with if a tag is easier.23:08
clarkbfungi: just saw that https://review.opendev.org/c/opendev/system-config/+/785675 (stop using our ubuntu ppa on debian for openafs) is passing now \o/23:14
clarkbianw: ^ that should be an easy review and a cleanup from the great openafs clock bug23:14
corvusianw: i imagine the latest builds from gerritforge is a pretty common deployment method23:14
corvusi think that's what happens if you use the plugin manager too?23:15
fungiclarkb: yeah, earlier failures were just arm testing shenanigans, the change was working23:15
clarkbcorvus: looks like the plugin manager tries to match gerrit versions to plugin versions, but this plugin doesn't do gerrit specific versioning so I suspect it is functionally equivalent to installing latest23:16
*** tosky has quit IRC23:34
openstackgerritMerged opendev/system-config master: Drop Debian PPA from openafs-client role  https://review.opendev.org/c/opendev/system-config/+/78567523:44
ianwhttps://review.opendev.org/c/openstack/openstacksdk/+/786148 is my guess at fixing the cloud-launcher security group issues with OSU OSL23:49
ianwi guess now i have some sort of solution i feel a bit better about either pinning openstacksdk or creating the rules by hand23:49
*** DSpider has quit IRC23:56

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!