| *** tbachman has joined #openstack-infra | 00:01 | |
| *** wolverineav has quit IRC | 00:01 | |
| *** wolverineav has joined #openstack-infra | 00:01 | |
| *** wolverineav has quit IRC | 00:04 | |
| *** yamamoto has quit IRC | 00:04 | |
| *** wolverineav has joined #openstack-infra | 00:05 | |
| *** wolverineav has quit IRC | 00:10 | |
| *** tbachman has quit IRC | 00:15 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Retire linaro-cn1 https://review.openstack.org/649773 | 00:19 |
|---|---|---|
| *** sshnaidm_ has quit IRC | 00:25 | |
| *** tbachman has joined #openstack-infra | 00:27 | |
| *** sshnaidm has joined #openstack-infra | 00:29 | |
| *** jamesmcarthur has joined #openstack-infra | 00:34 | |
| *** wolverineav has joined #openstack-infra | 00:35 | |
| *** igordc has quit IRC | 00:36 | |
| *** auristor has quit IRC | 00:47 | |
| *** auristor has joined #openstack-infra | 00:49 | |
| *** auristor has quit IRC | 00:54 | |
| *** wolverineav has quit IRC | 00:55 | |
| *** wolverineav has joined #openstack-infra | 00:56 | |
| *** auristor has joined #openstack-infra | 00:57 | |
| *** jamesmcarthur has quit IRC | 00:58 | |
| *** wolverineav has quit IRC | 01:00 | |
| ianw | nowster / jrosser : i added bionic to our openafs builds in the infra ppa @ https://launchpad.net/~openstack-ci-core/+archive/ubuntu/openafs-arm64/ ... it's building, see what happens | 01:01 |
| *** mriedem has quit IRC | 01:09 | |
| *** auristor has quit IRC | 01:11 | |
| *** auristor has joined #openstack-infra | 01:11 | |
| *** rh-jelabarre has quit IRC | 01:15 | |
| *** aaronsheffield has quit IRC | 01:22 | |
| *** jamesmcarthur has joined #openstack-infra | 01:23 | |
| *** jamesmcarthur has quit IRC | 01:31 | |
| *** yamamoto has joined #openstack-infra | 01:45 | |
| *** roman_g has quit IRC | 01:59 | |
| *** whoami-rajat has joined #openstack-infra | 02:03 | |
| *** wolverineav has joined #openstack-infra | 02:03 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:06 | |
| *** wolverineav has quit IRC | 02:08 | |
| *** tinwood has quit IRC | 02:10 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] add buster tests https://review.openstack.org/649497 | 02:10 |
| *** tinwood has joined #openstack-infra | 02:12 | |
| *** hongbin has joined #openstack-infra | 02:12 | |
| *** bhavikdbavishi has quit IRC | 02:13 | |
| *** ruffian_sheep has joined #openstack-infra | 02:20 | |
| ianw | clarkb / gary_perkins : https://storyboard.openstack.org/#!/story/2005353#comment-120181 -- it looks like arm64ci has maybe leaked something, and thinks we're at quota but I can't actually see any servers | 02:21 |
| *** mriedem has joined #openstack-infra | 02:23 | |
| ruffian_sheep | fungi:Should I run all of the CI?If I want to upload ny driver of cinder | 02:23 |
| *** markvoelker has joined #openstack-infra | 02:29 | |
| *** jamesmcarthur has joined #openstack-infra | 02:29 | |
| *** mriedem has quit IRC | 02:29 | |
| ianw | niedbalski: maybe you can look at https://bugs.linaro.org/show_bug.cgi?id=4327 | 02:35 |
| openstack | bugs.linaro.org bug 4327 in Default "[uk cloud] Unable to boot instances in linaro london cloud" [Normal,Unconfirmed] - Assigned to gema.gomez-solano | 02:35 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Retire linaro-cn1 https://review.openstack.org/649773 | 02:38 |
| ianw | i think that's about all i can do on arm64 atm ... https://storyboard.openstack.org/#!/story/2005353 updated | 02:42 |
| *** psachin has joined #openstack-infra | 02:43 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:45 | |
| *** bhavikdbavishi has quit IRC | 02:50 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:54 | |
| *** jamesmcarthur has quit IRC | 02:57 | |
| *** rlandy|bbl is now known as rlandy | 02:59 | |
| *** markvoelker has quit IRC | 02:59 | |
| *** hwoarang has quit IRC | 03:01 | |
| *** hwoarang has joined #openstack-infra | 03:06 | |
| *** sthussey has quit IRC | 03:14 | |
| *** samueldmq has quit IRC | 03:16 | |
| *** jamesmcarthur has joined #openstack-infra | 03:17 | |
| *** wolverineav has joined #openstack-infra | 03:20 | |
| *** jamesmcarthur has quit IRC | 03:22 | |
| *** wolverineav has quit IRC | 03:25 | |
| *** rlandy has quit IRC | 03:47 | |
| *** hongbin has quit IRC | 03:52 | |
| *** jamesmcarthur has joined #openstack-infra | 03:54 | |
| *** markvoelker has joined #openstack-infra | 03:57 | |
| *** jamesmcarthur has quit IRC | 03:58 | |
| *** ykarel has joined #openstack-infra | 04:03 | |
| *** udesale has joined #openstack-infra | 04:09 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: config: add tenant.toDict() method and REST endpoint https://review.openstack.org/621344 | 04:21 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add config page https://review.openstack.org/633667 | 04:21 |
| *** markvoelker has quit IRC | 04:29 | |
| *** jamesmcarthur has joined #openstack-infra | 04:30 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: configure-pypi-mirrors: add initial role https://review.openstack.org/581165 | 04:35 |
| *** ramishra has joined #openstack-infra | 04:38 | |
| *** ricolin has joined #openstack-infra | 04:57 | |
| *** pcaruana has joined #openstack-infra | 04:58 | |
| *** adriant has quit IRC | 05:09 | |
| *** adriant has joined #openstack-infra | 05:09 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: debian-minimal buster support https://review.openstack.org/649496 | 05:10 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] add buster tests https://review.openstack.org/649497 | 05:10 |
| *** e0ne has joined #openstack-infra | 05:12 | |
| *** e0ne has quit IRC | 05:18 | |
| *** ykarel is now known as ykarel|afk | 05:19 | |
| *** ccamacho has quit IRC | 05:22 | |
| *** ykarel|afk has quit IRC | 05:23 | |
| *** markvoelker has joined #openstack-infra | 05:27 | |
| *** ykarel|afk has joined #openstack-infra | 05:42 | |
| *** ykarel|afk is now known as ykarel | 05:43 | |
| *** quiquell|off is now known as quiquell | 05:46 | |
| openstackgerrit | Brendan proposed openstack-infra/zuul master: gerrit: Add some quoting in 'gerrit query' commands https://review.openstack.org/649879 | 05:49 |
| *** armax has quit IRC | 05:53 | |
| *** quiquell is now known as quiquell|rover | 05:57 | |
| *** armax has joined #openstack-infra | 06:00 | |
| *** markvoelker has quit IRC | 06:00 | |
| *** ruffian_sheep has quit IRC | 06:05 | |
| *** janki has joined #openstack-infra | 06:13 | |
| *** kopecmartin|off is now known as kopecmartin | 06:14 | |
| *** slaweq has joined #openstack-infra | 06:21 | |
| *** quiquell|rover is now known as quiquell|brb | 06:23 | |
| *** dpawlik has joined #openstack-infra | 06:26 | |
| *** pgaxatte has joined #openstack-infra | 06:32 | |
| *** kjackal has joined #openstack-infra | 06:33 | |
| *** hwoarang has quit IRC | 06:39 | |
| *** hwoarang has joined #openstack-infra | 06:41 | |
| *** rascasoft has quit IRC | 06:44 | |
| *** e0ne has joined #openstack-infra | 06:47 | |
| *** rpittau|afk is now known as rpittau | 06:48 | |
| *** jamesmcarthur has quit IRC | 06:52 | |
| *** kjackal has quit IRC | 06:53 | |
| tobiash | ianw: do you have plans for a diskimage-builder release? I would love a release containing https://review.openstack.org/642953 | 06:54 |
| *** markvoelker has joined #openstack-infra | 06:57 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Cleanup executor specific requirements https://review.openstack.org/649428 | 06:58 |
| ianw | tobish: oh, i thought i did. can do one | 07:01 |
| ianw | tobiash: there we go, 2.21.0 | 07:03 |
| tobiash | ianw: that was quick :) | 07:03 |
| tobiash | ianw: thank you very much! | 07:04 |
| ianw | fungi / zigo : so successful build & boot of buster : http://logs.openstack.org/14/649214/2/experimental/nodepool-functional-py35-debian-src/3b6cba1/controller/logs/builds/debian-buster-0000000001_log.txt.gz | 07:04 |
| ianw | if we want to go this path, dib fixes need to merge and release, and we need to roll out the fixed debootstrap to builders | 07:05 |
| *** quiquell|brb is now known as quiquell|rover | 07:08 | |
| *** ccamacho has joined #openstack-infra | 07:11 | |
| *** ginopc has joined #openstack-infra | 07:13 | |
| openstackgerrit | Brendan proposed openstack-infra/zuul master: gerrit: Add support for 'oldValue' comment-added field https://review.openstack.org/649900 | 07:14 |
| *** wolverineav has joined #openstack-infra | 07:21 | |
| *** rosmaita has quit IRC | 07:21 | |
| *** dims has quit IRC | 07:22 | |
| *** dims has joined #openstack-infra | 07:24 | |
| *** wolverineav has quit IRC | 07:25 | |
| *** rosmaita has joined #openstack-infra | 07:26 | |
| *** tosky has joined #openstack-infra | 07:26 | |
| *** rascasoft has joined #openstack-infra | 07:30 | |
| *** rascasoft has quit IRC | 07:30 | |
| *** markvoelker has quit IRC | 07:30 | |
| *** dims has quit IRC | 07:31 | |
| *** rascasoft has joined #openstack-infra | 07:31 | |
| *** helenafm has joined #openstack-infra | 07:32 | |
| *** dims has joined #openstack-infra | 07:34 | |
| *** jistr is now known as jistr|afk | 07:35 | |
| *** apetrich has joined #openstack-infra | 07:37 | |
| *** kjackal has joined #openstack-infra | 07:42 | |
| *** jpena|off is now known as jpena | 07:43 | |
| *** elod_off has quit IRC | 07:45 | |
| zigo | ianw: Thanks a lot ! | 07:46 |
| zigo | ianw: Where's the patch for dib? Do I need to get it into Buster? | 07:47 |
| *** ykarel is now known as ykarel|lunch | 07:48 | |
| *** elod_off has joined #openstack-infra | 07:52 | |
| *** jpich has joined #openstack-infra | 07:53 | |
| *** spsurya has joined #openstack-infra | 07:55 | |
| *** gfidente has joined #openstack-infra | 08:02 | |
| yamamoto | i have a question about zuul definition | 08:03 |
| yamamoto | with https://review.openstack.org/#/c/649878/9/.zuul.yaml only tap-as-a-service-tempest-plugin-ocata doesn't run | 08:03 |
| yamamoto | i couldn't find any obvious reasons | 08:03 |
| frickler | yamamoto: the "devstack" job doesn't exists in devstack for stable/ocata, so "devstack-tempest" doesn't find a parent there and cannot run | 08:08 |
| yamamoto | ah, job definition is taken from that branch? i thought it was master | 08:09 |
| *** iurygregory has joined #openstack-infra | 08:10 | |
| frickler | yamamoto: for projects that have the same branches, like devstack, it is taken from the branch, as the job definition may change per cycle | 08:10 |
| yamamoto | i got it. thank you! | 08:12 |
| *** tkajinam has quit IRC | 08:22 | |
| *** dtantsur|afk is now known as dtantsur | 08:25 | |
| *** markvoelker has joined #openstack-infra | 08:28 | |
| openstackgerrit | Brendan proposed openstack-infra/zuul master: gerrit: Add some quoting in 'gerrit query' commands https://review.openstack.org/649879 | 08:28 |
| *** e0ne has quit IRC | 08:30 | |
| *** roman_g has joined #openstack-infra | 08:36 | |
| *** e0ne has joined #openstack-infra | 08:40 | |
| *** e0ne has quit IRC | 08:42 | |
| *** yamamoto has quit IRC | 08:43 | |
| *** ykarel|lunch is now known as ykarel | 08:45 | |
| *** jamesmcarthur has joined #openstack-infra | 08:49 | |
| *** e0ne has joined #openstack-infra | 08:50 | |
| *** yamamoto has joined #openstack-infra | 08:52 | |
| *** yamamoto has quit IRC | 08:53 | |
| *** ricolin has quit IRC | 08:53 | |
| *** jamesmcarthur has quit IRC | 08:54 | |
| *** jaosorior has quit IRC | 08:54 | |
| *** electrofelix has joined #openstack-infra | 08:58 | |
| *** yamamoto has joined #openstack-infra | 08:59 | |
| *** ykarel is now known as ykarel|away | 09:00 | |
| *** markvoelker has quit IRC | 09:01 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: Add proper __repr__ to merger repo https://review.openstack.org/649949 | 09:05 |
| *** ykarel|away has quit IRC | 09:05 | |
| nowster | ianw: I'd taken the long term approach. ;-) | 09:12 |
| *** yamamoto has quit IRC | 09:18 | |
| nowster | ianw: looks like it built. Do you need any resources for building wheels? | 09:22 |
| *** ykarel|away has joined #openstack-infra | 09:33 | |
| *** ykarel|away is now known as ykarel | 09:33 | |
| *** markmcd has joined #openstack-infra | 09:38 | |
| *** e0ne has quit IRC | 09:44 | |
| *** wolverineav has joined #openstack-infra | 09:45 | |
| *** wolverineav has quit IRC | 09:49 | |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config master: Retire oslosphinx https://review.openstack.org/649072 | 09:51 |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config master: Initiate retirement of oslosphinx https://review.openstack.org/649954 | 09:51 |
| *** e0ne has joined #openstack-infra | 09:54 | |
| *** markvoelker has joined #openstack-infra | 09:58 | |
| zigo | FYI, clarkb's pip patch reached Buster yesterday. | 10:03 |
| zigo | So, it's fixed over there. | 10:03 |
| *** dtantsur is now known as dtantsur|brb | 10:07 | |
| *** shachar has quit IRC | 10:09 | |
| *** shachar has joined #openstack-infra | 10:09 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add grenade-py3 job to Neutron Grafana dashboard https://review.openstack.org/649352 | 10:17 |
| *** bhavikdbavishi has quit IRC | 10:22 | |
| *** jistr|afk is now known as jistr | 10:23 | |
| *** markvoelker has quit IRC | 10:31 | |
| *** ramishra has quit IRC | 10:33 | |
| *** derekh has joined #openstack-infra | 10:35 | |
| *** lmiccini has joined #openstack-infra | 10:38 | |
| *** ramishra has joined #openstack-infra | 10:39 | |
| *** tbachman has quit IRC | 10:44 | |
| *** nicolasbock has joined #openstack-infra | 10:45 | |
| *** yamamoto has joined #openstack-infra | 10:50 | |
| *** jaosorior has joined #openstack-infra | 10:52 | |
| *** udesale has quit IRC | 10:54 | |
| *** nicolasbock has quit IRC | 10:57 | |
| *** jpena is now known as jpena|lunch | 10:58 | |
| *** jaosorior has quit IRC | 10:59 | |
| *** zbr has quit IRC | 11:14 | |
| *** kjackal has quit IRC | 11:15 | |
| *** nicolasbock has joined #openstack-infra | 11:20 | |
| *** helenafm has quit IRC | 11:22 | |
| *** bhavikdbavishi has joined #openstack-infra | 11:25 | |
| *** markvoelker has joined #openstack-infra | 11:28 | |
| *** jamesmcarthur has joined #openstack-infra | 11:30 | |
| *** zbr has joined #openstack-infra | 11:31 | |
| *** jamesmcarthur has quit IRC | 11:34 | |
| *** kjackal has joined #openstack-infra | 11:36 | |
| *** quiquell|rover is now known as quiquell|rover|e | 11:40 | |
| *** quiquell|rover|e is now known as quique|rover|eat | 11:40 | |
| *** samueldmq has joined #openstack-infra | 11:40 | |
| *** ykarel has quit IRC | 11:45 | |
| *** shachar has quit IRC | 11:46 | |
| *** snapiri has joined #openstack-infra | 11:46 | |
| *** ykarel has joined #openstack-infra | 11:46 | |
| openstackgerrit | Brendan proposed openstack-infra/zuul master: gerrit: Add support for 'oldValue' comment-added field https://review.openstack.org/649900 | 11:46 |
| *** e0ne has quit IRC | 11:50 | |
| *** rh-jelabarre has joined #openstack-infra | 11:54 | |
| *** dtantsur|brb is now known as dtantsur | 11:57 | |
| *** rlandy has joined #openstack-infra | 11:58 | |
| *** trown|outtypewww is now known as trown | 12:01 | |
| *** markvoelker has quit IRC | 12:02 | |
| *** tbachman has joined #openstack-infra | 12:05 | |
| *** ramishra_ has joined #openstack-infra | 12:07 | |
| *** ramishra has quit IRC | 12:07 | |
| *** quique|rover|eat is now known as quiquell|rover | 12:13 | |
| *** jamesdenton has quit IRC | 12:13 | |
| *** jamesmcarthur has joined #openstack-infra | 12:18 | |
| *** jpena|lunch is now known as jpena | 12:22 | |
| *** kgiusti has joined #openstack-infra | 12:30 | |
| *** jamesmcarthur has quit IRC | 12:32 | |
| *** kgiusti has quit IRC | 12:36 | |
| openstackgerrit | Brendan proposed openstack-infra/zuul master: Break stuff https://review.openstack.org/649976 | 12:38 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: A reporter for Elasticsearch with the capability to index build and buildset results in an index. https://review.openstack.org/644927 | 12:38 |
| openstackgerrit | Brendan proposed openstack-infra/zuul master: Break stuff https://review.openstack.org/649976 | 12:39 |
| mordred | fungi: where are we keeping the list of project renames? | 12:39 |
| *** bhavikdbavishi has quit IRC | 12:41 | |
| *** zul has joined #openstack-infra | 12:52 | |
| *** kgiusti has joined #openstack-infra | 12:53 | |
| zbr | what is the trick that i would need to do to make zuul point to a specific result html file on a job instead of the default output console? | 12:53 |
| *** mriedem has joined #openstack-infra | 12:54 | |
| frickler | mordred: this one? https://ethercalc.openstack.org/opendev-transition | 12:55 |
| frickler | zbr: you specify a success-url like here https://opendev.org/openstack-infra/zuul-jobs/src/branch/master/zuul.yaml#L193 | 12:58 |
| zbr | frickler: thanks! | 12:58 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Zuul CLI: allow access via REST https://review.openstack.org/636315 | 12:58 |
| *** lseki has joined #openstack-infra | 13:00 | |
| *** aaronsheffield has joined #openstack-infra | 13:01 | |
| *** dklyle has quit IRC | 13:02 | |
| *** udesale has joined #openstack-infra | 13:05 | |
| *** quiquell|rover is now known as quiquell|off | 13:13 | |
| *** bhavikdbavishi has joined #openstack-infra | 13:14 | |
| *** e0ne has joined #openstack-infra | 13:22 | |
| *** Sundar has joined #openstack-infra | 13:25 | |
| gary_perkins | ianw: Thanks for the heads-up regarding https://storyboard.openstack.org/#!/story/2005353#comment-120181 -- It's something we can fix, but we're also in the process of preparing a larger cloud in Packet's EWR1 region that will replace that little one at NRT1 | 13:26 |
| Sundar | Is this the right channel to ask about PTG preparation? | 13:28 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Zuul CLI: allow access via REST https://review.openstack.org/636315 | 13:38 |
| fungi | Sundar: likely #openstack-ptg or #openstack-foundation are good places, but it's more about finding the right people with answers to whatever question you have | 13:39 |
| fungi | sometimes asking on the openstack-discuss mailing list may yield better results | 13:40 |
| dmsimard | mordred, frickler: thanks, I added the ara repos in there | 13:40 |
| Sundar | fungi: Thanks. Makes sense. | 13:41 |
| *** wolverineav has joined #openstack-infra | 13:46 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Remove City Cloud LA1 region https://review.openstack.org/649999 | 13:48 |
| dmsimard | The upload-git-mirror job is finally working \o/ http://zuul.openstack.org/builds?job_name=ara-upload-git-mirror | 13:50 |
| *** wolverineav has quit IRC | 13:50 | |
| *** spsurya has quit IRC | 13:51 | |
| *** bhavikdbavishi has quit IRC | 13:51 | |
| *** altlogbot_3 has quit IRC | 13:55 | |
| fungi | #status log deleted nova instance "mirror01.la1.citycloud.openstack.org" and cinder volume "newvolume" from citycloud-la1 as that region is being taken permanently offline | 13:56 |
| openstackstatus | fungi: finished logging | 13:57 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Zuul CLI: allow access via REST https://review.openstack.org/636315 | 13:58 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Retrieve full list of jobs with details per tenant via API https://review.openstack.org/635714 | 13:58 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Add new merger job to get role definitions from a repository https://review.openstack.org/637181 | 13:58 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Provide roles per tenant via Zuul API https://review.openstack.org/650012 | 13:58 |
| *** e0ne has quit IRC | 13:59 | |
| *** imacdonn has quit IRC | 14:03 | |
| *** e0ne has joined #openstack-infra | 14:05 | |
| *** udesale has quit IRC | 14:06 | |
| *** udesale has joined #openstack-infra | 14:07 | |
| dmsimard | infra-root: now that upload-git-mirror works and the github org transfer script is ready, I'd like to start moving the ara github repos. Would a patch to edit the gerrit replication regex in https://opendev.org/openstack-infra/system-config/src/branch/master/modules/openstack_project/manifests/review.pp#L237-L250 be sufficient or is there something | 14:10 |
| dmsimard | else I could be missing ? | 14:10 |
| *** udesale has quit IRC | 14:11 | |
| fungi | that would need to coincide with a gerrit restart, unless we've recently enabled the feature to allow dynamic reloading of the replication config | 14:11 |
| *** udesale has joined #openstack-infra | 14:11 | |
| dmsimard | https://gerrit.googlesource.com/plugins/replication/+doc/v2.16.7/src/main/resources/Documentation/config.md#replication-configuration mentions "ssh -p 29418 localhost gerrit plugin reload replication" -- I'll read into it a bit | 14:13 |
| *** janki has quit IRC | 14:17 | |
| dmsimard | Querying gerrit shows that the replication plugin is loaded (replication v2.13.12 ENABLED replication.jar) but I'm not sure how to tell if it supports the reload command or not | 14:18 |
| dmsimard | I could try on review-dev perhaps. | 14:19 |
| dmsimard | hmm, my user isn't privileged on review-dev | 14:22 |
| fungi | i believe there's a config option we have to enable for gerrit to allow dynamic plugin reconfiguration | 14:22 |
| *** dpawlik has quit IRC | 14:22 | |
| mordred | yeah. and I think that config option takes a restart :) | 14:23 |
| fungi | plugins.allowRemoteAdmin = true | 14:23 |
| dmsimard | https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#plugins.checkFrequency mentions "If set to 0, automatic plugin reloading is disabled. Administrators may force reloading with gerrit plugin reload." | 14:24 |
| dmsimard | allowRemoteAdmin feels like it's just for installation and enabling/disabling ? | 14:25 |
| fungi | that seems to say the same on our version too https://review.openstack.org/Documentation/config-gerrit.html#plugins.checkFrequency | 14:25 |
| fungi | yeah, allowRemoteAdmin looks like it might be specifically for loading updated plugins, not just reconfiguring them | 14:26 |
| fungi | at any rate, in the past we've observed that changing the replication configuration doesn't get picked up until a gerrit restart | 14:26 |
| openstackgerrit | Gary Perkins proposed openstack-infra/system-config master: Changed region of arm64ci from nrt1 to ewr1 https://review.openstack.org/650021 | 14:27 |
| fungi | but it's worth testing again on review-dev, i suppose | 14:27 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Zuul CLI: allow access via REST https://review.openstack.org/636315 | 14:30 |
| dmsimard | fungi: I can do the legwork to double check the behavior but I'm unprivileged on the review-dev gerrit. Would you be able to add me to the right group ? | 14:31 |
| fungi | sure, just a sec | 14:31 |
| fungi | dmsimard: you're now a member of https://review-dev.openstack.org/#/admin/groups/Administrators | 14:33 |
| dmsimard | thanks ! | 14:33 |
| fungi | thank you for testing | 14:33 |
| dmsimard | well there you go: fatal: remote plugin administration is disabled | 14:34 |
| dmsimard | so we probably need to enable allowRemoteAdmin first :P | 14:34 |
| * dmsimard looks | 14:34 | |
| fungi | the other thing to work out is whether changing the config gets magically noticed by gerrit after some period | 14:35 |
| fungi | it's possible the last time we tested it was on an older release of gerrit when plugins were still a relatively new concept | 14:35 |
| dmsimard | yes, that seems like it is the plugins.checkFrequency configuration | 14:35 |
| fungi | which defaults to 1 minute | 14:37 |
| fungi | though again the description sounds like it's about loading, unloading and updating plugins themselves not their configs | 14:37 |
| fungi | though maybe if it doesn't automatically reload the changed config, touching the plugin jar so it has a more recent timestamp will cause gerrit to reload the plugin which will in turn reload the config for that plugin? | 14:39 |
| fungi | we see similar behavior with the javascript and the site header file | 14:40 |
| clarkb | re allow remote admin I didnt enable that lasttime we changed replication config (for gitea01-8) because it allows far more than config updates and needs more consideration (testing and understanding of attack surface) | 14:40 |
| dmsimard | clarkb: that is my understanding as well | 14:40 |
| fungi | (it won't automatically reload changes to the javascript files but will reload them if the site header gets a new timestamp) | 14:40 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Add new merger job to get role definitions from a repository https://review.openstack.org/637181 | 14:45 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Provide roles per tenant via Zuul API https://review.openstack.org/650012 | 14:45 |
| *** e0ne has quit IRC | 14:46 | |
| dmsimard | clarkb: do you know if there a concept of non-remote admin? :) | 14:50 |
| dmsimard | like, locally instead of over ssh or something | 14:50 |
| clarkb | not that I have seen other than restrting the service | 14:51 |
| *** e0ne has joined #openstack-infra | 14:52 | |
| *** Sundar has quit IRC | 14:53 | |
| *** yamamoto has quit IRC | 14:54 | |
| *** yamamoto has joined #openstack-infra | 14:55 | |
| dmsimard | ok I think what we want is "gerrit.autoReload" -> If true, automatically reloads replication destinations and settings after replication.config file is updated, without the need to restart the replication plugin. When the reload takes place, pending replication events based on old settings are discarded. By default, false. | 14:55 |
| dmsimard | From https://gerrit.googlesource.com/plugins/replication/+doc/master/src/main/resources/Documentation/config.md#file-1 | 14:55 |
| mordred | fungi: if I want to rename an official openstack project as part of the rename, who do I need to run that through? | 14:56 |
| fungi | mordred: likely want to at least stage a related governance change so the tc is aware | 14:56 |
| dmsimard | but enabling autoReload will require a restart | 14:56 |
| mordred | (seems like a good time to rename shade from openstack-infra/shade to openstack/shade) | 14:56 |
| jamespage | mordred, clarkb, frickler: uploaded prepared for ubuntu-sru team review under https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1822842 | 14:57 |
| openstack | Launchpad bug 1822842 in python-pip (Ubuntu Cosmic) "pip fails to install package from source when extra-index-url returns 404" [Medium,Triaged] | 14:57 |
| mordred | has never been worth the time before - and still isn't since it's largely vestigal now, but since the urls are changing anyway) | 14:57 |
| fungi | mordred: oh, for that we might just coordinate with jroll as it's just changing the namespace | 14:57 |
| * mordred hands jamespage a pie | 14:57 | |
| *** e0ne has quit IRC | 14:57 | |
| mordred | fungi: cool | 14:57 |
| jamespage | PPA with fixes in the interim | 14:57 |
| jamespage | https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3690/+packages | 14:57 |
| mordred | jroll: hey! we should rename openstack-infra/shade to openstack/shade during the rename window | 14:57 |
| dmsimard | mordred: is shade still a thing ? I thought it had merged to openstacksdk | 14:58 |
| fungi | mordred: maybe part of jroll's plan could be to renamespace any official non-infra projects which are in the openstack-infra namespace into openstack, and similarly renamespace any infra team repos out of the openstack namespace | 14:58 |
| mordred | dmsimard: the repo is still live for backwards compat reasons | 14:58 |
| dmsimard | makes sense | 14:58 |
| mordred | fungi: that would make a lot of sense | 14:58 |
| mordred | fungi: oh - also - what zuul tenant should non-official non-openstack things in opendev use? | 14:59 |
| fungi | i think the release team may have one of those as well | 14:59 |
| clarkb | dmsimard: is that setting documented on our gerrit? | 14:59 |
| mordred | fungi: like - let me phrase this with a specific example | 14:59 |
| fungi | mordred: right, the openstack tc suggested something like "unknown" as the catch-all namespace, but noted that it's ultimately up to the opendev sysadmins to choose since things which aren't part of openstack aren't really their jurisdiction | 14:59 |
| *** yamamoto has quit IRC | 15:00 | |
| mordred | fungi: if I wanted to import my inaugust repos - what zuul tenant would be the right place to put them? | 15:00 |
| openstackgerrit | Gary Perkins proposed openstack-infra/system-config master: Changed region of arm64ci from nrt1 to ewr1 Also updated the URLs https://review.openstack.org/650021 | 15:00 |
| mordred | fungi: yeah - this is really an opendev question for us I'd imagine | 15:00 |
| fungi | mordred: oh, in that case you can pick a new namespace. i know how you love naming things | 15:00 |
| jroll | mordred: see line 27 https://review.openstack.org/#/c/645601/2/resolutions/20190322-namespace-unofficial-projects.rst | 15:00 |
| mordred | fungi: oh - not just namespace- but which zuul tenant would gate teh patches? | 15:01 |
| jroll | mordred: current plan is anything in openstack-infra/dev that is staying openstack, is going to the openstack namespace | 15:01 |
| mordred | obviously not the openstack one | 15:01 |
| mordred | jroll: \o/ | 15:01 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Retrieve full list of jobs with details per tenant via API https://review.openstack.org/635714 | 15:01 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Add new merger job to get role definitions from a repository https://review.openstack.org/637181 | 15:01 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Provide roles per tenant via Zuul API https://review.openstack.org/650012 | 15:01 |
| jroll | :) | 15:01 |
| fungi | the bigger question was what to do about all the non-openstack repos currently in the openstack namespace where nobody steps forward to pick custom namespaces for them. the tc is basically saying "you don't have to go home but you can't stay here" | 15:01 |
| dmsimard | clarkb: https://review.openstack.org/Documentation/config-plugins.html#replication links to the upstream docs but I'm trying on review-dev | 15:02 |
| mordred | fungi: I think it'll be the same question I have as for like gertty | 15:02 |
| fungi | mordred: oh! right, i don't know the zuul tenant question | 15:02 |
| fungi | er, rather, don't know the answer to it | 15:02 |
| mordred | yeah | 15:02 |
| mordred | I think we decided the opendev tenant was for opendev the service - not as a catchall for arbitrary unaffiliated projects | 15:02 |
| clarkb | this is the why people like the idea of stackforge I think | 15:04 |
| mordred | yeah - but even stackforge wouldn't be right | 15:04 |
| clarkb | no but its 80% good enough | 15:04 |
| dmsimard | clarkb: autoReload works on review-dev but it requires a restart first | 15:05 |
| mordred | because gertty and inaugust aren't "possible openstack projects but arent' official" - they're just other open source projects but happen to be small enough that their own zuul tenant maybe seems like overkill | 15:05 |
| mordred | but maybe I'm stuck too much on that second thing and maybe there just should be another zuul tenant | 15:05 |
| clarkb | mordred: I dont think that washow we definied stackforge | 15:05 |
| fungi | we didn't especially define stackforge | 15:06 |
| fungi | we mostly only undefined stackforge | 15:06 |
| mordred | sure - I'm just sauing - if there was an opendev.org/inaugust/inaugust.com repo - why would it make any sense to go to zuul.opendev.org/t/stackforge/status to see jobs running? | 15:06 |
| mordred | from an opendev service perspective | 15:07 |
| clarkb | mordred: becayse for most people theyll justlook at the gerrit comment | 15:07 |
| mordred | I would think zuul.opendev.org/t/opendev/status or zuul.opendev.org/t/inaugust/status both make more sense from a "make opendev a nice service" POV | 15:07 |
| clarkb | its a level of detail that doesnt pop out to most I expect | 15:08 |
| clarkb | mordred: yup but we also arent gonna self provision zuul tenants yet | 15:08 |
| mordred | what do you mean by self provision? | 15:08 |
| clarkb | decoupling us from the process of spinning up a new tenant | 15:09 |
| clarkb | it is a fair bit of work | 15:09 |
| mordred | what is it other than just lines in the zuul config file? (which have to be added for repos anyway) | 15:10 |
| *** jamesdenton has joined #openstack-infra | 15:10 | |
| clarkb | creating basejob repos, writing basejobs that function | 15:10 |
| clarkb | we are sort of half way there with the common basejob repo now | 15:10 |
| mordred | yah. so - I think I have an idea ... | 15:10 |
| mordred | yeah | 15:10 |
| clarkb | but iirc all the trnants customize on top of that | 15:11 |
| mordred | like - actually, the opendev/project-config repo would be perfectly fine project-config repo for most small projects | 15:11 |
| mordred | so instead of re-using the opendev _tenant_ - we could have smaller tenants just reuse the opendev project-config for base jobs and pipelines and whantot | 15:11 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/puppet-gerrit master: Add support for enabling replication configuration autoReload https://review.openstack.org/650049 | 15:12 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/system-config master: Enable gerrit replication.autoReload for review-dev https://review.openstack.org/650050 | 15:12 |
| mordred | anyway - not urgent - mostly was just thinking about it over the last week while working on slides | 15:12 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/system-config master: Enable gerrit replication.autoReload for review https://review.openstack.org/650051 | 15:14 |
| clarkb | ya that may be one approach where base config bakes in a little more (like pipelines) and other tenants can ignore the bitsthey dont want. Though I know zuul wamts different check pipeline than openstack so still some details to sort out | 15:14 |
| mordred | totally | 15:15 |
| mordred | but yeah - it should be reasonably easy to make a generic batteries-included config repo *for opendev* that could be re-used by anyone who didn't have explicit opinions | 15:15 |
| *** ramishra_ has quit IRC | 15:20 | |
| clarkb | dmsimard: re restart first that is expected, we have to load that config into gerrit via restart then after it should work. So basically if the plugin config changes it will notice the file update and reload its config without restarting gerrit? Seems like a good option (and far safer than remote admin) | 15:24 |
| *** dklyle has joined #openstack-infra | 15:27 | |
| *** udesale has quit IRC | 15:32 | |
| corvus | mordred, fungi, clarkb: i think it would be fine for the opendev zuul tenant to host ci for unaffiliated projects | 15:33 |
| corvus | (i do believe we agreed that the opendev *namespace* should only be for opendev itself) | 15:33 |
| fungi | yeah, so perhaps the opendev zuul tenant is tied to the opendev namespace, the unknown namespace and whatever other namespaces want to use it? | 15:34 |
| fungi | or does the unknown namespace continue using the openstack tenant for simplicity of migration? | 15:34 |
| *** pgaxatte has quit IRC | 15:36 | |
| clarkb | I'm guessing openstack would prefer they didn't but its probably a slower migration that the 4/19 change | 15:36 |
| corvus | i was imagining that we would not move projects between zuul tenants on 4/19, and that would follow on later | 15:36 |
| clarkb | ++ | 15:37 |
| fungi | wfm | 15:37 |
| clarkb | that was my understanding too | 15:37 |
| clarkb | fungi: re citycloud la1 were you going to respond to daniel or maybe you did and I wasn't cc'd? | 15:37 |
| fungi | i hadn't gotten to that mailbox yet. i responded to pierre's e-mail to infra-root though | 15:37 |
| fungi | and cc'd the infra-root alias on my reply | 15:38 |
| clarkb | oh I haven't gotten to that email yet | 15:38 |
| clarkb | daniel sent one to our openstack.org addresses too | 15:38 |
| *** kopecmartin is now known as kopecmartin|off | 15:41 | |
| *** gyee has joined #openstack-infra | 15:43 | |
| *** e0ne has joined #openstack-infra | 15:46 | |
| clarkb | infra-root if I can get a second reviewer on https://review.openstack.org/#/c/647875/ I;ll watch that go in and make sure the kata lists server is happy after the puppet-4 upgrade | 15:47 |
| fungi | approved, thanks! | 15:47 |
| *** e0ne has quit IRC | 15:48 | |
| *** e0ne has joined #openstack-infra | 15:48 | |
| *** e0ne has quit IRC | 15:48 | |
| openstackgerrit | Sean McGinnis proposed openstack-dev/hacking master: Fix python 3.6 escape char warnings in strings https://review.openstack.org/650056 | 15:50 |
| *** sshnaidm_ has joined #openstack-infra | 15:52 | |
| *** sshnaidm has quit IRC | 15:54 | |
| *** efried is now known as efried_mtg | 15:54 | |
| *** bhavikdbavishi has joined #openstack-infra | 15:55 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove City Cloud LA1 region https://review.openstack.org/649999 | 15:57 |
| clarkb | fungi: ^ fyi | 15:58 |
| *** jamesmcarthur has joined #openstack-infra | 15:58 | |
| fungi | thanks! | 15:59 |
| clarkb | hrm would anyone be offended if I removed hacking from our gerritbot list? | 15:59 |
| fungi | feel free, as far as i'm concerned | 15:59 |
| clarkb | I like to use theose bot events as reminders for things that I may want to take action on and I don't think we are actively working on hacking or even use it in our projects at this point | 16:00 |
| mordred | clarkb: ++ | 16:01 |
| *** e0ne has joined #openstack-infra | 16:05 | |
| clarkb | dmsimard: comment on https://review.openstack.org/#/c/650050/1 | 16:06 |
| *** ykarel is now known as ykarel|away | 16:07 | |
| *** diablo_rojo has joined #openstack-infra | 16:10 | |
| *** e0ne has quit IRC | 16:12 | |
| openstackgerrit | Merged openstack-infra/system-config master: Upgrade lists.katacontainers.io to puppet 4 https://review.openstack.org/647875 | 16:17 |
| *** ironfoot has joined #openstack-infra | 16:19 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Identify *POST* timeout failures individually https://review.openstack.org/625573 | 16:20 |
| *** tbachman has quit IRC | 16:22 | |
| *** dpawlik has joined #openstack-infra | 16:25 | |
| *** tbachman has joined #openstack-infra | 16:25 | |
| *** ginopc has quit IRC | 16:30 | |
| gmann | clarkb: fungi ianw frickler can you review these for stein release things- https://review.openstack.org/#/c/647899/ and this(did recheck) https://review.openstack.org/#/c/647912/ | 16:32 |
| *** igordc has joined #openstack-infra | 16:33 | |
| *** yolanda has quit IRC | 16:33 | |
| *** jpich has quit IRC | 16:34 | |
| *** dpawlik has quit IRC | 16:38 | |
| *** tosky has quit IRC | 16:43 | |
| *** rpittau is now known as rpittau|afk | 16:44 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/system-config master: Enable gerrit replication.autoReload for review-dev https://review.openstack.org/650050 | 16:48 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/system-config master: Enable gerrit replication.autoReload for review https://review.openstack.org/650051 | 16:48 |
| dmsimard | clarkb: fixed, thanks | 16:51 |
| *** tbachman has quit IRC | 16:51 | |
| clarkb | gmann: see comment on https://review.openstack.org/#/c/647912/1 | 16:51 |
| clarkb | dmsimard: yup +2 | 16:51 |
| clarkb | dmsimard: maybe wip 650051 until we are happy with the review-dev results? | 16:52 |
| dmsimard | clarkb: that was the intent, I'll -W | 16:52 |
| dmsimard | I also suppose we need to babysit the patch since puppet will restart gerrit after modifying the replication config ? | 16:52 |
| dmsimard | well, for 650051 that is | 16:53 |
| *** kjackal has quit IRC | 16:53 | |
| dmsimard | oh, actually, 650051 would also restart gerrit because it'll add autoReload = false :/ | 16:53 |
| clarkb | no we don't auto restart gerrit | 16:53 |
| dmsimard | oh, good then | 16:53 |
| gmann | clarkb: yeah, i think testing ocata only is needed there. i will update the patch to remove other stable zuulv3 native jobs. | 16:53 |
| gmann | thanks | 16:53 |
| dmsimard | less trouble | 16:53 |
| clarkb | so these changes should be fine as long as we sequence them | 16:53 |
| clarkb | gmann: thank you! | 16:54 |
| clarkb | gmann: we may also want to add a representative legacy job back in? | 16:54 |
| *** rkukura has quit IRC | 16:55 | |
| clarkb | I think grenade is already covered but for tempest stuff we don't have good coverage there anymore | 16:55 |
| clarkb | (so your change ot branches should be fine) | 16:55 |
| gmann | for testing coverage of other legacy jobs still present in project side ? | 16:56 |
| clarkb | ya | 16:56 |
| clarkb | I'm not sure that ocata is representative of the others | 16:56 |
| clarkb | thought maybe it is close enough for what devstack-gate does | 16:56 |
| gmann | yeah but there might be lot of other things or new testing for each stable/ jobs. | 16:57 |
| clarkb | ya I'm trying to balance the idea that this is legacy code we should be getting off of against breaking the jobs that haven't gotten off of it yet | 16:58 |
| *** electrofelix has quit IRC | 16:58 | |
| clarkb | grenade is the big one and we are covered there so maybe that is good enough | 16:58 |
| gmann | clarkb: how about keeping the stable/latest(stable/stein in this release case) legacy job testing only. | 16:58 |
| *** ccamacho has quit IRC | 16:58 | |
| gmann | that can be most close | 16:59 |
| clarkb | that works for me if we haev those jobs | 16:59 |
| *** tbachman has joined #openstack-infra | 16:59 | |
| gmann | we do not have :) i need to create one for devdtack-gate | 16:59 |
| gmann | since pike tempest-full legacy is gone | 16:59 |
| * gmann hoping to have community goal of moving legacy jobs to zuulv3 native soon | 17:00 | |
| *** dtantsur is now known as dtantsur|afk | 17:00 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Remove TaskManager and just use keystoneauth https://review.openstack.org/640643 | 17:03 |
| *** ijw has joined #openstack-infra | 17:07 | |
| *** jpena is now known as jpena|off | 17:08 | |
| *** fresta has quit IRC | 17:13 | |
| *** fresta has joined #openstack-infra | 17:13 | |
| *** gfidente has quit IRC | 17:16 | |
| *** derekh has quit IRC | 17:18 | |
| *** jamesmcarthur has quit IRC | 17:19 | |
| *** diablo_rojo has quit IRC | 17:21 | |
| *** rlandy is now known as rlandy|ruck | 17:27 | |
| *** imacdonn has joined #openstack-infra | 17:27 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Simplify and clean up infra gerritbot irc channel configs https://review.openstack.org/650083 | 17:27 |
| clarkb | ok there is the gerritbot config change | 17:27 |
| anteaya | clarkb: that change only affects the opendev channel, yes? | 17:32 |
| clarkb | anteaya: and this channel | 17:32 |
| anteaya | ah there's the rest of it | 17:33 |
| anteaya | thanks | 17:34 |
| openstackgerrit | Brendan proposed openstack-infra/zuul master: gerrit: Add some quoting in 'gerrit query' commands https://review.openstack.org/649879 | 17:34 |
| *** shachar has joined #openstack-infra | 17:34 | |
| *** snapiri has quit IRC | 17:37 | |
| *** spsurya has joined #openstack-infra | 17:37 | |
| clarkb | puppet-4 looks happy on lists.katacontainers.io | 17:38 |
| clarkb | complains about lack of group var but that is expected since this server isn't in an ansible/hiera group | 17:38 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Proposed spec: tenant-scoped admin web API https://review.openstack.org/562321 | 17:40 |
| *** gmann is now known as gmann_afk | 17:44 | |
| *** wolverineav has joined #openstack-infra | 17:45 | |
| *** wolverineav has quit IRC | 17:45 | |
| *** wolverineav has joined #openstack-infra | 17:45 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP: Fix SQL error when querying for tuples https://review.openstack.org/650086 | 17:46 |
| *** rkukura has joined #openstack-infra | 17:47 | |
| *** sshnaidm_ is now known as sshnaidm|off | 17:50 | |
| fungi | i wonder if we should artificially lower our max-servers in limestone-regionone to reduce resource contention. seeing lots of jobs failing there because of job timeouts as well as individual test timeouts | 17:53 |
| *** diablo_rojo has joined #openstack-infra | 17:56 | |
| *** diablo_rojo_ has joined #openstack-infra | 17:56 | |
| *** diablo_rojo_ has quit IRC | 17:56 | |
| *** diablo_rojo has quit IRC | 17:56 | |
| *** diablo_rojo has joined #openstack-infra | 17:57 | |
| *** wolverineav has quit IRC | 18:01 | |
| *** yamamoto has joined #openstack-infra | 18:01 | |
| *** wolverineav has joined #openstack-infra | 18:02 | |
| *** wolverineav has quit IRC | 18:04 | |
| *** wolverineav has joined #openstack-infra | 18:04 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Fix debug log message https://review.openstack.org/650088 | 18:05 |
| *** derekh has joined #openstack-infra | 18:09 | |
| *** ijw has quit IRC | 18:09 | |
| *** derekh has quit IRC | 18:10 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Update dib stats https://review.openstack.org/638533 | 18:14 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/irc-meetings master: First Contact SIG update https://review.openstack.org/648782 | 18:19 |
| *** bhavikdbavishi has quit IRC | 18:22 | |
| *** psachin has quit IRC | 18:28 | |
| *** owalsh has quit IRC | 18:34 | |
| *** owalsh_ has joined #openstack-infra | 18:34 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix SQL error when querying for tuples https://review.openstack.org/650086 | 18:36 |
| fungi | clarkb: i'm replying to daniel's e-mail now | 18:39 |
| fungi | will cc ypi | 18:39 |
| fungi | er, will cc you | 18:39 |
| *** owalsh_ has quit IRC | 18:39 | |
| clarkb | thanks | 18:39 |
| *** owalsh has joined #openstack-infra | 18:41 | |
| *** rh-jelabarre has quit IRC | 18:43 | |
| *** rh-jelabarre has joined #openstack-infra | 18:44 | |
| *** tbachman has quit IRC | 18:44 | |
| openstackgerrit | Merged openstack-infra/nodepool master: doc: add provider driver development instructions https://review.openstack.org/639595 | 18:47 |
| *** kjackal has joined #openstack-infra | 18:50 | |
| *** eharney has quit IRC | 18:55 | |
| *** eharney has joined #openstack-infra | 18:58 | |
| *** wolverineav has quit IRC | 19:02 | |
| *** wolverineav has joined #openstack-infra | 19:03 | |
| *** wolverineav has quit IRC | 19:07 | |
| *** tbachman has joined #openstack-infra | 19:08 | |
| *** rkukura has quit IRC | 19:14 | |
| *** rkukura has joined #openstack-infra | 19:23 | |
| *** owalsh_ has joined #openstack-infra | 19:25 | |
| *** owalsh has quit IRC | 19:25 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix debug log message https://review.openstack.org/650088 | 19:29 |
| *** tosky has joined #openstack-infra | 19:33 | |
| *** wolverineav has joined #openstack-infra | 19:38 | |
| *** tbachman has quit IRC | 19:38 | |
| *** owalsh has joined #openstack-infra | 19:39 | |
| *** owalsh_ has quit IRC | 19:40 | |
| *** wolverineav has quit IRC | 19:42 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Whitelist pageup and down keys in xterm https://review.openstack.org/649694 | 19:44 |
| *** artom has quit IRC | 19:46 | |
| *** rkukura has quit IRC | 19:46 | |
| *** efried_mtg is now known as efried | 19:46 | |
| *** gmann_afk is now known as gmann | 19:49 | |
| *** rkukura has joined #openstack-infra | 19:49 | |
| *** _erlon_ has joined #openstack-infra | 19:56 | |
| *** wolverineav has joined #openstack-infra | 19:58 | |
| logan- | fungi: i'll look into it. that cloud went p->q->r a few weeks ago so I wonder if there are performance impacts or a config regression or something. | 20:02 |
| *** wolverineav has quit IRC | 20:03 | |
| fungi | if nothing else, might be good to know where we might be running up against hardware resource constraints | 20:03 |
| fungi | and thanks as always! | 20:03 |
| *** wolverineav has joined #openstack-infra | 20:04 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix SQL error when querying for tuples https://review.openstack.org/650086 | 20:05 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix SQL error when querying for tuples https://review.openstack.org/650086 | 20:08 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix SQL error when querying for tuples https://review.openstack.org/650086 | 20:08 |
| gmann | clarkb: fungi any of you can take care of lxkong request of adding in trove stable core team. bionic migration is also pending due to that. http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004543.html | 20:08 |
| fungi | gmann: those groups are managed by stable-maint-core | 20:10 |
| gmann | ohk. | 20:10 |
| fungi | heading out for a walk but should return shortly | 20:11 |
| gmann | mriedem: tonyb ^^ | 20:11 |
| mriedem | gmann: i replied | 20:12 |
| gmann | mriedem: there are more patches stuck on trove stable/stein and bionic migration depends on those. | 20:14 |
| gmann | is it ok to add him as core there as per his original request ? | 20:15 |
| gmann | he is trove current PTL | 20:15 |
| mriedem | being PTL on master doesn't mean a person knows stable branch review policy | 20:17 |
| mriedem | so it probably depends on if he's already been doing stable reviews and knows the process | 20:17 |
| mriedem | what's appropriate, what's not, etc | 20:17 |
| mriedem | having said that, i don't really want to have to do the homework to find out if he is for trove which is on life support anyway it seems | 20:17 |
| ianw | clarkb: could you look at https://review.openstack.org/#/c/649773/ just to confirm that was your discussion (retire linaro cn1 cloud) | 20:19 |
| gmann | ok, make sense. can you check the patches i mentioned in that ML thread which holding the bionic migration. and there are no trove active core present except him. | 20:19 |
| *** jamesmcarthur has joined #openstack-infra | 20:21 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for pymysql.err.InterfaceError bug 1823251 https://review.openstack.org/650136 | 20:21 |
| openstack | bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [Medium,Confirmed] https://launchpad.net/bugs/1823251 | 20:21 |
| clarkb | ianw yup just finishing up lunch then can look | 20:22 |
| *** tbachman has joined #openstack-infra | 20:22 | |
| ianw | clarkb: thanks; it looks like we have some ready nodes in linaro london now, that didn't seem to be happening yesterday so i'll try the wheel build stuff again shortly | 20:25 |
| *** jamesmcarthur has quit IRC | 20:25 | |
| *** pcaruana has quit IRC | 20:29 | |
| *** pcaruana has joined #openstack-infra | 20:30 | |
| *** dpawlik has joined #openstack-infra | 20:31 | |
| openstackgerrit | Merged openstack-infra/zuul master: Increase timeout of zuul-quick-start https://review.openstack.org/649384 | 20:35 |
| *** dpawlik has quit IRC | 20:35 | |
| *** raissa has joined #openstack-infra | 20:38 | |
| *** diablo_rojo has quit IRC | 20:43 | |
| *** kgiusti has quit IRC | 20:45 | |
| clarkb | ianw: ya https://review.openstack.org/#/c/649773/2 is what I think we want. But we'll have to manually clear out any zk records for it? | 20:45 |
| *** wolverineav has quit IRC | 20:46 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for pymysql.err.InterfaceError bug 1823251 https://review.openstack.org/650136 | 20:47 |
| openstack | bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed] https://launchpad.net/bugs/1823251 | 20:47 |
| *** wolverineav has joined #openstack-infra | 20:47 | |
| clarkb | I'm listing things now to double check | 20:47 |
| clarkb | ya there are a bunch of linaro-cn1 images listed but failed. My hunch is we'll have to manually clear those out | 20:48 |
| clarkb | I went ahead and approved the chagne and we can figure out zk cleanup as a followup | 20:49 |
| *** wolverineav has quit IRC | 20:51 | |
| *** wolverineav has joined #openstack-infra | 20:51 | |
| *** spsurya has quit IRC | 20:54 | |
| *** sshnaidm|off has quit IRC | 20:55 | |
| clarkb | logan-: fungi: I don't have evidence for this at all but potentially better spectre/meltdown type mitigations on newer platform? Assuming the OS upgraded as well | 20:57 |
| *** slaweq has quit IRC | 20:58 | |
| *** sshnaidm|off has joined #openstack-infra | 20:58 | |
| *** mhu has quit IRC | 20:58 | |
| *** otherwiseguy has quit IRC | 20:59 | |
| *** jrist has quit IRC | 20:59 | |
| logan- | the host is still xenial+HWE kernel. but maybe some impact from that by pulling in the rocky UCA vs pike UCA for libvirt/qemu packages. side note -- config issue addressed https://review.openstack.org/#/c/650135/. I deployed it manually so we should see all new nodes launching with writeback now. confirmed on a couple launches already. | 21:00 |
| *** jrist has joined #openstack-infra | 21:01 | |
| *** wolverineav has quit IRC | 21:03 | |
| *** mhu has joined #openstack-infra | 21:04 | |
| *** otherwiseguy has joined #openstack-infra | 21:04 | |
| *** dpawlik has joined #openstack-infra | 21:04 | |
| *** wolverineav has joined #openstack-infra | 21:05 | |
| openstackgerrit | Merged openstack-infra/project-config master: Retire linaro-cn1 https://review.openstack.org/649773 | 21:08 |
| clarkb | logan-: oh that writeback setting will probably make a big difference | 21:11 |
| logan- | yeah, i think last time I manually fixed it and forgot to push the patch :( | 21:11 |
| *** kjackal has quit IRC | 21:12 | |
| clarkb | ianw https://review.openstack.org/#/c/647899/ is a semi important change for grenade if you can take a quick look today | 21:12 |
| clarkb | sets up testing for stein properly | 21:12 |
| *** michael-beaver has joined #openstack-infra | 21:18 | |
| *** rlandy|ruck is now known as rlandy|ruck|brb | 21:21 | |
| *** slaweq has joined #openstack-infra | 21:25 | |
| *** bobh has joined #openstack-infra | 21:26 | |
| *** bobh has quit IRC | 21:27 | |
| *** slaweq has quit IRC | 21:30 | |
| fungi | logan-: ooh, thanks! i agree the symptoms looks outwardly like they could be i/o bottlenecks, so writethrough would potentially have explained a lot of it | 21:31 |
| openstackgerrit | Merged openstack-infra/zuul master: Use opendev docker jobs https://review.openstack.org/649466 | 21:31 |
| *** rlandy|ruck|brb is now known as rlandy|ruck | 21:39 | |
| *** slaweq has joined #openstack-infra | 21:47 | |
| *** raissa has quit IRC | 21:51 | |
| *** slaweq has quit IRC | 21:51 | |
| *** pcaruana has quit IRC | 21:51 | |
| *** slaweq has joined #openstack-infra | 21:52 | |
| mriedem | gmann: done on those trove stable changes, and rebased them into a series | 21:52 |
| *** michael-beaver has left #openstack-infra | 21:52 | |
| clarkb | mriedem: as a sanity check e-r has been happy since we made those improvements ya? | 21:53 |
| gmann | mriedem: nice, thanks a lot :) | 21:53 |
| *** dpawlik has quit IRC | 21:53 | |
| mriedem | clarkb: the indexes are up to date | 21:54 |
| mriedem | Data Last Updated: Thu Apr 04 2019 16:30:02 GMT-0500 (Central Daylight Time) | 21:54 |
| mriedem | clarkb: i haven't seen e-r bot comment on a failed change in a long time though | 21:54 |
| clarkb | hrm | 21:55 |
| clarkb | its possible we haven't completely fixed the queue backup behavior it has there | 21:55 |
| clarkb | I can restart it now and see if that comes back, if it does there is a good chance it is still backing up that queue | 21:55 |
| *** auristor has quit IRC | 21:57 | |
| *** slaweq has quit IRC | 21:57 | |
| *** bnemec has quit IRC | 22:01 | |
| *** mattw4 has joined #openstack-infra | 22:04 | |
| *** dpawlik has joined #openstack-infra | 22:05 | |
| *** rascasoft has quit IRC | 22:05 | |
| *** rascasoft has joined #openstack-infra | 22:06 | |
| ianw | hrm, we have ready arm64 nodes, but https://review.openstack.org/552700 really doesn't seem to want to use one | 22:07 |
| *** dpawlik has quit IRC | 22:09 | |
| clarkb | ok time to review letsencrypt changes | 22:12 |
| *** auristor has joined #openstack-infra | 22:14 | |
| clarkb | ianw: every host verifying a letsencrypt cert needs the CNAME from opendev.org to acme.opendev.org? (we can't do a global domain level redirect I guess). Also did the setup of acme.opendev.org go through ansible we just don't manage the zone file for it? | 22:14 |
| ianw | clarkb: to your second question first; yes acme.opendev.org is an "unmanaged domain" per https://review.openstack.org/#/c/641155/ ... so if there's no zone file there, ansible will put a blank one in, otherwise it will leave it alone. this just ensures it's always valid | 22:16 |
| ianw | for "host.opendev.org" LE will be looking for _acme-challenge.host.opendev.org ; so yeah, each host needs to point that at acme.opendev.org | 22:17 |
| ianw | clarkb: sample records currently are -> https://git.openstack.org/cgit/openstack-infra/zone-opendev.org/tree/zones/opendev.org/zone.db#n61 | 22:18 |
| ianw | that was what i used to validate the cert generation | 22:19 |
| clarkb | --yes-I-know-dns-manual-mode-enough-go-ahead-please made me laugh a little | 22:19 |
| ianw | yeah, acme.sh really seems to not want you to do this | 22:20 |
| ianw | i imagine though it's probably been a constant source of bugs from people not quite understanding what's going on | 22:20 |
| *** diablo_rojo has joined #openstack-infra | 22:20 | |
| ianw | it is quite geared towards dns-as-a-service implementations | 22:20 |
| ianw | ultimately, i guess our dns-as-a-service is ansible :) | 22:21 |
| *** rcernin has joined #openstack-infra | 22:23 | |
| ianw | (that said i still think it's the best tool for the job. having no dependencies but standard unixy tools is great, ultimately all it's really doing is a few wgets and then openssl magic) | 22:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Use the buildset registry in zuul-quick-start https://review.openstack.org/649467 | 22:27 |
| *** diablo_rojo has quit IRC | 22:28 | |
| *** diablo_rojo has joined #openstack-infra | 22:28 | |
| *** diablo_rojo has quit IRC | 22:31 | |
| *** whoami-rajat has quit IRC | 22:32 | |
| *** ykarel|away has quit IRC | 22:33 | |
| tonyb | gmann: I'll look at it I got side tracked and lost that item on my todo list | 22:34 |
| *** igordc has quit IRC | 22:34 | |
| *** diablo_rojo has joined #openstack-infra | 22:34 | |
| *** wolverineav has quit IRC | 22:35 | |
| *** harlowja has quit IRC | 22:36 | |
| *** ykarel has joined #openstack-infra | 22:36 | |
| clarkb | ianw: ok reviewed. Looks good though I had some comments (but left a +2 as it seems to be working as is) | 22:39 |
| *** mrhillsman_bbiab is now known as mrhillsman | 22:40 | |
| *** aaronsheffield has quit IRC | 22:40 | |
| ianw | clarkb: thanks, will respond. just quickly the renewal does require re-authenticating a new txt record, it's more or less like new issue | 22:42 |
| clarkb | ianw: do we need to grep out the txt data there then? | 22:42 |
| clarkb | (where I left my comment asking the question) | 22:42 |
| ianw | clarkb: and yeah, you're right on only using one value of the tuple. in prior changes it was referencing it, i thought maybe it may be useful in the future, i think i put in comments sort of noting that | 22:43 |
| ianw | clarkb: no, so we always do issue -- only if there's no cert or it needs renewal do we get the TXT record outputs. then we do renew -- the ansible should skip doing this unless there is actually something to renew | 22:44 |
| clarkb | ah ya I see that now only when acme_txt_required's length is > 0 do we run the renew command | 22:46 |
| corvus | ianw: http://paste.openstack.org/show/748908/ | 22:46 |
| ianw | corvus: hrm, i was looking at node request 300-0002639163 | 22:47 |
| corvus | ianw: you didn't say so :) | 22:47 |
| ianw | corvus: oh, i dunno, i'm flying blind here just trying to figure out why this job isn't launching :) | 22:48 |
| corvus | ianw: node 0004768871 from that request has the same error | 22:48 |
| corvus | unfortunately i don't know what "openstack.exceptions.SDKException: Error in creating the server" means :( | 22:49 |
| ianw | i think it's timeouts? | 22:49 |
| clarkb | could be fallout from the sdk upgrade thing that just happened maybe? | 22:49 |
| ianw | the weird thing is, before we started this, we had an active node | 22:50 |
| corvus | ianw: it's 6 seconds after start of creation | 22:50 |
| clarkb | ianw: last question about letsencrypt (I hope) acme.sh will only emit txt records on issue if the cert we already have does not exist or is at least 60 days old? (want to make sure I understand the mechanism that ensure we renew within 90 days) | 22:51 |
| *** diablo_rojo has quit IRC | 22:51 | |
| ianw | clarkb: yep, you run acme.sh and it looks at the cert and says "this doesn't need to be renewed" | 22:52 |
| clarkb | ya looks like --days is the flag to set if we want to change the default of 60 days | 22:52 |
| ianw | clarkb: specifically in the logs it says "[Tue Apr 2 07:39:14 AEDT 2019] Skip, Next renewal time is: Fri May 31 05:26:48 UTC 2019" | 22:52 |
| clarkb | then because we run this cron about 30 times a day we should catch it before 90 days are up | 22:52 |
| *** diablo_rojo has joined #openstack-infra | 22:53 | |
| *** slaweq has joined #openstack-infra | 22:54 | |
| fungi | i'm more concerned about not noticing it's broken | 22:55 |
| fungi | though if we continue doing the certcheck independent of that we should be covered | 22:55 |
| fungi | still, renewing after 30 days might make some sense | 22:56 |
| clarkb | fungi: ya I think checking externally via certcheck is likely our best bet there | 22:56 |
| *** wolverineav has joined #openstack-infra | 22:56 | |
| clarkb | then its harder to break the whole thing and not notice | 22:56 |
| *** slaweq has quit IRC | 22:58 | |
| *** tkajinam has joined #openstack-infra | 22:59 | |
| fungi | though we do still need to move to a new enough certcheck that we can properly test through sni | 22:59 |
| fungi | that's how the zuul-ci.org expiration slipped past us | 23:00 |
| clarkb | do we run that on status.o.o today? | 23:01 |
| clarkb | no cacti | 23:02 |
| corvus | ianw: as you may have seen in #zuul -- unfortunately we seem to have hit a case where nova simply says "error" with no additional information, so the only recourse is a support request to the cloud provider i think | 23:02 |
| *** rlandy|ruck is now known as rlandy | 23:03 | |
| fungi | clarkb: yeah, the certcheck in xenial is just barely too old to do sni properly | 23:03 |
| mordred | efried, mriedem: ^^ if we get a server in ERROR state and there is no fault object (or no message in the fault object) - is there any other place we should be looking api-wise to try to learn why it's in ERROR state? | 23:03 |
| fungi | (or is the api for that "open a support ticket with your provider?") | 23:04 |
| fungi | clarkb: on the other hand, if we wanted a dedicated certcheck host, i bet that would be easy to containerize | 23:05 |
| clarkb | https://github.com/Matty9191/ssl-cert-check/commit/f0a7230a9672360a5e62aba4fa1641fb2918220d is that the commit we need? | 23:05 |
| clarkb | maybe instead of using the distro package we should just pull that git repo down | 23:06 |
| clarkb | I bet I can make that change pretty quickly /me gives it a try | 23:06 |
| mriedem | mordred: hmm, weird, you could check the instance actions list for the server | 23:07 |
| dabukalam | any chance someone could have a look at this when they have a free sec? https://review.openstack.org/#/c/644679/\ | 23:07 |
| mriedem | nova instance-action-list <server> | 23:07 |
| mriedem | mordred: are you admin role or not? | 23:07 |
| mordred | mriedem: non admin | 23:07 |
| mordred | mriedem: just normal cloud user | 23:07 |
| fungi | clarkb: it's been a while since i looked so i honestly don't recall... but the changelog (or package changelog maybe?) mentioned it i think | 23:07 |
| mriedem | the instance action might show which event failed | 23:07 |
| mriedem | https://docs.openstack.org/python-novaclient/latest/cli/nova.html#nova-instance-action-list | 23:08 |
| fungi | clarkb: but yeah, that seems like a safe enough thing to maybe not bother sticking to the distro package on | 23:08 |
| mriedem | https://docs.openstack.org/python-novaclient/latest/cli/nova.html#nova-instance-action | 23:08 |
| mordred | mriedem: k - but there's no additional places to look on the server object itself without making another api call in this case, yeah? | 23:08 |
| mriedem | mordred: just the fault | 23:08 |
| mordred | kk | 23:08 |
| mordred | mriedem: this is in the nodepool context trying to provide additional debug logging in the cases where nodes can be boot ... so I think we want to avoid the extra calls - in this case I thnik we'll have to stick with "shrug" | 23:09 |
| mordred | thanks! | 23:09 |
| mordred | s/can be/cannot be/ | 23:09 |
| mriedem | if you can get some logs to figure out where it failed, i can see if we have fault handling b/c anytime the server is in ERROR status there should be a fault | 23:09 |
| corvus | i'll try to find the action list | 23:10 |
| corvus | the server has already been deleted | 23:12 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-ssl_cert_check master: Run ssl-cert-check out of git source https://review.openstack.org/650162 | 23:12 |
| clarkb | fungi: ^ | 23:12 |
| clarkb | corvus: mordred ianw mriedem may need to boot a server manually outside of nodepool | 23:13 |
| mordred | corvus: I see a bug | 23:13 |
| mordred | corvus: that is likely causing this to be hidden | 23:13 |
| mordred | corvus: nevermind | 23:14 |
| mordred | corvus: I was wrong- I was thinking we might be normalizing the fault away - but we shouldn't be | 23:14 |
| clarkb | ianw 650162 may interest you as well given the slight relation to the letsencrypt work | 23:15 |
| fungi | clarkb: i half expect that as soon as we approve that change we'll be promptly spammed about some number of certs we didn't know are about to expire | 23:15 |
| clarkb | fungi: :) | 23:15 |
| corvus | fungi: best avoid it then! | 23:15 |
| * fungi sticks his head back in the sand | 23:16 | |
| mordred | fungi: sand is your friend | 23:18 |
| *** dklyle has quit IRC | 23:19 | |
| fungi | i have a car full of the stuff if you need any | 23:19 |
| clarkb | bah there is a puppet syntax bug in that | 23:19 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-ssl_cert_check master: Run ssl-cert-check out of git source https://review.openstack.org/650162 | 23:20 |
| clarkb | now there is one fewer syntax error | 23:20 |
| fungi | also every time i vacuum the house, it's like half of what i pick up is just loose sand | 23:20 |
| clarkb | fungi: sounds like you should replace your floors with grass | 23:20 |
| clarkb | hold the sand in place and fight erosion :) | 23:20 |
| corvus | all you have to do for that to happen is: nothing. | 23:21 |
| fungi | hah | 23:21 |
| clarkb | infra-root not sure if you all saw but I sent email to the infra list solicting feedback on dmarc_moderation_action handling options for a xenial listserv | 23:21 |
| fungi | dunno, every time my yard gets inundated with salt water it seems like the grass takes a major hit. marsh reeds is what i really need | 23:22 |
| corvus | clarkb: what's your timetable for the list upgrade? | 23:23 |
| clarkb | corvus: apparently our permanent DST bill requires california and washington also agree to switch to permanent DST, but it just passed in the state senate and is headed to the house next :) | 23:24 |
| clarkb | corvus: I'd like to do it sometime this month since trusty is EOL in like 3 weeks maybe 2 at this point | 23:24 |
| mordred | fungi: have you tried planting coral? | 23:24 |
| clarkb | corvus: looking at a calendar probably the week after the 19th | 23:24 |
| clarkb | corvus: 22nd or 23rd maybe? | 23:24 |
| *** _erlon_ has quit IRC | 23:25 | |
| fungi | mordred: (unfortunately?) most of my living space is too dry for coral | 23:25 |
| mordred | fungi: of course, then you'd just wind up with the parrot fish eating it and end up with sand anyway | 23:25 |
| mordred | fungi: that's probably for the best | 23:25 |
| fungi | permanently switching to pdt would basically be the same as just being on mst year round, right? | 23:26 |
| mordred | fungi: yes. but it would be called pdt | 23:26 |
| fungi | oh, right, to maintain an independence of identity | 23:27 |
| clarkb | well we might actually have to switch to mountain time because congress ahs to authorize changes to dst | 23:27 |
| mordred | fungi: so you could have signs on the interstate saying "leaving mountain time, entering pacific time" - but have no action needed | 23:27 |
| clarkb | but states can choose their non dst timezone aiui | 23:27 |
| mordred | clarkb: hahahaha | 23:27 |
| corvus | clarkb: i think for a complete answer we need to address the question of whether we want to implement the reject-on-mailman-will-alter-message behavior. in other words, broadly speaking, we either try to avoid dmarc issues by not altering messages and rejecting in the cases where doing so would break dmarc, or give up and wrap or munge. | 23:27 |
| *** wolverineav has quit IRC | 23:27 | |
| corvus | clarkb: most of the options on the etherpad are about avoiding altering messages, but i think if we're going to continue down that path we need to commit to rejecting when we can't | 23:28 |
| corvus | (because right now, we are accepting messages we know we can't deliver to some people, which is not good) | 23:28 |
| mordred | clarkb: and congress is not needed to approve ceasing to observe DST right? so the bill could be "stop observing DST and move to mountain" | 23:28 |
| clarkb | corvus: do you think we need to couple that to the upgrade? | 23:28 |
| fungi | it's a sticky topic. the merits to giving up are that the software is written to do that. the software needed to do the rejecting would need someone to write it | 23:29 |
| clarkb | corvus: or do you think it is safe to upgrade with a mostly status quo then address it one way or the other? | 23:29 |
| clarkb | (or is the shiny new button too much of a potential honey pot to have to untangle later) | 23:29 |
| corvus | clarkb: i think status quo upgrade is okay. and let's just ask people not to change that right now. | 23:29 |
| fungi | people do seem to rather like clicking shiny new buttons | 23:30 |
| mordred | corvus: reject-on-mailman-will-alter-message would be an exim input filter I suppose? | 23:30 |
| corvus | clarkb: then maybe we can hash it out at ptg or something | 23:30 |
| corvus | mordred: yep | 23:30 |
| fungi | mordred: yes | 23:30 |
| clarkb | corvus: ok we should be able to note that in the upgrade announcement, do the upgrade then ya dig in at the ptg or later | 23:30 |
| clarkb | corvus: fungi it could potentially be a mailman pipeline entry too? | 23:31 |
| fungi | we would not want to bounce after accepting into the queue, as 90% of what we receive is actually spam | 23:31 |
| mordred | yeah | 23:31 |
| corvus | mordred: rough idea on implementation would be something like "write a python script that either looks for the known breakage cases, or maybe even does "import mailman" and sees what it would do to the message" | 23:31 |
| *** wolverineav has joined #openstack-infra | 23:31 | |
| mordred | corvus: oh fun | 23:31 |
| corvus | mordred: (if the cases are really simple, we could probably just write exim rules) | 23:31 |
| *** wolverineav has quit IRC | 23:31 | |
| *** wolverineav has joined #openstack-infra | 23:32 | |
| fungi | most of the unaddressed problems at this point are around mailman's insistence on normalizing header indents | 23:32 |
| clarkb | if we wrote a custom handler it should be able to go ahead of the spamdetect handler which is where we first start the munging/handling of this stuff aiui | 23:32 |
| clarkb | hrm though maybe not of the normalized header indents | 23:32 |
| fungi | and also it seems to rewrite some headers and redo the line wrapping in so doing | 23:33 |
| fungi | which gets screwy of those headers are malformed to begin with | 23:33 |
| corvus | clarkb: maybe -- if it's a mm filter, we'd have to be able to generate a bounce message to the sender. if we do it in exim, we can reject at smtp time which is always preferable. so if there's not a compelling reason to do it in mailman, exim would be preferred i think. | 23:33 |
| clarkb | gotcha | 23:33 |
| clarkb | (mostly just mentioning it at as an option as I totally got a custom handler working while testing this upgrade :) ) | 23:34 |
| corvus | clarkb: though, perhaps it could be the last filter in the pipeline and then it could just do "does dmarc validate? if no, generate bounce to sender" | 23:34 |
| clarkb | given this discussion I'll probably use next weeks meeting to schedule the upgrade. The only reason I'm wary of doing it next week (or the big reason at least) is that corvus will be afk. If we don't think short notice is a problem and others are cool with it maybe I'll do it fridayish | 23:35 |
| corvus | that might be the simplest and most reliable implementation | 23:35 |
| clarkb | thats a week from tomorrow | 23:35 |
| fungi | languishing on my todo list is to also implement score-based spam filtering at rcpt time in exim, to handle things like the unending flood of (spoofed?) @qq.com spam to -owner aliases | 23:35 |
| clarkb | corvus: that would definitely be fairly accurate. Except for things like the base64 conversion which happen even later in the mm processing | 23:35 |
| clarkb | but ya that should be really accurate | 23:36 |
| ianw | corvus: sorry, the moment i asked for help the electrician arrived. long story short, the guy who put a screw through the cat6 cable is pretty lucky he didn't hit the live 240v cable next to it ... | 23:36 |
| clarkb | ianw: careful I think stuff like that is why we insist on 110v here | 23:36 |
| fungi | though also i skim and discard a few hundred spam posts per day for the openstack-discuss ml alone | 23:36 |
| fungi | yeah, i could stick my tongue in an electrical outlet here in the states and survive only mildly shaken | 23:37 |
| anteaya | based on experience? | 23:37 |
| fungi | let's say it's speculation ;) | 23:38 |
| anteaya | ha ha ha | 23:38 |
| clarkb | corvus: to consider the other major option the downsides to wrapping/munging are rendering "issues" in clients and inability to properly use things like contact lists? | 23:38 |
| * mordred agrees with fungi from related experience - other than it would be very hard to get the tongue into the US electrical outlet | 23:38 | |
| clarkb | When I was debugging the base64 stuff I found mention that apple devices really hate the wrap method | 23:38 |
| fungi | though i can say from experience than holding both leads of a pots line when a ring pulse comes through is far worse than grabbing live home electrical wires here | 23:38 |
| clarkb | because tehy don't show you the original email | 23:38 |
| anteaya | fungi glad I can benefit from your experience | 23:39 |
| mordred | yeah. home electrical shock is just annoying. it's like "dang it, why did the previous person wire this backward" | 23:39 |
| *** igordc has joined #openstack-infra | 23:39 | |
| mordred | incidentally- it feels exactly like getting a tattoo | 23:40 |
| fungi | (and then you remember that the previous person was also you) | 23:40 |
| mordred | fungi: sssh | 23:40 |
| corvus | mordred: maybe your tattooist should have their grounding checked? | 23:40 |
| mordred | corvus: sssh | 23:40 |
| *** eharney has quit IRC | 23:40 | |
| corvus | mordred: i guess tattoo guns are 60hz solenoids? | 23:41 |
| fungi | that would make sense | 23:41 |
| fungi | simple in design | 23:41 |
| mordred | corvus: yeah. it's the buzz the feels the same | 23:41 |
| mordred | so - getting a tattoo _basically_ feels like a continuous shock more or less. I think this is why it didn't bother me too much - and why sandy was so annoyed that it didn't bother me too much :) | 23:42 |
| mordred | maybe that indicates that I've been shocked enough for it to not bother me - which might not make people believe strongly in my electrical skills | 23:43 |
| fungi | i suppose in some parts of the world, a tattoo takes 20% longer | 23:43 |
| mordred | hahaha | 23:43 |
| *** tosky has quit IRC | 23:43 | |
| clarkb | the original tappy tap tapmethods are definitely not 60 hz | 23:43 |
| ianw | we got more volts, but we're only 50hz | 23:44 |
| fungi | yeah, those parts of the world | 23:44 |
| ianw | i also have great respect for power points, especially after i melted multimeter probes and learnt what a "floating voltage" means | 23:44 |
| mordred | ianw: when I'm working on higher voltage panels, I tend to be more careful and actually disconnect things before working on it | 23:45 |
| mordred | ianw: when I'm working on 110, I usually just shrug and make sure to use insulated tools | 23:45 |
| corvus | speaking of 60 cycle hums.... https://www.youtube.com/watch?v=Lzlc0-Mx0sk | 23:46 |
| corvus | (we had that on reel to reel at the stew) | 23:46 |
| clarkb | a friend and I shorted a car battery with some high guague wire thankfully so it basically just melted and some how didn't catch everything on fire (this was back in high school and now I fear electricity) | 23:46 |
| mordred | corvus: ++ | 23:47 |
| ianw | this was me being an idiot and thinking maybe i could hot-rod a motor speed controller with an arduino or something. sure the panel runs on 9v, but through a capacative dropper ... | 23:47 |
| mordred | ha | 23:48 |
| mordred | corvus: I don't think I ever listened to that on the r2r | 23:49 |
| mordred | corvus: I'm also fairly certain it must have been there - since the likelihood new reels were gotten between me being there and you arriving is pretty low :) | 23:50 |
| corvus | i um... spent a lot of time with the r2r | 23:50 |
| anteaya | I love the song | 23:50 |
| anteaya | sounds like soundtrack for the Hours | 23:51 |
| corvus | i did not know until now it was a collaboration with paul simon. that's pretty cool. | 23:51 |
| anteaya | it says nothing in the notes about paul simon | 23:51 |
| clarkb | I hda to google that reel to reel is what it sounded like | 23:51 |
| anteaya | it is awesome | 23:51 |
| clarkb | at least I know what a floppy disk is right? | 23:52 |
| anteaya | do you? | 23:52 |
| mordred | anteaya: ++ | 23:52 |
| clarkb | (I actually have a 3.5" floppy usb drive because windows XP didn't have sata driver and so to avoid installing via IDE emulation I sideloaded drivers via floppy which was the only side loading medium the installer supported) | 23:53 |
| clarkb | it wasn't long after that I realized this windows stuff wasn't for me | 23:53 |
| anteaya | ha ha ha | 23:53 |
| clarkb | you know what didn't have problems with sata? | 23:53 |
| clarkb | the linux kernel | 23:53 |
| corvus | clarkb: that is awesome you should keep that | 23:53 |
| mordred | anteaya: the lyrics are paul simon | 23:53 |
| anteaya | the lyrics are awesome | 23:54 |
| anteaya | I wouldn't have recognized his voice at all | 23:54 |
| mordred | corvus: oh wow - that was recorded at the living room | 23:54 |
| clarkb | corvus: ya its in a box in my office closet | 23:54 |
| clarkb | corvus: I imagine in 20 years someone will have some file they have to get off of a floppy drive and I'll be their rescuer | 23:55 |
| clarkb | more likely we'll dig out old copies of where in the world is carmen san diego and play it on dosbox | 23:55 |
| anteaya | clarkb: ha ha ha | 23:55 |
| clarkb | also I already have a usb A to usb C adaptor for other reasons so should be covered on port compatibility for a while | 23:55 |
| *** tjgresha has joined #openstack-infra | 23:56 | |
| mordred | I recently got one of those as a happy accident | 23:56 |
| ianw | clarkb: yeah well, glass houses and stones ... in those days installing slackware via floppies and getting X11 working was not exactly user friendly :) | 23:56 |
| mordred | I needed a usb-c -> hdmi adaptor - and the one I got also had usb A ports - which it turns out work great for plugging in clickers | 23:56 |
| clarkb | ianw: oh ya remember when you had to edit your x11 config by hand? | 23:56 |
| clarkb | mordred: I actually got mine so that I can yubikey on my phone | 23:57 |
| anteaya | this is a really long song | 23:57 |
| mordred | clarkb: hah. nice | 23:57 |
| mordred | anteaya: philip glass isn't going to rush things | 23:57 |
| corvus | i recently unearthed a box of floppies, and i sure did find a numbered set of 20+ and i think it might be slackware | 23:57 |
| anteaya | mordred: my style of music | 23:57 |
| corvus | or maybe sls | 23:57 |
| clarkb | corvus: should I bring the floppy drive to the summit so you can install it again? | 23:58 |
| corvus | yeah! i'll bring my work laptop... | 23:58 |
| clarkb | ha | 23:58 |
| *** gyee has quit IRC | 23:58 | |
| mordred | I think nova should add support for boot-from-floppy | 23:59 |
| clarkb | ianw: I do have to say for all of my fiddling with linux ubuntu in the last half decade or so has been really user friendly for people that are not like us (eg working on linux as day job) | 23:59 |
| openstackgerrit | Merged openstack-infra/zuul master: Use the buildset registry in zuul-quick-start https://review.openstack.org/649467 | 23:59 |
| igordc | mordred, +1 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!