Wednesday, 2019-04-03

*** wolverineav has joined #openstack-infra00:00
clarkbI've also learned a lot about mailman00:01
clarkbheres an idea, we can reset the subscriber list for openstack-discuss on the new server then use it to test maybe?00:02
clarkbthat would at least help narrow it down to list config if it continues to have the trusty behavior00:02
clarkbI won't go doing that with extra eyeballs as I don't want to spam a ton of people00:02
clarkband I should step away for a bit to figure out dinner00:02
clarkbthanks again for the help. Much of this is new to me and I am learning a bunch :)00:02
*** igordc has joined #openstack-infra00:05
*** wolverineav has quit IRC00:06
igordchi all, who can help me with deleting one of my gerrit's associated email addresses?00:07
*** sthussey has quit IRC00:07
clarkbigordc: you need an infra root for that. It helps if you can identify the active account so we can merge the openids of the other accounts into that one00:11
clarkbIm no longerat my computer but if you pass that info along we should be able to sort it out00:12
igordcclarkb, gerrit account that needs email address removed is "igordcard"00:13
igordcclarkb, thank you00:13
clarkbis that the one you actively use now?00:14
*** zxiiro is now known as zxiiro-away00:20
*** armax has quit IRC00:24
*** jamesmcarthur has quit IRC00:24
*** hamzy has joined #openstack-infra00:28
*** wolverineav has joined #openstack-infra00:28
igordcclarkb, that is the one I've used until now00:30
igordcclarkb, I want to switch00:30
igordcclarkb, and migrate the previous email address00:31
openstackgerritMerged openstack-infra/nodepool master: Use yaml.safe_load instead of load
*** jamesmcarthur has joined #openstack-infra00:55
*** jamesmcarthur has quit IRC00:55
*** jamesmcarthur has joined #openstack-infra00:56
*** wolverineav has quit IRC00:58
*** wolverineav has joined #openstack-infra01:02
*** wolverineav has quit IRC01:06
fungiigordc: did you end up with an extra gerrit account? otherwise you should just be able to register the new address to your existing account and set it as preferred01:06
igordcfungi, the former01:06
fungigot it. i can associate the new openid with the old account and then deactivate the new account if that's preferable... do you have any active reviews or group memberships under the old account?01:08
*** kmarc- has joined #openstack-infra01:11
*** benj_- has joined #openstack-infra01:11
*** lathiat_ has joined #openstack-infra01:12
*** manfred has joined #openstack-infra01:12
*** kmarc has quit IRC01:12
*** benj_ has quit IRC01:12
*** mudpuppy has quit IRC01:12
*** toabctl has quit IRC01:12
*** spotz has quit IRC01:12
*** lathiat has quit IRC01:12
*** gtmanfred has quit IRC01:12
*** manfred is now known as gtmanfred01:12
*** ijw has quit IRC01:12
*** rascasoft has quit IRC01:16
*** ijw has joined #openstack-infra01:16
*** rascasoft has joined #openstack-infra01:17
*** wolverineav has joined #openstack-infra01:18
*** ruffian_sheep has joined #openstack-infra01:20
*** ijw has quit IRC01:21
*** wolverineav has quit IRC01:22
ruffian_sheepfrickler:If I upload my driver of cinder,should I run all of the CI?01:23
*** dtantsur|afk has quit IRC01:24
*** dtantsur has joined #openstack-infra01:25
ruffian_sheepfrickler:Like that :
igordcfungi, so first I'm looking to move completely to the new account, in time, .. but yes there are still pending things on the old account01:27
igordcfungi, is it not possible simply execute gerrit set-account --delete-email from your side?01:28
igordcsorry I have to go now but will be back online01:29
*** igordc has quit IRC01:29
*** wolverineav has joined #openstack-infra01:39
*** hongbin has joined #openstack-infra01:41
*** wolverineav has quit IRC01:43
*** whoami-rajat has joined #openstack-infra01:57
*** ykarel|away has joined #openstack-infra02:06
*** mriedem has quit IRC02:07
*** spsurya has joined #openstack-infra02:12
*** bhavikdbavishi has joined #openstack-infra02:26
*** bhavikdbavishi has quit IRC02:30
*** jamesmcarthur has quit IRC02:31
*** bhavikdbavishi has joined #openstack-infra02:37
openstackgerritJoshua Hesketh proposed openstack-infra/project-config master: Retire oslosphinx
*** hongbin has quit IRC03:00
*** jamesmcarthur has joined #openstack-infra03:01
*** Sundar has quit IRC03:02
openstackgerritIan Wienand proposed openstack/diskimage-builder master: debian-minimal buster support
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [WIP] add buster tests
ianwfungi / zigo : ^ so it's not straight forward; i don't really know who wants to be on the hook for driving this ...03:14
*** psachin has joined #openstack-infra03:16
*** hwoarang has quit IRC03:17
*** hwoarang has joined #openstack-infra03:18
*** roman_g has quit IRC03:22
*** nicolasbock has quit IRC03:33
*** wolverineav has joined #openstack-infra03:40
*** armax has joined #openstack-infra03:41
*** wolverineav has quit IRC03:44
*** eernst has joined #openstack-infra03:45
*** udesale has joined #openstack-infra04:00
*** ramishra has joined #openstack-infra04:17
*** igordc has joined #openstack-infra04:18
*** hwoarang has quit IRC04:19
*** hwoarang has joined #openstack-infra04:21
*** ykarel|away has quit IRC04:26
*** ykarel|away has joined #openstack-infra04:42
*** whoami-rajat has quit IRC04:57
openstackgerritMerged openstack-infra/zuul master: Add web / fingergw connections for components graph
*** eernst has quit IRC05:04
*** ykarel|away has quit IRC05:18
*** toabctl has joined #openstack-infra05:21
*** ykarel|away has joined #openstack-infra05:24
*** quiquell|off is now known as quiquell05:42
*** kjackal has joined #openstack-infra05:54
*** janki has joined #openstack-infra06:17
*** adam_g has joined #openstack-infra06:24
*** slaweq has joined #openstack-infra06:26
*** bhavikdbavishi1 has joined #openstack-infra06:32
*** bhavikdbavishi has quit IRC06:33
*** bhavikdbavishi1 is now known as bhavikdbavishi06:33
*** pcaruana has joined #openstack-infra06:36
*** pcaruana has quit IRC06:38
*** pcaruana has joined #openstack-infra06:38
*** pgaxatte has joined #openstack-infra06:44
*** dpawlik has joined #openstack-infra06:47
*** rpittau|afk is now known as rpittau06:50
jamespagefungi, clarkb: hey - python-pip issues in bionic? is there a bug I can reference?06:50
jamespagehappy to help drive an SRU through06:50
*** jpena|off is now known as jpena06:55
*** quiquell is now known as quiquell|brb06:56
*** kopecmartin|off is now known as kopecmartin06:57
*** tkajinam has quit IRC06:58
zigojamespage: I've fixed it in Sid/Buster (currently waiting for migration), the patch from clarkb is quite strait forward.07:02
zigoThough maybe you'll prefer upstream patch...07:02
zigoI'm still waiting for approval for the Stretch P-U.07:03
*** tkajinam has joined #openstack-infra07:03
openstackgerritMerged openstack-infra/nodepool master: Use opendev docker jobs
zigojamespage: How about fixing debootstrap in Xenial/Bionic, so it can bootstrap Buster?07:03
zigoCould you also work on that?07:03
zigojamespage: See ...07:04
openstackDebian bug 901977 in debootstrap "--variant=minbase + --exclude={tzdata,e2fsprogs,e2fslibs} causes install failure" [Normal,Fixed]07:04
zigoianw: If the PPA is a faster solution, I can work on a patched package, would that work?07:07
*** tosky has joined #openstack-infra07:09
*** whoami-rajat has joined #openstack-infra07:13
openstackgerritMonty Taylor proposed openstack-infra/zuul-website master: Slide presentation for Zuul's OIP Confirmation
*** ruffian_sheep has quit IRC07:17
*** xek has joined #openstack-infra07:19
*** jistr is now known as jistr|afk07:22
zigojamespage: When we're at it, Debian is't really affected because on Azure, we don't support cloud-init deployed images, but I still would like to fix #926043, and I can't find a patch. The launchpad bug is a mess, with multiple commits. How can I find out what commit to apply to fix CVE-2019-0816 then?07:25
*** e0ne has joined #openstack-infra07:27
*** shachar has quit IRC07:30
*** yolanda has joined #openstack-infra07:33
*** lmiccini has quit IRC07:33
openstackgerritMerged openstack-infra/irc-meetings master: Remove neutron_upgrades meeting
openstackgerritMerged openstack-infra/irc-meetings master: Update keystone meeting chair
openstackgerritMonty Taylor proposed openstack-infra/zuul-website master: Slide presentation for Zuul's OIP Confirmation
*** roman_g has joined #openstack-infra07:42
*** helenafm has joined #openstack-infra07:43
*** iurygregory has joined #openstack-infra07:45
*** snapiri has joined #openstack-infra07:46
*** quiquell|brb is now known as quiquell07:53
*** jpich has joined #openstack-infra07:56
*** igordc has quit IRC08:04
*** lennyb has joined #openstack-infra08:04
*** e0ne has quit IRC08:07
*** ramishra has quit IRC08:07
*** e0ne has joined #openstack-infra08:15
*** armax has quit IRC08:17
*** e0ne has quit IRC08:17
*** ramishra has joined #openstack-infra08:19
*** lucasagomes has joined #openstack-infra08:20
fricklerjamespage: I created for the bionic pkg08:20
openstackLaunchpad bug 1822842 in python-pip (Ubuntu) "pip fails to install package from source when extra-index-url returns 404" [Undecided,New]08:20
jamespagefrickler: thanks!08:21
* jamespage looks08:21
*** e0ne has joined #openstack-infra08:24
*** ccamacho has joined #openstack-infra08:25
*** e0ne has quit IRC08:26
ianwzigo: it's not much, just being a single script + library file.  i think if anything the sticking point might be updating so many versions in lts branches and other regressions08:27
jamespagefrickler: I'm right in thinking this impacts all pip versions in ubuntu releases?08:28
*** priteau has joined #openstack-infra08:28
ianwzigo: a ppa can work, we have a infra user where we keep a bunch of other things that we use patched.  it would have to go in there, then puppet would have to be updated to install on nodepool builders, and dib jobs updated to install it too08:29
ianwzigo: should probably get fungi's input, now we have more an idea what's going on too08:29
fricklerjamespage: most likely yes, though I only checked myself on bionic08:30
*** ykarel|away is now known as ykarel|lunch08:30
jrosseris it possible to get wheels build for amd64 on the infra mirrors?08:33
jrosseraarch64. i will get this right eventually :)08:34
*** ykarel|lunch has quit IRC08:35
jrossersomeone asked me about aarch64 CI jobs yesterday, we did some work on aarch64 for openstack-ansible a while back but there seems to be little chance of getting a job to run within the time limit with nothing prebuilt08:37
ianwjrosser: we do?08:37
jrosserdo we? i looked here
ianw hrm08:39
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add tags to jobs list
*** kopecmartin is now known as kopecmartin|lunc08:40
ianwjrosser: i'd say we never got to the "until the builds are tested bit" of
ianwjrosser: but the answer should be yes, it should be possible, modulo debugging that job (and expanding for bionic)08:41
jrosserianw: ahha, well ok. there is some renewed interest in this08:41
jrosserwe've got successful deploys on aarch64 of openstack-ansible and all the patches to make that work exist08:42
jrosserjust none of it has ever been tested in CI which is a bit sad08:42
*** owalsh_ is now known as owalsh08:43
jrossernowster: ^^ see this re. aarch64 wheels08:44
*** tkajinam has quit IRC08:45
*** wolverineav has joined #openstack-infra08:45
ianwjrosser: ok ... it's a bit late for me now, but i can trigger that job tomorrow and get some fresh logs, and we can go from there.08:46
ianwjrosser: could you add a story on storyboard, against openstack-infra/project-config i guess, and we can keep track of it there?08:46
jrosserianw: sure - there are some folks from codethink poking generally at the aarch64 stuff, if some extra input is needed they may be able to help08:46
*** derekh has joined #openstack-infra08:46
ianwjrosser: it's been a while since I've actively looked, i think our arm64 nodes are in ok shape and the daily image builds are working.08:47
jrosserwe've certainly run the equivalent of a gate job on a vexxhost arm node and whilst very slow it does work08:48
ianwanother infra-root can help with the job kick off too, but i think that's the place to start right now.  get some fresh logs and evaluate what's going on08:48
jrossergreat - thanks for the help08:48
ianwwe'll also want a bionic variant, but that shouldn't be hard08:48
ianwok, i gotta go, but i'm sure we can get progress on this08:49
*** wolverineav has quit IRC08:50
zigoianw: Ok, I'll try to find time to make a patched version then.08:52
*** ykarel|lunch has joined #openstack-infra09:00
chandankumarianw: please have a look at this patch when free, thanks!09:01
nowstermorning jrosser, ianw09:15
jrossernowster: good morning - i made this as requested to track the aarch64 stuff!/story/200535309:16
nowsterI'm preparing a patch for Debian/Ubuntu to allow openafs to build in stretch/bionic. It's already in buster/cosmic.09:16
nowster(openafs on arm64/aarch64, that is)09:19
*** xek has quit IRC09:20
*** xek has joined #openstack-infra09:21
*** xek has quit IRC09:22
*** jpich has quit IRC09:27
*** jpich has joined #openstack-infra09:28
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: web: add tenant and project scoped, JWT-protected actions
*** bhavikdbavishi has quit IRC09:30
slaweqfrickler: ianw: hi, I have one more time question to hold some nodes for me to debug09:34
slaweqfrickler: ianw: can You do it for me once again?09:34
slaweqI need to debug why job tobiko-devstack-neutron-fedora is failing every time and same job based on ubuntu is running fine09:35
*** dhellmann has quit IRC09:37
*** whoami-rajat has quit IRC09:37
*** dhellmann has joined #openstack-infra09:46
*** electrofelix has joined #openstack-infra09:49
*** dhellmann has quit IRC09:53
*** markvoelker has joined #openstack-infra09:55
*** yamamoto has quit IRC09:58
*** gfidente has joined #openstack-infra09:58
*** yamamoto has joined #openstack-infra10:01
*** dhellmann has joined #openstack-infra10:02
*** yamamoto has quit IRC10:03
*** e0ne has joined #openstack-infra10:03
dulekinfra-root: Hey, you can release 1a854d23-eecc-4d7a-97f8-f7769d6d53f7 on limestone-regionone (id 0004609144). I don't need it anymore, we reproduced the issue locally and already know partial root cause.10:07
*** e0ne has quit IRC10:11
fricklerslaweq: do you have a special patch you want to check?10:13
slaweqfrickler: no, it fails on every patch on this job10:14
slaweqbut I can push some DNM patch if You want10:14
fricklerslaweq: probably safer to avoid some potentially broken patch10:14
slaweqfrickler: sure, give me a minute10:14
fricklerdulek: thanks for the feedback, deleting that node now10:14
*** e0ne has joined #openstack-infra10:15
*** jpena has quit IRC10:16
slaweqfrickler: so it's patch
slaweqjob didn't start yet10:17
fricklerslaweq: o.k., I'll set up the hold10:18
slaweqfrickler: thx a lot10:18
*** ykarel|lunch is now known as ykarel10:20
*** yamamoto has joined #openstack-infra10:22
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Allow operator to generate auth tokens through the CLI
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Zuul CLI: allow access via REST
*** markvoelker has quit IRC10:27
zbrianw: can you please have a look at ?10:27
*** sshnaidm|rover is now known as sshnaidm|bbl10:27
*** jpena has joined #openstack-infra10:29
*** nicolasbock has joined #openstack-infra10:33
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Add Authorization Rules configuration
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Web: plug the authorization engine
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Zuul Web: add /api/user/authorizations endpoint
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: authentication config: add optional token_expiry
dulekWhy do I see some combined log messages in Like SUCCESS,FAILURE and ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode,kuryr-kubernetes-tempest?10:44
dulekAny way to search only for kuryr-kubernetes-tempest job without those combined ones?10:44
*** wolverineav has joined #openstack-infra10:46
*** quiquell is now known as quiquell|lunch10:48
*** wolverineav has quit IRC10:51
*** tbachman has quit IRC10:51
*** e0ne has quit IRC10:53
*** ramishra has quit IRC10:56
*** jistr|afk is now known as jistr11:02
*** jpena is now known as jpena|lunch11:02
*** ramishra has joined #openstack-infra11:04
*** helenafm has quit IRC11:04
*** gfidente has quit IRC11:09
*** kopecmartin|lunc is now known as kopecmartin11:15
*** yamamoto has quit IRC11:15
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Add Authorization Rules configuration
*** yamamoto has joined #openstack-infra11:20
*** gfidente has joined #openstack-infra11:21
*** quiquell|lunch is now known as quiquell11:24
*** gfidente has quit IRC11:24
*** markvoelker has joined #openstack-infra11:24
*** udesale has quit IRC11:26
*** dikonoor has joined #openstack-infra11:30
fricklerslaweq: nodes, and are waiting for you11:30
slaweqfrickler: thx a lot11:30
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Web: plug the authorization engine
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: Zuul Web: add /api/user/authorizations endpoint
fricklerslaweq: always welcome, let me know if you need anything else11:37
*** rcernin has quit IRC11:38
*** apetrich has quit IRC11:48
*** apetrich has joined #openstack-infra11:49
zbrdoc8 package seems unmaintained, anyone in infra-core willing to help with few reviews there?11:51
*** rlandy has joined #openstack-infra11:55
*** markvoelker has quit IRC11:57
*** bhavikdbavishi has joined #openstack-infra11:59
*** jpena|lunch is now known as jpena12:02
*** apetrich has quit IRC12:05
*** rh-jelabarre has quit IRC12:06
*** rh-jelabarre has joined #openstack-infra12:06
*** trown|outtypewww is now known as trown12:07
*** ramishra has quit IRC12:10
*** tbachman has joined #openstack-infra12:12
*** odyssey4me has quit IRC12:12
*** ramishra has joined #openstack-infra12:12
*** quiquell is now known as quiquell|brb12:13
*** whoami-rajat has joined #openstack-infra12:19
*** e0ne has joined #openstack-infra12:20
*** apetrich has joined #openstack-infra12:23
*** odyssey4me has joined #openstack-infra12:26
*** jamesmcarthur has quit IRC12:26
*** jamesmcarthur has joined #openstack-infra12:27
*** eharney has quit IRC12:33
openstackgerritIvan Kolodyazhny proposed openstack-infra/project-config master: Stop project gating before retirement
*** dtantsur is now known as dtantsur|brb12:36
*** jamesmcarthur has quit IRC12:36
*** priteau has quit IRC12:36
*** dtantsur|brb is now known as dtantsur12:36
*** priteau has joined #openstack-infra12:37
*** janki has quit IRC12:38
*** kgiusti has joined #openstack-infra12:43
*** jamesmcarthur_ has joined #openstack-infra12:43
*** priteau has quit IRC12:44
*** wolverineav has joined #openstack-infra12:47
openstackgerritLiam Young proposed openstack-infra/project-config master: New charms & interface for Masakari integration
*** wolverineav has quit IRC12:52
*** ricolin_phone has joined #openstack-infra12:53
*** helenafm has joined #openstack-infra12:54
*** quiquell|brb is now known as quiquell12:54
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: authentication config: add optional token_expiry
*** sshnaidm|bbl is now known as sshnaidm13:03
*** gfidente has joined #openstack-infra13:07
fricklermnaser: I just had a post_failure in sjc1, possibly a duplicated node, if you want to look:
*** mriedem has joined #openstack-infra13:10
*** lseki has joined #openstack-infra13:11
*** wolverineav has joined #openstack-infra13:19
*** wolverineav has quit IRC13:23
*** ricolin_phone has quit IRC13:25
*** bhavikdbavishi has quit IRC13:27
*** ricolin has joined #openstack-infra13:30
openstackDebian bug 926321 in src:openafs "openafs: FTBFS on arm64" [Normal,Open]13:30
*** dklyle has quit IRC13:30
*** dklyle has joined #openstack-infra13:31
*** elod_off has quit IRC13:31
*** elod_off has joined #openstack-infra13:32
*** e0ne has quit IRC13:35
*** ykarel has quit IRC13:41
*** e0ne has joined #openstack-infra13:48
*** shachar has joined #openstack-infra13:53
*** oanson has quit IRC13:55
*** eharney has joined #openstack-infra13:55
*** yolanda has quit IRC13:55
*** snapiri has quit IRC13:56
*** udesale has joined #openstack-infra13:56
*** igordc has joined #openstack-infra14:01
*** ricolin_phone has joined #openstack-infra14:02
*** serverascode has joined #openstack-infra14:03
*** ricolin_phone has quit IRC14:05
openstackgerritStephen Finucane proposed openstack-infra/zuul master: tox: Integrate tox-docker
*** yolanda has joined #openstack-infra14:12
*** dklyle has quit IRC14:15
*** ramishra has quit IRC14:16
*** e0ne has quit IRC14:22
fricklerinfra-root: the telemetry team has recently been assigned a new PTL via the TC, who has now asked to be made core for some projects. I'd be willing to execute that request but I'd like to make sure that we agree on this, first.
*** e0ne has joined #openstack-infra14:23
corvusfrickler: looks legit to me14:24
*** kjackal has quit IRC14:25
*** kjackal has joined #openstack-infra14:26
*** yamamoto has quit IRC14:26
*** yamamoto has joined #openstack-infra14:26
*** yamamoto has quit IRC14:26
*** yamamoto has joined #openstack-infra14:27
*** yamamoto has quit IRC14:27
*** eernst has joined #openstack-infra14:28
*** yamamoto has joined #openstack-infra14:28
*** ykarel has joined #openstack-infra14:28
fricklero.k., I take that as 2 x +214:28
frickler#status log added Trinh Nguyen (new telemetry PTL) to gerrit groups {ceilometer,aodh,panko}-{core,stable-maint}
openstackstatusfrickler: finished logging14:29
*** ramishra has joined #openstack-infra14:32
openstackgerritMerged openstack-infra/nodepool master: Make local dib file cleanup method a static method
*** armax has joined #openstack-infra14:35
*** e0ne has quit IRC14:37
*** bhavikdbavishi has joined #openstack-infra14:39
fungifrickler: yeah, we can simply confirm from if in doubt14:39
fungiand thanks for getting that, it was on my to do list to grab if nobody beat me to it14:40
*** dklyle has joined #openstack-infra14:41
*** dpawlik has quit IRC14:44
*** e0ne has joined #openstack-infra14:44
openstackgerritNicolas Hicher proposed openstack-infra/zuul-jobs master: Propose submit-logstash role to export logs
*** yolanda has quit IRC14:50
*** bhavikdbavishi has quit IRC14:51
*** bhavikdbavishi has joined #openstack-infra14:51
*** aaronsheffield has joined #openstack-infra14:52
*** ramishra has quit IRC14:57
*** yolanda has joined #openstack-infra14:57
openstackgerritMerged openstack-infra/zuul master: Prepend path with bin dir of ansible virtualenv
*** sthussey has joined #openstack-infra15:19
openstackgerritMerged openstack-infra/zuul master: Add link to builds in job page
*** artom has joined #openstack-infra15:22
clarkbinfra-root as an fyi I've been recontacted by the people/person wanting the 'nimble' package name on pypi. They still want the package transfer and are ready for it now. I've been asked to transfer nimble to if there are no last minute objections I'll do that in a bit15:24
clarkb(I had previously got confirmation from Zhenguo Niu that the name is no longer used and can be transfered this went to the discuss list)15:25
fungiclarkb: sounds good, thanks15:25
*** eernst has quit IRC15:28
corvusclarkb: ++15:28
clarkbalso while I'm spinning the day up, corvus not sure if you saw my sumamry of what I/we learned about mailman yesterday. But I did confirm that trusty and xenial have the same utf8 -> base64 encoding behavior in python's email.message.Message15:29
clarkbI think my next step is to do a direct comparison of the installed code on trusty and xenial15:30
clarkbso far I've only compraed within bzr and that may not be accurate because well reality15:30
fungiclarkb: also i second your idea of wiping the subscriber list for openstack-discuss on th test server and adding the same members as the test list15:30
clarkband ya ^ was going to be the other thing I try15:31
corvusclarkb: istr mailman having a vendored email package at some point15:34
openstackgerritArtom Lifshitz proposed openstack-infra/project-config master: Add whitebox-tempest-plugin project
*** bhavikdbavishi has quit IRC15:36
*** e0ne has quit IRC15:37
*** e0ne has joined #openstack-infra15:37
*** quiquell is now known as quiquell|off15:38
*** rlandy is now known as rlandy|ruck15:40
*** Dobroslaw is now known as Dobroslaw_afk15:40
*** anteaya has quit IRC15:45
mnaserstephenfin: thanks for
stephenfinmnaser: Just got to figure out why the tests are moaning now \o/15:47
mnaserstephenfin: I guess we'll have to drop it for now in openstack ansible until it is unbroken and erleased15:48
stephenfinmnaser: I'd say so, yeah. What are you using it for?15:48
clarkbfungi: corvus: in comparing the Decorate files directly between installs if not header and not footer: return is in both of them15:49
mnaserstephenfin: im not a sphinx expert but..
clarkbthis happens before the block that I believe to be resulting in base64 ification15:49
clarkbnow wondering why -discuss seems to be hitting that and not getting converted but my test list isn't15:49
clarkbI'm pretty sure I've disabled the footer, but maybe header is not?15:50
artomSo, coming from Nova where reviews are, ahem, hard to come by, should I be poking -infra for reviews? Or let things take their course?15:50
*** pgaxatte has quit IRC15:50
mnaserartom: infra core ping is usually helpful (with a - instead of a space between the two words)15:50
artommnaser, much thanks - I'm blaming you if I get any heat for it ;)15:51
fungiclarkb: ought to be able to etll with a diff of the cli config dump15:51
artominfra-core, request to a add a new project here: Thanks in advance!15:51
fungimnaser: artom: infra-root15:51
artominfra-root, request to a add a new project here: Thanks in advance!15:52
fungibut in the case of configuration changes, config-core is more relevant15:52
stephenfinmnaser: Feels like something we could do in openstackdocstheme. Maybe bring it up on #openstack-doc15:52
fungi(that's a broader set of reviewers)15:52
stephenfinmnaser: In any case, I'd remove your use of that extension across the board. It's pretty much dead15:53
stephenfinmnaser: Oh, btw, this is the important piece
*** iurygregory has quit IRC15:53
stephenfinmnaser: I _think_ the header we have at the top of docs now gets us 90% of the way there, IMO15:53
artomfungi, cheers!15:53
*** jamesmcarthur_ has quit IRC15:53
mnaserstephenfin: ill try to dig and understand that a bit more when I can :>15:55
*** rpittau is now known as rpittau|afk15:57
*** bhavikdbavishi has joined #openstack-infra15:58
clarkbmsg_header and msg_footer are both '' arg, I mean that explains why -discuss isn't getting transformed but -infra is for example. But doesn't explain why my test list isn't the same.15:58
*** e0ne has quit IRC15:59
*** nicolasbock has quit IRC16:02
*** anteaya has joined #openstack-infra16:02
*** dpawlik has joined #openstack-infra16:02
*** ykarel is now known as ykarel|away16:02
*** dikonoor has quit IRC16:05
*** e0ne has joined #openstack-infra16:05
*** bhavikdbavishi has quit IRC16:05
*** e0ne has quit IRC16:05
*** bhavikdbavishi has joined #openstack-infra16:06
*** e0ne has joined #openstack-infra16:06
clarkbfungi: corvus: ok I think most recent email (from a minute ago) produced logging that shows are aren't hitting that short circuit on xenial even though both header and footer appaer to be empty16:07
*** lucasagomes has quit IRC16:07
corvusclarkb: oh, wow, are you saying that openstack-infra currently b64s plain text?16:07
clarkbin particular footer appears to have an empty newline in it16:07
clarkbcorvus: yes16:07
corvus(i just dug into my archives and found an example of that)16:08
clarkbcorvus: basically this is only a regression for -discuss where we do not set footer or header16:08
clarkbfor all the lists where we set footer this was existing behavior16:08
corvusclarkb: ack, so we're narrowing the field16:08
clarkbnote that it only affects text/plain emails not multipart emails16:08
corvusyeah, i observe that in the archives too16:08
corvus(mail from fungi is never base64 encoded because it's always multipart :)16:09
clarkbyup I think I've narrowed it down to footer being '\n' and not '' once processed by decorate so the short circuit return I pasted above never hits16:09
clarkbI don't know why that is and I'm going to add a bit more debugging to actually be sure of that, but ya16:09
*** trown is now known as trown|lunch16:10
fungiwell, mail from me is *usually* multipart. i sometimes don't sign my messages but usually if i'm sending to a public mailing list i do16:12
openstackgerritMerged openstack-infra/nodepool master: Fix for image build leaks
*** artom has quit IRC16:12
*** imacdonn has quit IRC16:12
clarkbcurrently trying to sort out what additional debugging would be useful to avoid spamming yall with more emails than encessary16:12
*** jamesmcarthur has joined #openstack-infra16:12
fungii think the zuul lists may also not set footer/header? so would be a regression for them as well16:13
*** imacdonn has joined #openstack-infra16:13
corvusclarkb: i don't mind the spam16:13
fungisame, i get vastly more actual spam anyway16:13
clarkbcorvus: ok I won't worry too much about it then16:13
*** jpena is now known as jpena|off16:14
*** jamesmcarthur has quit IRC16:15
*** gfidente is now known as gfidente|afk16:17
*** michael-beaver has joined #openstack-infra16:17
*** udesale has quit IRC16:19
*** michael-beaver has left #openstack-infra16:20
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Use the buildset registry in zuul-quick-start
clarkbok my debug logging says msg_footer is '\r\n'16:22
clarkbthe value set in the config is ''16:22
*** jamesmcarthur has joined #openstack-infra16:23
clarkbthat gets decorated down to '\n' whcih is not a false value in python so we go through the conversion16:23
clarkbfinally we know what is causing the issue. Now to understand why16:23
corvusclarkb: when you say "the value set in the config is ''"  which config?16:23
clarkbcorvus: the value I get from config_list command is msgfooter = ''16:24
clarkber msg_footer = ''16:24
*** dtantsur is now known as dtantsur|afk16:24
*** kopecmartin is now known as kopecmartin|off16:25
clarkbcorvus: However I added debug logging to /usr/lib/mailman/Mailman/Handlers/ that syslog('error', 'clarkb decortate msg_footer raw %s', repr(mlist.msg_footer)) and I get '\r\n' in the error log16:25
openstackgerritJason Lee proposed openstack-infra/storyboard master: WORK IN PROGRESS: Added script for webscraping Launchpad Blueprint pages, generating a list of blueprints in a project.
clarkbfungi: your test emails had trailing newlines right? One thing I don't think mine have done is edned with a newline and I wouldn't be surprised if mailman thought it had to add them16:26
clarkbbut I probably shouldn't worry about testing that until after I've reviewed the config processing for msg_footer16:27
fungiclarkb: i think your test emails will always have trailing newlines because you have to end the message with . on a line by itself16:27
*** jpich has quit IRC16:27
clarkbok so smtp isn't filtering out that newline and . just the .16:28
fungii mean, technically it's up to exim, but pretty sure it's passing through the trailing newline16:29
*** ykarel|away has quit IRC16:30
*** yamamoto has quit IRC16:30
*** psachin has quit IRC16:34
openstackgerritMerged openstack-infra/zuul-sphinx master: Run opendev-release-python on release
*** ginopc has quit IRC16:41
*** rh-jelabarre has quit IRC16:44
*** spsurya has quit IRC16:46
clarkbok that last email did what we wanted it to16:48
clarkbThis must've been a serialization bug all along from the first time I removed the footer16:48
clarkbmaybe I left a newline in the input box when saved?16:48
clarkbin any case I added a footer back then removed it and now things work as expected16:49
clarkbugh so probably pebcak all along16:49
*** rh-jelabarre has joined #openstack-infra16:49
clarkbwhat is weird is that config_list showed ''16:49
clarkbbut when I looked at the pickle file directly there was definitely a ^M in there16:49
clarkbI think this means that if we observe this issue in the future we have to reset the footer carefully16:50
clarkband otherwise this should work as before and now I may need to go do something other than computer for a few minutes16:50
Shrewsinfra-root: any objections to me restarting nodepool-builder on nb01 to validate the new leak fix?16:51
*** e0ne has quit IRC16:51
clarkbShrews: not from me16:51
corvusShrews: ++16:52
clarkbon the upside I think I know how mailman works now which might be worth something16:52
*** jamesmcarthur has quit IRC16:52
clarkbif there is a bug here it is in config_list showing the '\r\n' as ''16:52
clarkbthat might be worth a report if I can debug it a bit more16:53
corvusclarkb: wow!  i'm glad we understand that now16:53
corvusit sounds like the upgrade itself should be a non issue, but i'm glad we confirmed that; if that had been a behavior change it would have been an unpleasant surprise16:53
clarkbso that leaves us with how do we want to handle the dmarc config settings16:53
*** helenafm has quit IRC16:54
clarkbwhich in my debugging of the other issues I think I found that if we remove WrapMessage from the global pipeline that some of thsoe functions end up being noops16:54
fungiShrews: thanks! i say go for it16:54
clarkbso people could click on the shiny buttons but they won't do anything16:54
fungiclarkb: that's excellent news16:54
clarkbcorvus: fungi ^ maybe you can take a look at that idea and see if I've missed anything?16:54
clarkband now I am going to take a short break16:55
*** jamesmcarthur has joined #openstack-infra16:56
*** derekh has quit IRC16:56
Shrewsi suppose i might as well restart all builders16:57
*** wolverineav has joined #openstack-infra16:57
mordredShrews: since you're restarting nodepool-builders - it's a good moment to remind people that we cut an sdk release with taskmanager removal in it and might want to land this: until such a time as we have the other bits, including possibly the filter thing you were working on, sorted16:58
mordredcorvus: ^^16:58
fungiclarkb: msg['From'] = msgdata.get('add_header').get('From') and so on are how it seems to be getting the new versions of those headers and replacing them, so looks like it will take care of skipping that too if we omit it from the pipeline16:58
*** rkukura has joined #openstack-infra16:59
*** trown|lunch is now known as trown17:00
Shrewsmordred: do you think we should have that in before restarting the builders (too late, already did, but can be undone)?17:01
mordredShrews: no - I don't think it matters for the builders a ton. I do think we should do it before we restart launchers just to be safe - and also because of the statsd metrics17:03
Shrewsmordred: +3d17:06
mordredShrews: thanks!17:06
*** artom has joined #openstack-infra17:14
*** e0ne has joined #openstack-infra17:14
*** e0ne has quit IRC17:15
*** yamamoto has joined #openstack-infra17:22
Shrews#status log Nodepool builders restarted with new code for cleaning up leaked DIB builds17:22
openstackstatusShrews: finished logging17:22
fungithat's exciting17:22
openstackgerritArtom Lifshitz proposed openstack-infra/project-config master: Add whitebox-tempest-plugin project
Shrewsfungi: it may take a while to actually see it in action. will watch the logs for a couple of days17:24
*** spsurya has joined #openstack-infra17:28
openstackgerritMerged openstack-infra/nodepool master: Pin openstacsksdk to <0.27.0
clarkbok back17:29
*** yamamoto has quit IRC17:32
*** yamamoto has joined #openstack-infra17:33
*** jamesmcarthur has quit IRC17:35
clarkbfungi: reading more I think removing WrapMessage will only handle the wrap option not the munge option17:36
clarkbstill not quite sure how munge is processed17:36
*** ijw has joined #openstack-infra17:40
clarkbfungi: corvus we could use a patched SpamDetect that seems to be the handler that does the first step in handing dmarc_moderation_action. For munge and wrap it sets from_is_list appropriately for subsequent handlers to process things17:44
clarkbCleanseDKIM is the other place where munge is handled along with WrapMessage which handles wrap and munge17:45
*** dpawlik has quit IRC17:45
clarkbI think removing WrapMessage gets us 90% of the way there17:46
clarkbfungi: kata is using munge not wrap right? and thats the bit where we'd 10% fail17:46
clarkbbut maybe we can tell people "it doesn't work sorry" :)17:46
*** apetrich has quit IRC17:46
*** electrofelix has quit IRC17:50
*** deray has joined #openstack-infra17:53
clarkbok infra-root I've written down dmarc_moderation_action handling options at under bullet 5.2.6. Let me know what you think and if you have other ideas17:54
*** wolverineav has quit IRC17:56
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Use a more visible selection color
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Use a more visible selection color
*** wolverineav has joined #openstack-infra17:57
*** yamamoto has quit IRC18:00
derayI need some advice regarding oslo_concurrency/fasteners usage. My objective is to limit the number (some configurable no.) of short-lived processes18:03
deray(doing some operation) by using soem kind of a smart file lock (fcntl.lockf) which can have not just one process to exclusively acquire the lock but18:03
derayactually a specified no. (just like the concept of counting semaphore). Is that readily available out of the box or I need to handcraft it?18:03
fungideray: i'm not sure this channel is the one you're looking for18:04
fungithis is where we run services to support the openstack community (code review system, continuous integration, mailing list servers, et cetera)18:05
fungimaybe you want #openstack-oslo?18:05
derayfungi, ok.. then could you tell me which channel I can ask this18:05
derayfungi, ok sure thx18:05
clarkbalrgith I'm going to context switch out of mailman things now that I think we have a good handle on stuff and just have to think through dmarc moderation action policy18:08
clarkbgoing to handle over the nimble pypi package now18:08
*** bhavikdbavishi has quit IRC18:08
*** wolverineav has quit IRC18:09
*** wolverineav has joined #openstack-infra18:10
clarkbinfra-root I just killed a gpg-agent on bridge. Don't forget to use `edit-secrets`18:11
fungiwhat's going on with the nimble package?18:12
clarkbfungi: thats the package that we are transfering ownership for on pypi18:13
clarkb(I made metnion of it at 15:24UTC today)18:13
fungioh, yes that one18:13
clarkbyup and now it is done18:13
fungii had already forgotten that's what it was named18:13
fungiso much switching, so little context18:14
clarkb#status log Added PyPi user 'spencer' as owner on PyPi package 'nimble' as we werent using the name and were asked nicely to give it to another group.18:14
openstackstatusclarkb: finished logging18:14
*** igordc has quit IRC18:14
anteayafungi: I'd like that on a t-shirt18:14
fungithat's my new career: selling shirts with glib phrases out of the back of a van in concert parking lots18:16
anteayasounds like a stable choice18:17
anteayashould be able to withstand climate change18:17
*** jamesmcarthur has joined #openstack-infra18:17
fungii'll just count on shorter sleeves gaining popularity18:18
anteayasounds like it should take18:19
clarkbtheir website doesn't seem to be working properly but inspirobot is a bot that generates motivation poster quotes18:20
clarkbyou need the fungi version of that stamping out tshirts then you can retire18:20
anteayasounds awesome18:21
*** jamesmcarthur has quit IRC18:21
*** samueldmq has joined #openstack-infra18:25
*** jamesmcarthur has joined #openstack-infra18:31
*** deray has quit IRC18:33
*** yamamoto has joined #openstack-infra18:37
*** ricolin has quit IRC18:38
*** yamamoto has quit IRC18:45
*** e0ne has joined #openstack-infra18:48
*** igordc has joined #openstack-infra18:50
*** nicolasbock has joined #openstack-infra18:51
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Remove custom key handler from xterm
*** sshnaidm is now known as sshnaidm|off19:00
*** raissa has joined #openstack-infra19:02
*** tbachman has quit IRC19:08
*** tbachman has joined #openstack-infra19:10
*** wolverineav has quit IRC19:10
*** wolverineav has joined #openstack-infra19:10
*** wolverineav has quit IRC19:15
*** ijw has quit IRC19:20
*** rkukura has quit IRC19:21
*** e0ne has quit IRC19:30
*** sshnaidm_ has joined #openstack-infra19:36
*** sshnaidm|off has quit IRC19:36
slaweqclarkb: hi19:37
slaweqclarkb: frickler hold for me nodes, and today19:37
*** eharney has quit IRC19:37
slaweqas he is probably not available now, You can clean them19:37
slaweqclarkb: frickler thx for help19:37
clarkbslaweq: thanks for letting us know. I'll get those cleaned up shortly19:37
slaweqThank You!19:38
*** krtaylor has joined #openstack-infra19:38
*** rkukura has joined #openstack-infra19:43
*** tosky has quit IRC19:44
openstackgerritJames E. Blair proposed opendev/base-jobs master: Actually upload to pypi in the python release job
openstackgerritMerged openstack-infra/project-config master: Add whitebox-tempest-plugin project
*** jamesmcarthur has quit IRC19:55
*** wolverineav has joined #openstack-infra19:58
*** raissa has quit IRC19:58
*** mriedem has quit IRC20:01
corvusfungi: mind pushing through when you have a sec?20:02
*** mriedem has joined #openstack-infra20:06
*** bobh has joined #openstack-infra20:08
*** mrhillsman is now known as mrhillsman_bbiab20:11
*** igordc has quit IRC20:18
fungiwill do. just gave up waiting on the islands to reconnect to the 'net and pulled out my travel cell modem20:18
anteayafungi are you getting wind?20:20
anteayawind knocked a tree over at my place20:20
funginope, all the wind from the nor'easter ended overnight20:21
anteayayay, not due to wind for you then20:21
funginow it's pleasant and sunny out and the internet decided it was a good day to go surfin'20:21
anteayawell I don't blame it20:21
anteayapleasant and sunny sounds great to me20:21
artomShould I be worried about the infra-prod-zuul_reconfigure failure on
*** kgiusti has quit IRC20:23
clarkbartom: no20:24
clarkbartom: that is an in progress set of hcnages and odesn't work yet. The cron job shoudl pick things up20:24
artomclarkb, ack, thanks :)20:25
Shrewsmordred: when you not in the air and back in front of a computer, i'm seeing some weirdness from openstacksdk on nb03 (not nb01 or nb02 though)20:29
*** whoami-rajat has quit IRC20:29
Shrewsmordred: mostly around cert failures for linaro20:29
ianwShrews: we have a fake cert there20:30
openstackgerritMerged opendev/base-jobs master: Actually upload to pypi in the python release job
ianwShrews: it was due to it originally being in .cn and the only explanation we had was a weird transparent proxy; switching to ssl fixed things.  however, it is now running in the lon1 cloud20:31
Shrewsmordred: fyi ^^^. thx ianw20:31
ianwand with LE work, we should "soon" :) be able to get certs (and i guess, switch them to .opendev names)20:31
*** bgmccollum has joined #openstack-infra20:32
*** jamesmcarthur has joined #openstack-infra20:33
Shrewsthis is probably unrelated to the release of openstacksdk. looks like we're still running 0.23.0 there (according to pip3 freeze)20:35
Shrewsmordred: ianw: ok, yeah. this has been going on for a while according to logs20:36
*** gfidente|afk has quit IRC20:37
ianwShrews: sorry i'm still not caffeinated; what is going on?20:37
Shrewsi guess we haven't been uploading images to linaro for a bit. not sure how to correct it20:37
*** bgmccollum has quit IRC20:38
Shrewsianw: i restarted the builders today for the image leak fix. was just monitoring the logs and noticed this (we also had an openstacksdk release recently too)20:38
*** bgmccollum has joined #openstack-infra20:39
Shrewsbut doesn't look like we have the latest sdk there, so ruling that one out20:39
ianwShrews: hrm, so the daily builds have been ok, but not the uploads huh?20:39
*** jamesmcarthur has quit IRC20:39
Shrewsianw: looks that way from the logs20:39
*** jamesmcarthur has joined #openstack-infra20:41
Shrewsianw: also, after much more caffeine, your input has been requested on  :)20:41
mordredShrews, ianw we should be able to handle fake certs either by putting the ca/cert info into clouds.yaml or putting insecure: true20:43
mordredif needed20:43
mordredif we're doing that and it's still complaining, we should definitely make sure there's not an sdk bug20:44
*** rcernin has joined #openstack-infra20:44
ianwoh these were for http only though; the linaro cert is ok, and we don't do tls with zk so i think we're ok20:46
*** artom has quit IRC20:47
Shrewsi don't see the 'insecure' option in clouds.yaml on nb03 or a cert. not sure if something changed recently there or not20:48
*** wolverineav has quit IRC20:49
clarkbthe nb03 cert was for the webserver it shouldn't affect nodepool's builder20:49
clarkbthose two things don't communicate20:49
*** trown is now known as trown|outtypewww20:52
* Shrews needs to go pick up his jeep before the shop closes20:53
*** spsurya has quit IRC20:59
*** igordc has joined #openstack-infra20:59
openstackgerritMonty Taylor proposed openstack-infra/zuul-website master: Add note to community page about code of conduct
*** harlowja has quit IRC21:06
*** pcaruana has quit IRC21:06
*** owalsh has quit IRC21:07
openstackgerritMerged openstack-infra/zuul-website master: Slide presentation for Zuul's OIP Confirmation
openstackgerritMerged openstack-infra/zuul-website master: Add note to community page about code of conduct
*** owalsh has joined #openstack-infra21:16
*** bobh has quit IRC21:17
*** harlowja has joined #openstack-infra21:19
*** eharney has joined #openstack-infra21:21
zigoianw: I was thinking, would it be ok if I was to build the latest debootstrap from Buster in Bionic ?21:25
zigoclarkb: Your thoughts?21:27
ianwzigo: yeah, i mean for testing i just cloned from salsa and dpkg-buildpackaged and it was fine; it's not dependency heavy21:27
clarkbzigo: ianw I think I am missing some context, is bionic debootstrap unable to bootstrap a buster image?21:28
zigoianw: So, can I just build it in sbuild, and then we get it to the Infra's PPA ?21:28
fungiclarkb: yeah, it'd just outdated basically21:28
fungier, it's21:28
zigoclarkb: Right, due to a bug fixed in Debian but not backported to Ubuntu, basically...21:28
clarkbI see. I personally prefer to avoid building packages ourselves though we do this for the vhd image conversion tool. PPAs also don't work quite right in our existing tooling with respect to package updates21:29
ianwzigo: yeah, it's not hard to get into the ppa; we just have to carry the updated package though in nodepool builder puppet, the nodepool build jobs and dib21:29
ianwall those internal touch points for us are the annoying bit; so fixing upstream if we can is preferred ...21:30
clarkbwe have to manually update PPA'd packages or change our unattended upgrade policy21:30
ianwbut yeah, i can also understand that in these LTS releases putting in changes that come with changelogs "reopen <list of bugs>" is probably not that great21:30
zigoRight, though I don't have any rights to upload anything to Ubuntu ...21:32
zigoOk, package built, now what? :)21:32
ianwwell should we ask ubuntu about updating before we go hacking it in locally?21:34
clarkbgetting the fix into ubuntu is definitely the best option21:34
clarkbassuming it can be done21:34
zigoianw: It's going to take ages ...21:34
zigoIf they even accept.21:34
zigo <--- I uploaded it here.21:35
ianwzigo: well at any rate, i'd like to see buster booting in before we add it to the production builders21:36
*** slaweq has quit IRC21:38
zigoianw: Sorry, I'm not following... what can I do to get things to a further step? :)21:39
ianwzigo: that change will test building a buster image, uploading it to a cloud, booting it with glean, talking to it and growing the root disk; so basically if that doesn't work, it's not going to work in production anyway21:40
clarkbzigo: ianw: can probably build debootstrap or install it from the url above in the job as a confirmation that fixes the entire process?21:40
mordredclarkb: jamespage was offering earlier to help us get the fix in to ubuntu21:41
ianwclarkb: sure, as initial hack we can pull it in from anywhere21:41
ianwbut then obviously that needs to be rolled out to production hosts before the image will build21:41
clarkbya that way we can see if there are any other issues to correct21:41
clarkbseems like a good next step21:41
clarkband can run concurrently with fixing ubuntu21:41
ianwi can make a ppa of it under
*** rcernin has quit IRC21:42
zigoAlso, if there are any issues to fix in Buster itself, now is a good time to get these fixes in (as it's in freeze, we can still fix stuff...).21:43
zigoianw: That's be great!21:43
zigoThanks for all the help.21:43
ianwi've uploaded to and i guess it's processing21:53
*** slaweq has joined #openstack-infra21:55
*** slaweq has quit IRC22:00
ianw... so back to nb03 for a bit ...22:03
ianwssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645) ... so this *is* ssl connection to the linaro api22:04
clarkbdid the cert expire maybe?22:05
clarkbor did they not have a cert signed by a CA we trust?22:05
ianwit *was* working at one point ...22:06
*** wolverineav has joined #openstack-infra22:06
*** rlandy|ruck is now known as rlandy22:08
*** weshay is now known as weshay|ruck22:08
*** rlandy is now known as rlandy|bbl22:09
ianwok, it fails to chat on bridge.o.o too; i wish the error gave you a clue as to what was wrong :/22:10
*** kjackal has quit IRC22:11
clarkbianw: openssl s_client is good for checking that stuff22:11
*** wolverineav has quit IRC22:12
ianw    Verify return code: 10 (certificate has expired)22:13
ianwMar 18 20:33:33 2019 GMT22:13
ianwit's actually a LE cert22:13
*** jamesmcarthur has quit IRC22:14
ianwi guess we can either set it to be ignored, or disable the cn1 cloud22:15
clarkboh is it cn1?22:16
clarkbianw: niedbalski pinged me when I was out last week and I mentioend that you could help with decommissioning cn122:16
clarkbcn1 is being turned off in favor of lon1 aiui22:16
clarkbso I think maybe this is expected?22:16
ianwor at least keys not being renewed.  well let's remove it then.  i think lon1 is fine22:24
*** artom has joined #openstack-infra22:27
clarkbalright I've been spinning my wheels for most of the day since sorting out the mailman thing. I think I'm going to pop out a bit early now that the zuul stuff I wanted to make sure was set is done.22:28
clarkbcorvus: if you can weigh in on the dmarc/dkim stuff before you take your vacation that would be great. I'd like to try scheduling the server upgrade around the opendev work if possible and want to be prepared for dmarc changes when I do so22:29
*** jamesmcarthur has joined #openstack-infra22:32
*** jamesmcarthur has quit IRC22:32
*** bobh has joined #openstack-infra22:33
ianw++ i need to run an errand to get a cat6 inline coupler ... bib22:34
*** nicolasbock has quit IRC22:34
*** yamamoto has joined #openstack-infra22:43
*** yamamoto has quit IRC22:47
*** bobh has quit IRC22:47
*** yamamoto has joined #openstack-infra23:00
*** lathiat_ is now known as lathiat23:02
*** wolverineav has joined #openstack-infra23:02
*** tkajinam has joined #openstack-infra23:04
*** slaweq has joined #openstack-infra23:06
*** wolverineav has quit IRC23:06
*** slaweq has quit IRC23:10
openstackgerritMerged openstack-infra/zuul master: Fail jobs when requirements not met
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add Debian Buster boot tests
*** hwoarang has quit IRC23:50
*** hwoarang has joined #openstack-infra23:51
*** lseki has quit IRC23:53
*** wolverineav has joined #openstack-infra23:56
*** tbachman has quit IRC23:58

Generated by 2.15.3 by Marius Gedminas - find it at!