Sunday, 2019-01-20

*** martinkennelly has quit IRC00:02
*** wolverineav has joined #openstack-infra00:04
*** bgmccollum has quit IRC00:06
*** bgmccollum has joined #openstack-infra00:08
*** slaweq has joined #openstack-infra00:11
*** markvoelker has joined #openstack-infra00:15
*** slaweq has quit IRC00:24
*** bobh has joined #openstack-infra00:26
*** hongbin has quit IRC00:26
*** wolverineav has quit IRC00:26
*** eumel8 has quit IRC00:30
*** bobh has quit IRC00:30
*** wolverineav has joined #openstack-infra00:31
*** wolverineav has quit IRC00:35
*** smarcet has joined #openstack-infra00:46
*** markvoelker has quit IRC00:48
*** smarcet has quit IRC00:51
*** smarcet has joined #openstack-infra00:56
*** smarcet has quit IRC01:00
*** slaweq has joined #openstack-infra01:12
*** xek_ has joined #openstack-infra01:20
*** xek has quit IRC01:21
*** Adri2000 has quit IRC01:24
*** slaweq has quit IRC01:25
*** Adri2000 has joined #openstack-infra01:31
*** bobh has joined #openstack-infra01:36
*** smarcet has joined #openstack-infra01:37
*** smarcet has quit IRC01:41
*** markvoelker has joined #openstack-infra01:45
*** jamesmcarthur has joined #openstack-infra01:47
*** smarcet has joined #openstack-infra01:48
*** auristor has quit IRC01:49
*** jamesmcarthur has quit IRC01:51
*** auristor has joined #openstack-infra01:54
*** wolverineav has joined #openstack-infra01:54
*** smarcet has quit IRC01:58
*** wolverineav has quit IRC01:59
*** smarcet has joined #openstack-infra02:02
*** smarcet has left #openstack-infra02:03
*** slaweq has joined #openstack-infra02:11
*** markvoelker has quit IRC02:18
*** slaweq has quit IRC02:24
*** bobh has quit IRC02:29
*** zbitter has quit IRC02:33
*** zbitter has joined #openstack-infra02:35
*** zaneb has joined #openstack-infra02:37
*** zbitter has quit IRC02:40
*** zaneb has quit IRC02:41
*** wolverineav has joined #openstack-infra02:45
*** wolverineav has quit IRC03:00
*** wolverineav has joined #openstack-infra03:00
*** slaweq has joined #openstack-infra03:11
*** markvoelker has joined #openstack-infra03:16
*** slaweq has quit IRC03:25
*** wolverineav has quit IRC03:31
*** wolverineav has joined #openstack-infra03:32
*** hwoarang has quit IRC03:36
*** hwoarang has joined #openstack-infra03:36
*** efried has joined #openstack-infra03:37
*** fried_rice has quit IRC03:38
*** hwoarang has quit IRC03:44
*** markvoelker has quit IRC03:48
*** hwoarang has joined #openstack-infra03:49
*** wolverineav has quit IRC03:50
*** bobh has joined #openstack-infra03:56
*** jamesmcarthur has joined #openstack-infra04:03
*** jamesmcarthur has quit IRC04:07
*** jamesmcarthur_ has joined #openstack-infra04:08
*** slaweq has joined #openstack-infra04:16
*** Bhujay has joined #openstack-infra04:21
*** Bhujay has quit IRC04:22
*** Bhujay has joined #openstack-infra04:23
*** wolverineav has joined #openstack-infra04:24
*** slaweq has quit IRC04:24
*** ykarel has joined #openstack-infra04:28
*** jamesmcarthur_ has quit IRC04:34
*** wolverineav has quit IRC04:35
*** jamesmcarthur has joined #openstack-infra04:38
*** markvoelker has joined #openstack-infra04:45
*** jamesmcarthur has quit IRC04:46
*** ijw has joined #openstack-infra04:50
*** ijw_ has joined #openstack-infra04:51
*** ijw has quit IRC04:55
*** jamesmcarthur_ has joined #openstack-infra04:58
*** whoami-rajat has joined #openstack-infra05:01
*** jamesmcarthur_ has quit IRC05:02
*** jamesmcarthur has joined #openstack-infra05:10
*** slaweq has joined #openstack-infra05:12
*** ykarel has quit IRC05:14
*** jamesmcarthur has quit IRC05:14
*** markvoelker has quit IRC05:18
*** slaweq has quit IRC05:24
*** hwoarang has quit IRC05:37
*** Bhujay has quit IRC05:37
*** hwoarang has joined #openstack-infra05:37
*** Bhujay has joined #openstack-infra06:08
*** Bhujay has joined #openstack-infra06:10
*** hwoarang has quit IRC06:10
*** hwoarang has joined #openstack-infra06:11
*** slaweq has joined #openstack-infra06:12
*** wolverineav has joined #openstack-infra06:15
*** wolverineav has quit IRC06:22
*** slaweq has quit IRC06:25
*** ijw has joined #openstack-infra06:25
*** ykarel has joined #openstack-infra06:25
*** ijw_ has quit IRC06:29
*** lpetrut has joined #openstack-infra06:54
*** wolverineav has joined #openstack-infra07:01
*** slaweq has joined #openstack-infra07:01
*** jamesmcarthur has joined #openstack-infra07:03
*** wolverineav has quit IRC07:05
*** jamesmcarthur has quit IRC07:08
*** slaweq has quit IRC07:24
*** pcaruana has joined #openstack-infra07:29
*** markvoelker has joined #openstack-infra07:45
*** lpetrut has quit IRC07:57
*** ijw has quit IRC07:57
*** ijw has joined #openstack-infra07:58
*** ijw has quit IRC08:01
*** ijw has joined #openstack-infra08:02
*** slaweq has joined #openstack-infra08:11
*** wolverineav has joined #openstack-infra08:13
*** wolverineav has quit IRC08:17
*** markvoelker has quit IRC08:19
*** jbadiapa has joined #openstack-infra08:20
*** slaweq has quit IRC08:25
*** whoami-rajat has quit IRC08:51
*** slaweq has joined #openstack-infra09:03
*** jamesmcarthur has joined #openstack-infra09:05
*** jamesmcarthur has quit IRC09:09
*** ykarel has quit IRC09:10
*** Bhujay has quit IRC09:11
*** Bhujay has joined #openstack-infra09:12
*** markvoelker has joined #openstack-infra09:16
*** Bhujay has quit IRC09:32
*** Bhujay has joined #openstack-infra09:33
*** ssbarnea|bkp2 has joined #openstack-infra09:38
*** ssbarnea|rover has quit IRC09:41
*** markvoelker has quit IRC09:48
*** zaneb has joined #openstack-infra09:54
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Retry jobs on winrm send_input failed  https://review.openstack.org/63196409:59
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Manage ansible installation within zuul  https://review.openstack.org/63193010:07
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Manage ansible installation within zuul  https://review.openstack.org/63193010:19
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Retry jobs on winrm send_input failed  https://review.openstack.org/63196410:23
*** markvoelker has joined #openstack-infra10:45
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Manage ansible installation within zuul  https://review.openstack.org/63193010:46
*** Bhujay has quit IRC10:50
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Manage ansible installation within zuul  https://review.openstack.org/63193010:53
*** ykarel has joined #openstack-infra10:58
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Manage ansible installation within zuul  https://review.openstack.org/63193010:59
*** jamesmcarthur has joined #openstack-infra11:05
*** jamesmcarthur has quit IRC11:10
*** markvoelker has quit IRC11:19
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Manage ansible installation within zuul  https://review.openstack.org/63193011:20
*** Bhujay has joined #openstack-infra11:32
*** ykarel has quit IRC11:33
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test  https://review.openstack.org/63197011:34
*** whoami-rajat has joined #openstack-infra11:34
*** wolverineav has joined #openstack-infra11:49
*** wolverineav has quit IRC11:53
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test  https://review.openstack.org/63197011:58
*** tosky has joined #openstack-infra12:07
*** markvoelker has joined #openstack-infra12:16
*** wolverineav has joined #openstack-infra12:31
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test  https://review.openstack.org/63197012:35
*** wolverineav has quit IRC12:36
*** markvoelker has quit IRC12:48
*** dkehn has quit IRC12:57
*** jamesmcarthur has joined #openstack-infra13:06
*** jamesmcarthur has quit IRC13:11
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test  https://review.openstack.org/63197013:14
*** wolverineav has joined #openstack-infra13:30
*** e0ne has joined #openstack-infra13:38
*** wolverineav has quit IRC13:39
*** ssbarnea|rover has joined #openstack-infra13:41
*** ssbarnea|bkp2 has quit IRC13:42
*** markvoelker has joined #openstack-infra13:45
*** ykarel has joined #openstack-infra14:13
*** markvoelker has quit IRC14:19
*** e0ne has quit IRC14:22
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test  https://review.openstack.org/63197014:55
*** xek_ has quit IRC15:06
*** jamesmcarthur has joined #openstack-infra15:07
fungiclarkb: were you doing the ssl cert for (www.)zuul-ci.org? "zuul-ci.org uses an invalid security certificate. The certificate expired on 19 January 2019, 23.59.59 GMT."15:11
*** jamesmcarthur has quit IRC15:11
fungii guess we never added that to our certcheck list15:11
fungioh, huh it's on there15:13
fungihttps://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/ssl_cert_check/ssldomains#n2315:14
*** markvoelker has joined #openstack-infra15:16
fungiokay, that's weird... `ssl-cert-check -f /var/lib/certcheck/ssldomains -x 30` on cacti reports "zuul-ci.org:443 Valid Jun 22 2019"15:19
fungiohh...15:19
fungii bet ssl-cert-check doesn't support sni15:20
fungiso it's checking the default cert on files02.openstack.org:44315:20
fungiyep, according to https://metadata.ftp-master.debian.org/changelogs/main/s/ssl-cert-check/ssl-cert-check_3.30-2_changelog the 3.30-1 upload to debian included "debian/patches: debian/patches/0001-support-SNI-for-OpenSSL-1.1.0.patch" to address https://bugs.debian.org/76995215:23
openstackDebian bug 769952 in ssl-cert-check "ssl-cert-check: Please support SNI" [Normal,Fixed]15:23
*** ykarel has quit IRC15:24
fungiubuntu xenial provides ssl-cert-check 3.27-2, we need the version from bionic or later for sni15:24
fungiclarkb: i'll go ahead and get the renewal started using the previous csr for it on bridge.o.o15:30
*** tosky has quit IRC15:32
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Fix typo with openstack-dev/ci-sandbox exclude  https://review.openstack.org/63198415:42
fungiclarkb: for some reason namecheap couldn't parse the csr we had on record, so i'm just generating a new key entirely15:43
*** Bhujay has quit IRC15:47
*** Bhujay has joined #openstack-infra15:48
*** Bhujay has quit IRC15:49
*** markvoelker has quit IRC15:49
*** Bhujay has joined #openstack-infra15:49
openstackgerritJeremy Stanley proposed openstack-infra/zone-zuul-ci.org master: Add temporary cert renewal cname  https://review.openstack.org/63198615:50
openstackgerritJeremy Stanley proposed openstack-infra/zone-zuul-ci.org master: Remove temporary cert renewal cname  https://review.openstack.org/63198715:50
openstackgerritJeremy Stanley proposed openstack-infra/zone-zuul-ci.org master: Add temporary cert renewal cname  https://review.openstack.org/63198615:53
openstackgerritJeremy Stanley proposed openstack-infra/zone-zuul-ci.org master: Remove temporary cert renewal cname  https://review.openstack.org/63198715:53
fungiinfra-root: i'm self-approving 631986 now15:53
*** Bhujay has quit IRC15:54
openstackgerritMerged openstack-infra/zone-zuul-ci.org master: Add temporary cert renewal cname  https://review.openstack.org/63198615:54
fungiand also manually applying it to adns115:55
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Build a python-base image for use with python-builder  https://review.openstack.org/63198815:57
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Build zuul containers with dockerfile not pbrx  https://review.openstack.org/63184015:57
*** dkehn has joined #openstack-infra16:03
fungiokay, the cert/key/chain have been committed to the appropriate ansible group_vars, so going to use kick.sh now to update files01.o.o with them16:07
*** e0ne has joined #openstack-infra16:09
fungier, i mean files0216:10
fungiand i manually reloaded the apache2 service on there to get it to pick up the updated cert16:11
fungilooks like it's all working now16:11
fungii'll approve the temporary domain validation cname record removal now16:11
openstackgerritMerged openstack-infra/zone-zuul-ci.org master: Remove temporary cert renewal cname  https://review.openstack.org/63198716:12
fungi#status log replaced expired zuul-ci.org ssl cert (and associated key/chain) due to unanticipated expiration16:13
openstackstatusfungi: finished logging16:13
* fungi gets back to his coffee16:14
fungiinfra-root: how crazy of a solution is this... retire all the ssl-cert-check puppetry for the cacti server in favor of a periodic job for system-config which installs ssl-cert-check on a xenial node and runs through our ssldomains list? figuring out who/where to send e-mail is the tricky part16:22
fungier, on a bionic node i mean16:22
fungixenial's is what's too old16:22
fungiright now we e-mail root which gets delivered to the list of sysadmin addresses in private hiera16:23
clarkbfungi thanks. I dont think I did the original csr? Not surewhy the old one was unhappy16:25
clarkbas for certchecking we could also just s_client in a for loop amd grep the expire date? I guess there is moreto check than that16:26
fungii guess once we get certbot going, if we tune the ssl-cert-check warning to something comfortably less than the renewal interval, the messages from it would be infrequent (i.e., only if certbot broke)16:26
fungiclarkb: it's possible it decided the csr was too old or something, but it didn't really say other than just claiming it couldn't be parsed (i tried copying and pasting a few different ways before i gave up and redid it all from scratch)16:28
AJaegerfungi, an ugly way: make it a test for system-config - and have it block if less than 15 days left. That forces us to merge a fix ;)16:29
AJaegerfungi, thanks for fixing!16:29
fungiheh, yeah that would be amusing in an really annoying sort of way16:30
AJaegeryeah ;(16:30
*** jamesmcarthur has joined #openstack-infra16:30
AJaegerA daily check with a mail that is not filtered away like you propose could work - mailing to openstack-infra? We could even send that mail always on the first of a month - as a heads-up - and then daily if any certificate less than X days...16:31
fungiAJaeger: actually, yeah, while the tool itself isn't that smart, with some parsing and math we could have it not send that every day; the bigger concern is that there's not a lot of point in sending that notification to (and unnecessarily annoying) lots of people who can't do anything about it16:34
*** gouthamr has quit IRC16:34
*** jamesmcarthur has quit IRC16:35
fungianyway, good ideas to mull over. i've got packing and pre-travel chores to get back to16:35
*** dmellado has quit IRC16:36
pabelangerfungi: I don't think it is crazy, zuul job for ssldomain. The part that we'd need to figure out is notification step.16:37
clarkbpabelanger: we can use zuul email reporter on failure16:40
clarkbthen fail job if a cert expires soon16:40
*** wolverineav has joined #openstack-infra16:43
fungiwell, we also have jobs that send to mailing lists directly (release announcements)16:46
*** markvoelker has joined #openstack-infra16:46
*** e0ne has quit IRC16:47
*** whoami-rajat has quit IRC16:47
*** wolverineav has quit IRC16:48
*** e0ne has joined #openstack-infra16:51
pabelangerclarkb: yah, for that we'd need a specific pipeline, but agree16:54
pabelangerI think it is worth experimenting16:54
pabelangerclarkb: fungi: if still around, and up for a quick review: https://review.openstack.org/631984/ fixes a typo with recent opentack-dev/ci-sandbox change to exclude pipelines16:58
fungiwhoops, yeah16:59
fungithat wasn't valid yaml, huh16:59
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/nodepool master: Amazon EC2 driver  https://review.openstack.org/53555817:00
fungi#status log switched storyboard-dev.o.o to using a local database service instead of trove17:06
openstackstatusfungi: finished logging17:06
openstackgerritMerged openstack-infra/project-config master: Fix typo with openstack-dev/ci-sandbox exclude  https://review.openstack.org/63198417:06
*** ykarel has joined #openstack-infra17:15
*** markvoelker has quit IRC17:18
*** bobh has quit IRC17:26
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/project-config master: Add gate job for Slack notifier in zuul-jobs  https://review.openstack.org/62359317:28
* fungi grumbles at rackspace's trove service not being listed in the keystone catalog in any way openstacksdk seems to find17:28
fungi"publicURL endpoint for database service not found"17:28
openstackgerritMerged openstack-infra/system-config master: Switch StoryBoard database backups to local  https://review.openstack.org/62329117:31
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul-jobs master: Add a slack-notify role  https://review.openstack.org/62359417:34
*** ykarel has quit IRC17:37
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/project-config master: Add gate job for Slack notifier in zuul-jobs  https://review.openstack.org/62359317:44
*** bobh has joined #openstack-infra17:57
*** bobh has quit IRC17:57
*** bobh has joined #openstack-infra17:57
*** bobh has quit IRC18:02
openstackgerritJeremy Stanley proposed openstack-infra/system-config master: Prepare for storyboard server replacement  https://review.openstack.org/63198918:08
*** rtjure has quit IRC18:15
*** markvoelker has joined #openstack-infra18:16
*** rtjure has joined #openstack-infra18:20
*** e0ne has quit IRC18:20
*** e0ne has joined #openstack-infra18:21
*** tosky has joined #openstack-infra18:22
*** e0ne has quit IRC18:22
*** gouthamr has joined #openstack-infra18:29
*** e0ne has joined #openstack-infra18:32
*** e0ne has quit IRC18:33
*** wolverineav has joined #openstack-infra18:34
*** dmellado has joined #openstack-infra18:34
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Test coverage to exclude pipeline with untrusted project  https://review.openstack.org/63199018:36
*** wolverineav has quit IRC18:38
*** stevebaker has joined #openstack-infra18:44
*** e0ne has joined #openstack-infra18:49
*** markvoelker has quit IRC18:50
*** spsurya has joined #openstack-infra19:01
*** ijw has quit IRC19:39
*** ijw_ has joined #openstack-infra19:39
*** ijw has joined #openstack-infra19:41
*** ijw_ has quit IRC19:43
*** markvoelker has joined #openstack-infra19:46
*** ijw has quit IRC19:52
*** xek_ has joined #openstack-infra20:04
*** ijw has joined #openstack-infra20:15
*** markvoelker has quit IRC20:19
*** xek_ has quit IRC20:48
*** xek_ has joined #openstack-infra20:48
*** wolverineav has joined #openstack-infra20:49
*** ijw_ has joined #openstack-infra20:51
*** wolverineav has quit IRC20:53
*** ijw__ has joined #openstack-infra20:54
*** ijw has quit IRC20:54
*** ijw_ has quit IRC20:56
*** wolverineav has joined #openstack-infra20:56
*** verdurin has quit IRC20:57
*** wolverineav has quit IRC21:01
*** bobh has joined #openstack-infra21:06
*** e0ne has quit IRC21:14
*** markvoelker has joined #openstack-infra21:16
*** xek_ has quit IRC21:18
*** pcaruana has quit IRC21:25
*** spsurya has quit IRC21:27
*** armstrong has joined #openstack-infra21:36
*** bobh has quit IRC21:49
*** markvoelker has quit IRC21:50
*** whoami-rajat has joined #openstack-infra21:55
*** yboaron has quit IRC21:55
*** martin_ken has quit IRC22:01
*** wolverineav has joined #openstack-infra22:07
*** wolverineav has quit IRC22:12
*** needssleep is now known as TheJulia22:36
*** yankcrime has quit IRC22:44
*** rcernin has joined #openstack-infra22:45
*** markvoelker has joined #openstack-infra22:46
*** yankcrime has joined #openstack-infra22:47
*** slaweq has quit IRC22:51
*** tkajinam has joined #openstack-infra23:03
*** yankcrime has quit IRC23:09
*** yankcrime has joined #openstack-infra23:11
*** slaweq has joined #openstack-infra23:16
*** markvoelker has quit IRC23:19
*** slaweq has quit IRC23:24
*** larainema has joined #openstack-infra23:28
openstackgerritMerged openstack/diskimage-builder master: [Core] Change openstack-dev to openstack-discuss.  https://review.openstack.org/62289523:41
*** ijw__ has quit IRC23:42
*** tosky has quit IRC23:46
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Set your-readthedocs-job-requires-attention host localhost  https://review.openstack.org/63200423:50
*** zxiiro-away is now known as zxiiro23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!