Saturday, 2018-08-18

clarkbfungi: logstash-worker08 failed to resolve git.openstack.org to pull down project-config00:00
clarkber not project-config, logstash-filters00:00
clarkbunbound is not running on it00:01
fungithat explains that at least00:01
fungialso git.openstack.org failed?00:01
fungiunreachable00:01
clarkbno unbound so no dns resolution00:02
clarkbtrying to start unbound doesn't seem to work00:02
clarkblooks like it died without systemd noticing so I had to stop start it00:03
clarkbdig works now00:03
fungioh, on git.o.o itself?00:03
clarkbno on logstash-worker0800:03
fungiahh00:03
clarkbI think I've resolved that problem by stop starting unbound00:04
fungialso git.openstack.org as marked unreachable by ansible though00:04
clarkboh00:04
clarkbfungi: it doesnt' have the updated ssh key with bridge's ip addr on it00:05
clarkbit isn't in the emergency file list so it must not have had puppet run there for some other reason00:05
clarkbwe will probably just need to update the key by hand and have ansible tell us what is wrong from there00:05
*** jamesmcarthur has joined #openstack-infra00:06
fungimaybe it's part of the git and gerrit playbook but that got run piecemeal and git.o.o ended up missed?00:06
fungioh, before then00:06
clarkbmordred: tl;dr to followup there is a non zero numbr of hosts that need that cleanup. Also we should figure out why ansible doesn't timeout to hosts like ethercalc02 whcih have the ssh key with the right ip addr but do not have the host key trusted on bridge.o.o00:06
clarkbfungi: ya puppet should've done this a few days back00:07
clarkbI've since fixed ethercalc02's host key issue on bridge.o.o00:07
clarkbwith that I think I am going to call it a week. Thanks again00:07
fungime too, for the most part. thanks!00:08
*** felipemonteiro has joined #openstack-infra00:12
*** rpioso is now known as rpioso|afk00:13
*** masayukig has joined #openstack-infra00:14
*** rosmaita has quit IRC00:18
*** felipemonteiro has quit IRC00:21
*** bobh has joined #openstack-infra00:26
*** bobh has quit IRC00:37
*** bobh has joined #openstack-infra00:38
*** mrsoul has joined #openstack-infra01:25
*** mschuppert has quit IRC01:28
*** armax has quit IRC01:51
*** jamesmcarthur has quit IRC02:02
*** bobh has quit IRC02:03
*** yamahata has quit IRC02:53
*** SteelyDan is now known as superdan02:58
*** dave-mccowan has quit IRC03:08
*** felipemonteiro has joined #openstack-infra03:50
*** jamesmcarthur has joined #openstack-infra04:02
*** jamesmcarthur has quit IRC04:07
*** felipemonteiro has quit IRC04:15
*** oanson has quit IRC04:26
*** yamahata has joined #openstack-infra04:30
*** ianychoi has quit IRC04:41
*** ianychoi has joined #openstack-infra04:47
*** Bhujay has joined #openstack-infra05:03
*** ykarel has joined #openstack-infra05:10
*** toabctl has quit IRC05:36
*** ykarel has quit IRC05:37
*** graphene has quit IRC06:02
*** graphene has joined #openstack-infra06:04
openstackgerritNguyen Hai proposed openstack-infra/project-config master: remove job settings for mistral repositories  https://review.openstack.org/59326406:06
*** e0ne has joined #openstack-infra06:24
*** jesusaur has quit IRC06:58
*** e0ne has quit IRC07:05
*** SteffanW has joined #openstack-infra07:34
*** apetrich has joined #openstack-infra07:42
*** raissa has joined #openstack-infra08:02
*** jistr is now known as jistr|off08:44
*** zhangfei has joined #openstack-infra09:12
*** slaweq has joined #openstack-infra09:44
*** raissa has quit IRC09:47
*** raissa has joined #openstack-infra09:47
*** raissa has quit IRC09:48
*** raissa has joined #openstack-infra09:48
*** raissa has joined #openstack-infra09:49
*** raissa has quit IRC09:49
*** raissa has joined #openstack-infra09:50
*** raissa has quit IRC09:50
*** raissa has joined #openstack-infra09:51
*** raissa has quit IRC09:51
*** raissa has joined #openstack-infra09:51
*** raissa has quit IRC09:52
*** raissa has joined #openstack-infra09:52
*** raissa has quit IRC09:52
*** raissa has joined #openstack-infra09:53
*** raissa has quit IRC09:53
*** raissa has joined #openstack-infra09:57
*** raissa has quit IRC10:08
*** raissa has joined #openstack-infra10:08
*** raissa has quit IRC10:11
*** raissa has joined #openstack-infra10:24
*** raissa has quit IRC10:24
*** raissa has joined #openstack-infra10:25
*** jesusaur has joined #openstack-infra10:26
*** apetrich has quit IRC10:42
*** slaweq has quit IRC11:06
*** dingyichen has quit IRC11:23
*** egonzalez has joined #openstack-infra11:49
egonzalezHi, ask.openstack.org is down11:50
*** SteffanW has quit IRC12:19
*** raissa has quit IRC12:28
*** raissa has joined #openstack-infra12:29
*** raissa has quit IRC12:29
*** raissa has joined #openstack-infra12:30
*** raissa has quit IRC12:30
*** raissa has joined #openstack-infra12:31
*** raissa has quit IRC12:31
*** roman_g_ has joined #openstack-infra12:31
*** raissa has joined #openstack-infra12:31
*** raissa has quit IRC12:32
*** raissa has joined #openstack-infra12:32
*** raissa has quit IRC12:32
*** raissa has joined #openstack-infra12:33
*** raissa has quit IRC12:33
*** raissa has joined #openstack-infra12:34
*** raissa has quit IRC12:34
*** rkukura has quit IRC12:36
*** roman_g_ has quit IRC12:42
fungiegonzalez: it's back up now. looks like apache died during log rotation leaving its pidfile behind. no corresponding errors in the apache log nor dmesg/syslog, and no coredump so not much of a trail to follow unfortunately12:58
fungi#status log removed stale pidfile and started apache on ask.o.o after it died silently during log rotation12:58
openstackstatusfungi: finished logging12:58
*** toabctl has joined #openstack-infra13:21
*** bobh has joined #openstack-infra13:25
*** pbourke has quit IRC14:01
*** pbourke has joined #openstack-infra14:03
*** slaweq has joined #openstack-infra14:11
*** graphene has quit IRC14:15
*** slaweq has quit IRC14:16
*** graphene has joined #openstack-infra14:17
*** dhill_ has quit IRC14:17
*** bobh has quit IRC14:24
*** jamesdenton has joined #openstack-infra14:34
*** bobh has joined #openstack-infra14:38
*** bobh has quit IRC14:38
*** bobh has joined #openstack-infra14:38
*** ianychoi has quit IRC14:54
*** zhangfei has quit IRC15:00
*** Bhujay has quit IRC15:05
*** felipemonteiro has joined #openstack-infra15:08
*** graphene has quit IRC15:09
*** graphene has joined #openstack-infra15:11
openstackgerritNguyen Hai proposed openstack-infra/project-config master: remove job settings for designate repositories  https://review.openstack.org/59286515:22
*** dave-mccowan has joined #openstack-infra15:25
openstackgerritNguyen Hai proposed openstack-infra/project-config master: remove job settings for designate repositories  https://review.openstack.org/59286515:26
*** dave-mccowan has quit IRC15:30
openstackgerritNguyen Hai proposed openstack-infra/project-config master: remove job settings for solum repositories  https://review.openstack.org/59292315:34
*** felipemonteiro has quit IRC15:42
*** ianychoi has joined #openstack-infra15:47
*** jamesdenton has quit IRC15:52
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Add IPv6 support for statsd  https://review.openstack.org/55632715:53
openstackgerritNguyen Hai proposed openstack-infra/project-config master: remove job settings for watcher repositories  https://review.openstack.org/59334016:07
*** bobh has quit IRC16:21
*** bobh has joined #openstack-infra16:36
*** bobh has quit IRC16:40
*** bobh has joined #openstack-infra16:45
*** felipemonteiro has joined #openstack-infra16:46
*** bobh has quit IRC16:49
*** felipemonteiro has quit IRC16:53
*** bobh has joined #openstack-infra16:57
*** bobh has quit IRC17:02
*** felipemonteiro has joined #openstack-infra17:02
*** bobh has joined #openstack-infra17:06
*** bobh has quit IRC17:10
*** felipemonteiro has quit IRC17:15
*** apetrich has joined #openstack-infra17:15
*** bobh has joined #openstack-infra17:17
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual master: Give examples for master only jobs  https://review.openstack.org/59334417:22
*** bobh has quit IRC17:22
*** bobh has joined #openstack-infra17:24
*** dave-mccowan has joined #openstack-infra17:24
AJaegerpabelanger: is the dox repo still used - or time to retire it?17:26
* fungi refreshes his memory re: dox17:26
*** felipemonteiro has joined #openstack-infra17:27
fungi"tool for using docker containers to run local tests" (last commit ~1.5 years ago)17:27
fungitox on docker i guess17:27
fungipabelanger: ^ you seem to be the last regular contributor on that17:28
*** bobh has quit IRC17:29
funginot an official deliverable repo of openstack17:29
*** armax has joined #openstack-infra17:30
*** felipemonteiro has quit IRC17:30
AJaegerfungi: there are lots of those with no commits for over a year - like fuel-ccp, astara. just stumpled upon dox...17:32
AJaegerif anybody wants to remove some infra-manual changes, please look at https://review.openstack.org/593344 and https://review.openstack.org/593138 . Those improve docs for the move to in-repo jobs.17:34
fungiskimming http://codesearch.openstack.org/?q=dox i don't think anything we're hosting is actively using it (on master branches at least)17:36
fungiAJaeger: yeah, i was lamenting in #openstack-tc last week that we cut fuel loose due to lack of activity rather than retiring it, so now a year or two later people are still trying to use it and finding out it's basically abandoned17:37
* fungi is working on a proposal to add the tc group from gerrit as a core reviewer to all projects we drop due to low activity17:38
fungibasically we need a compromise between forcing low-activity teams to retire projects and retiring projects which drop to no activity after we do that17:39
fungiso that we still have some recourse to retire them at that point17:39
AJaegerfungi: glad to see that you're on it.17:40
*** bobh has joined #openstack-infra17:40
* AJaeger just looked at openstack-zuul-jobs and some old jobs there - helping teams convert job is nice but not worth if those projects are dead ;(17:40
*** Bhujay has joined #openstack-infra17:40
fungicompletely agree17:41
* AJaeger wonders whether we should at least remove those jobs - far less jobs to maintain...17:41
AJaegerfungi: I look forward to your proposal! Thanks for looking into that.17:41
*** bobh has quit IRC17:44
fungian infra proposal to drop centrally-configured jobs which are used by projects which haven't merged a commit in over a year might be interesting, though i wonder how many of our own repos fall into that category17:44
fungithen again, i suppose most of those don't have unique jobs17:45
*** bobh has joined #openstack-infra17:46
AJaegerfungi: infra spec? Or proposal to TC?17:47
*** dave-mccowan has quit IRC17:48
*** bobh has quit IRC17:51
*** bauzas has quit IRC17:52
*** bobh has joined #openstack-infra17:54
*** bauzas has joined #openstack-infra17:54
*** graphene has quit IRC17:58
*** bobh has quit IRC17:59
*** graphene has joined #openstack-infra17:59
*** felipemonteiro has joined #openstack-infra18:00
fungiseparate. don't know18:02
fungitc resolution would be about continuance of control over repositories which are removed from governance due to lack of activity so they can still be properly retired once abandoned18:03
fungiinfra proposal would be more about whether and how we set policy about removing unused repos from our configuration when they're purely dead weight18:03
*** Bhujay has quit IRC18:04
*** bobh has joined #openstack-infra18:04
AJaegerno urgency - let's see whether I bring it up one irc meeting...18:06
AJaegerthanks, fungi. enjoy the weekend!18:06
*** bobh has quit IRC18:08
*** dave-mccowan has joined #openstack-infra18:08
*** armax has quit IRC18:10
*** felipemonteiro has quit IRC18:13
*** bobh has joined #openstack-infra18:14
*** bobh has quit IRC18:18
*** bobh has joined #openstack-infra18:24
fungiyou too!18:28
*** bobh has quit IRC18:28
*** graphene has quit IRC18:29
*** bobh has joined #openstack-infra18:35
*** bobh has quit IRC18:39
*** Miouge- has quit IRC18:42
*** Miouge has joined #openstack-infra18:42
*** bobh has joined #openstack-infra18:45
*** bobh has quit IRC18:49
*** e0ne has joined #openstack-infra18:53
*** bobh has joined #openstack-infra18:55
*** SteffanW has joined #openstack-infra18:58
*** bobh has quit IRC19:00
*** apetrich has quit IRC19:01
*** bobh has joined #openstack-infra19:05
*** zul has quit IRC19:09
*** bobh has quit IRC19:09
*** apetrich has joined #openstack-infra19:22
*** SteffanW has quit IRC19:28
*** bobh has joined #openstack-infra19:38
*** bobh has quit IRC19:43
*** bobh has joined #openstack-infra19:43
*** apetrich has quit IRC19:55
*** bobh has quit IRC19:55
*** bobh has joined #openstack-infra20:05
*** slagle has joined #openstack-infra20:08
*** bobh has quit IRC20:09
*** bobh has joined #openstack-infra20:13
*** apetrich has joined #openstack-infra20:18
*** bobh has quit IRC20:18
*** rf0lc0 has joined #openstack-infra20:23
*** rtjure has quit IRC20:24
*** rfolco has quit IRC20:25
*** rtjure has joined #openstack-infra20:26
*** bobh has joined #openstack-infra20:33
*** bobh has quit IRC20:37
*** owalsh has quit IRC20:39
*** owalsh has joined #openstack-infra20:40
*** bobh has joined #openstack-infra20:45
*** bobh has quit IRC20:50
*** bobh has joined #openstack-infra21:00
*** owalsh_ has joined #openstack-infra21:00
*** owalsh has quit IRC21:00
*** bobh_ has joined #openstack-infra21:04
*** bobh has quit IRC21:05
*** bobh_ has quit IRC21:08
*** e0ne has quit IRC21:10
*** felipemonteiro has joined #openstack-infra21:11
*** bobh has joined #openstack-infra21:13
*** graphene has joined #openstack-infra21:15
*** graphene has quit IRC21:34
*** graphene has joined #openstack-infra21:35
*** ijw has joined #openstack-infra21:42
*** apetrich has quit IRC21:46
*** efried_pto1 has joined #openstack-infra21:48
*** efried_pto has quit IRC21:50
*** efried_pto1 is now known as efried_pto21:50
*** ijw has quit IRC21:50
*** owalsh has joined #openstack-infra22:06
*** owalsh- has joined #openstack-infra22:09
*** owalsh_ has quit IRC22:09
*** owalsh has quit IRC22:12
*** felipemonteiro has quit IRC22:17
*** dave-mccowan has quit IRC22:36
dhellmannare there any examples of tools that know how to round-trip zuul config files that contain !encrypted/pkcs1-oaep items?22:37
dhellmannI figured out how to turn those into a list, but the elements of the list come out formatted differently (the newlines are lost)22:37
*** owalsh has joined #openstack-infra22:56
*** owalsh- has quit IRC22:57
*** armax has joined #openstack-infra23:23
*** bobh has quit IRC23:25
*** owalsh has quit IRC23:28
*** owalsh has joined #openstack-infra23:32
*** Hunner has quit IRC23:32
*** owalsh_ has joined #openstack-infra23:42
*** owalsh has quit IRC23:43
*** owalsh_ is now known as owalsh23:44
*** bobh has joined #openstack-infra23:51
*** bobh has quit IRC23:51
*** bobh has joined #openstack-infra23:52
*** bobh has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!