Monday, 2019-07-29

*** goldyfruit has joined #openstack-dns02:16
*** goldyfruit has quit IRC02:24
*** nafiux has quit IRC06:49
*** nafiux has joined #openstack-dns07:06
*** pcaruana has joined #openstack-dns07:31
*** nafiux has quit IRC07:45
*** ircuser-1 has joined #openstack-dns07:57
*** ivve has joined #openstack-dns08:15
*** stingrayza_ is now known as stingrayza09:10
*** trident has quit IRC09:16
*** trident has joined #openstack-dns09:17
*** baffle has quit IRC09:22
*** altlogbot_0 has quit IRC09:24
*** altlogbot_0 has joined #openstack-dns09:24
*** baffle has joined #openstack-dns09:30
*** jawad_axd has joined #openstack-dns10:38
*** ivve has quit IRC12:34
*** jawad_axd has quit IRC13:08
*** ivve has joined #openstack-dns13:11
*** ivve has quit IRC13:16
*** ivve has joined #openstack-dns14:55
*** nafiux has joined #openstack-dns15:29
*** openstackgerrit has joined #openstack-dns15:35
*** ChanServ sets mode: +v openstackgerrit15:35
openstackgerritErik Olof Gunnar Andersson proposed openstack/designate master: Removed deprecated pool-manager implementation  https://review.opendev.org/66547615:35
*** ginopc has quit IRC16:01
*** goldyfruit has joined #openstack-dns16:13
*** ivve has quit IRC16:21
*** ivve has joined #openstack-dns16:52
*** ivve has quit IRC16:59
*** goldyfruit has quit IRC17:02
*** nafiux has quit IRC17:42
*** nafiux has joined #openstack-dns17:45
*** trident has quit IRC17:47
*** trident has joined #openstack-dns17:51
*** goldyfruit has joined #openstack-dns17:59
*** JonathanD has quit IRC18:19
*** JonathanD has joined #openstack-dns18:37
*** nafiux has quit IRC19:17
*** abaindur has joined #openstack-dns19:28
*** nafiux has joined #openstack-dns19:30
*** goldyfruit has quit IRC19:37
*** goldyfruit has joined #openstack-dns20:16
openstackgerritCorey Bryant proposed openstack/designate master: Ensure coordination IDs are encoded  https://review.opendev.org/67336020:17
*** trident has quit IRC20:49
*** trident has joined #openstack-dns20:52
openstackgerritErik Olof Gunnar Andersson proposed openstack/designate master: Removed deprecated pool-manager implementation  https://review.opendev.org/66547621:05
*** goldyfruit has quit IRC21:07
abaindurtrying to understand DNS serials a bit here, can anyone help me understand why our zone is repeatedly going into ERROR state?21:20
abaindurso our rndc is failing to addzone due to Stderr: u"rndc: 'addzone' failed: already exists\n" since this was a zone that already existed21:21
abaindurafterwards we see: Letting worker send NOTIFYs instead notify_zone_changed and Sent NOTIFY to 10.195.0.121:53 for zone grafana.infra.21:22
eanderssonMaybe you have a duplicate entry or similar in the pools.yaml config?21:22
abaindurbut it fails on polling. we have two nameservers listed in our pool.syaml21:22
abaindurthey are returning different sderials?21:22
abaindurFound serial 1563914415 on 10.195.0.121 for zone grafana.infra. __call__21:22
abaindurFound serial 0 on 10.195.0.122 for zone grafana.infra. __call__21:22
abaindur Results for polling grafana.infra.-1563914415: DNSQueryResult(positives=1, no_zones=1, consensus_serial=1563914415, results=[1563914415, 0])21:22
abaindurUnsuccessful poll for grafana.infra. on attempt 4 _do_poll21:23
eanderssonMaybe the domain already existed on bind21:23
abaindurbasically it then repeatedly gets stuck in a loop like that - polling 10 times, then trying the rndc, then polling 10 times, etc..21:23
eanderssonWould it be possible to delete the zone in Designate, and make sure it is removed in bind?21:23
eanderssonand then re-add it21:23
abaindurYes, this zone already existed. the hosts where designate was installed on got shuffled around21:23
abaindurthe zone was already created by Designate and has many records21:24
eanderssonI see21:24
abaindurdon't want to delete it, is there another way to recover?21:24
abainduram i correct in guessing that its failing because one NS returns a serial, the other returns a serial of 0?21:25
eanderssonmugsie, ^21:26
eanderssonMaybe the zone is misconfigured on 10.195.0.122 (e.g. got the wrong master, should be your designate ip)21:26
eanderssonWhat happens if you do21:28
eanderssondig @10.195.0.122 grafana.infra AXFR21:28
eanderssondoes it list the records?21:28
*** nafiux has quit IRC21:43
*** nafiux has joined #openstack-dns22:02
abaindureandersson: sorry was in meeting22:20
abaindurit is saying Transfer failewd22:20
abaindur; Transfer failed22:20
eanderssonAnd if you do it against 121?22:20
abaindurSame22:20
eanderssonI'll try to set up bind9 tonight at home and see if I can reproduce it22:21
eanderssonmugsie, is probably sleeping already, but I am sure he would know as well22:21
abainduryea np, I don't have the most expertise with bind myself either, besides a basic setup22:22
abaindurIt is only happening with this one zone. there are other zones that show ACTIVE22:22
abaindureandersson: based on those DNSquery logs above,22:28
abaindurResults for polling grafana.infra.-1563914415: DNSQueryResult(positives=1, no_zones=1, consensus_serial=1563914415, results=[1563914415, 0])22:28
abaindurdoes that mean designate is polling two backend nameservers to see if the change was reflected22:28
abaindurconsensus serial is the serial# designate is looking for?22:28
eanderssonYea - I think it's saying that one of them isn't set up properly, or maybe that the server is denying the request22:29
abaindurand results=[1563914415, 0] indicated the response from each?22:29
eanderssonIn the bind config you should have masters configured for that zone22:29
eanderssonand maybe the ips are wrong22:29
abaindurhmm, let me see if we can remove the secondary nameserver from pools.yaml?22:29
abaindurFound serial 1563914415 on 10.195.0.12122:29
abaindurFound serial 0 on 10.195.0.12222:29
eandersson>  { type slave; masters { ;}; file "slave.grafana.infra.6f2a845e-4e27-4f5c-a40d-171e2f40e3e5"; };22:29
eanderssonYou should have a file like this22:29
eanderssonon the bind9 side22:30
eanderssonand thinking that maybe the masters portion isn't set properly22:30
abaindurpools.yaml is listing both those hosts under the nameservers: section22:30
eanderssonhow about masters22:30
eanderssonAre all your designate servers listed there?22:31
abaindurIs that in the named.conf file?22:31
eanderssonsorry under pools.yaml22:31
eanderssonhttps://github.com/openstack/designate/blob/master/devstack/designate_plugins/backend-bind9#L10322:31
abaindurah no... both IPs are listed under the nameservers: section22:34
abaindurbut under targets:22:34
abaindurour masters: section lists the IPs of our two hosts where Designate is running22:35
abaindurand the options: section only has the host/rndc_host: for the .121 IP22:35
abaindur(.121 and .122 are located elsewhere, on a different backend machines)22:36
abainduris the options: under targets: supposed to list all rndc targets for all nameservers?22:37
abainduractually the .122 is listed under the also_notifies: section22:40
eanderssonUnder options I think you can/should only target one, but it might be worth trying adding a second target22:44
eanderssonjust to see if that resolves it22:46
*** abaindur has quit IRC23:27
*** abaindur has joined #openstack-dns23:30

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!