| *** ircuser-1 has joined #openstack-dns | 00:56 | |
| *** w|zzy has quit IRC | 02:19 | |
| *** w|zzy has joined #openstack-dns | 03:33 | |
| *** w|zzy has quit IRC | 04:39 | |
| *** w|zzy has joined #openstack-dns | 04:40 | |
| *** w|zzy has quit IRC | 05:35 | |
| *** w|zzy has joined #openstack-dns | 05:41 | |
| *** _nyloc_ has quit IRC | 06:29 | |
| *** nyloc has joined #openstack-dns | 06:33 | |
| *** Emine has joined #openstack-dns | 07:01 | |
| *** ginopc has joined #openstack-dns | 08:12 | |
| *** pcaruana has joined #openstack-dns | 09:09 | |
| *** salmankhan has joined #openstack-dns | 10:02 | |
| *** salmankhan has quit IRC | 10:27 | |
| *** lkoranda has quit IRC | 11:09 | |
| *** lkoranda has joined #openstack-dns | 11:10 | |
| mugsie | andrewbogott: if you have any logs, (to see what it is trying to do when it gets the DuplicateDB error) that would help | 12:18 |
|---|---|---|
| mugsie | are the 4 pdns db's separate (i.e. no replication betweeen them) | 12:19 |
| andrewbogott | mugsie: hello! :) | 12:33 |
| andrewbogott | Yes, four separate dbs | 12:33 |
| andrewbogott | Here are some log tails from the last few minutes | 12:34 |
| andrewbogott | the pool manager on 208.80.155.117 is saying things like this: | 12:34 |
| andrewbogott | https://www.irccloud.com/pastebin/M9y0mA88/ | 12:34 |
| andrewbogott | And 208.80.154.135 is saying things like this: | 12:35 |
| andrewbogott | https://www.irccloud.com/pastebin/HhlHAtZD/ | 12:36 |
| andrewbogott | I can run more verbose logs on request | 12:36 |
| mugsie | andrewbogott: OK - it looks like the zone was already created in the pdns DB somehow | 12:37 |
| mugsie | what does openstack zone show 16.172.in-addr.arpa. look like? | 12:37 |
| andrewbogott | I've certainly run through this lots of times with different configs so it's easy to believe there's db cruft here and there | 12:38 |
| mugsie | yeah, i think that might be part of the issue - it is getting a DB error creating the domain, so it isnt updating the mdns masters, so it can;t get the new data set to update the serial | 12:39 |
| mugsie | (this *is* currently a guess though :) ) | 12:39 |
| andrewbogott | that domain is in noauth-project; currently trying to remember how to access that from the API :) | 12:42 |
| andrewbogott | but note that that's not the only domain I get that error for, it's just a sample | 12:42 |
| andrewbogott | well… wait, I take that back, today that seems to be the only one it's complaining of | 12:42 |
| andrewbogott | Are logs like 'For zone eqiad.wmflabs. the error serial is 0.' a sign of a problem or is that a good thing? | 12:43 |
| mugsie | sign of an issue | 12:46 |
| mugsie | andrewbogott: as an admin adding --sudo-project-id noauth-project should help get the status | 12:46 |
| andrewbogott | https://www.irccloud.com/pastebin/EbgORCq9/ | 12:47 |
| andrewbogott | I can certainly just wipe out that domain entry in each pdns database and see if it works better the next time around | 12:49 |
| mugsie | yeah, that should help | 12:49 |
| andrewbogott | hm, well, I take that back, I can only safely wipe it in the new databases. Let me see if that's the one it's complaining about... | 12:50 |
| andrewbogott | (I have production things pointed at ns0 and ns1 but not ns2 or ns3) | 12:50 |
| *** pcaruana has quit IRC | 12:52 | |
| andrewbogott | so — in case this was just a race I stopped both pool managers, deleted that record from all databases and restarted the pool managers… getting the same error again now anyway :( | 13:01 |
| andrewbogott | I guess my one pool manager is unable to detect that the zone was already created by the other | 13:01 |
| andrewbogott | I'm going to stop one of the pool managers and leave it stopped for a bit, just to see if one of them on its own is capable of happiness | 13:02 |
| andrewbogott | once a zone is in an error state will designate keep trying to create or sync it? Or does it despair and give up? | 13:20 |
| *** Emine has quit IRC | 13:28 | |
| *** Emine has joined #openstack-dns | 13:30 | |
| andrewbogott | well, even with only one mdns and pool-manager it still marks a lot of things as ERROR. That's probably a clue | 13:34 |
| *** rfreire has joined #openstack-dns | 13:49 | |
| mugsie | andrewbogott: it will try to create it forever | 13:54 |
| mugsie | (if it is in action create and status error | 13:54 |
| mugsie | ) | 13:54 |
| andrewbogott | ok — I see at least one place where axfr is clearly not working. Going to try to sort that out and see if that makes these other things happy. | 13:55 |
| mugsie | deleted the record or the the zone? | 13:55 |
| andrewbogott | does axfr happen entirely via port 53 or is there some other port that I might be accidentally blocking? | 13:55 |
| rfreire | mugsie, \o | 13:56 |
| mugsie | 5354 is the mdns port, and the client port could be anything | 13:57 |
| mugsie | rfreire: o/ | 13:57 |
| *** sapd1_ has joined #openstack-dns | 14:19 | |
| andrewbogott | mugsie: opening up port 5354 in a bunch of places has improved things quite a bit! Now I'm seeing errors from pdns like this: | 14:47 |
| andrewbogott | While checking domain freshness: Query to '208.80.155.117:5354' for SOA of 'staging.wmflabs.org' produced no results (error code: Refused) | 14:47 |
| andrewbogott | Note that in this case, 208.80.155.117 is the same host as the one logging the error | 14:48 |
| andrewbogott | any guess why mdns would 'refuse' a query? | 14:48 |
| mugsie | andrewbogott: refused is a weird one - do you have multiple pools? | 14:50 |
| andrewbogott | just one | 14:50 |
| mugsie | there should be something in the mdns logs - do you have tsig keys enabled anywhere? | 14:50 |
| mugsie | oh, is staging.wmflabs.org actually in designate? | 14:51 |
| andrewbogott | oh, good question, I will check | 14:51 |
| andrewbogott | looks like it isn't. | 14:53 |
| andrewbogott | Does axfr only add things and not purge deleted things? | 14:53 |
| mugsie | yeah, afxr only updates the zone contents | 14:54 |
| mugsie | it won't even create a zone | 14:54 |
| mugsie | (the SQL in the driver does that) | 14:54 |
| andrewbogott | purges happen via sql too? | 14:55 |
| mugsie | it should delete records though | 14:55 |
| mugsie | yea, in the version of the pdns backend you use, they do | 14:55 |
| mugsie | (as a heads up that will go away at some point in the future) | 14:55 |
| andrewbogott | mdns is saying some things like "Got lower serial for 'eqiad.wmflabs.' to '208.80.154.11:53'. Expected:'1535986277'. Got:'1535984915'.Retries left='0'" | 14:56 |
| andrewbogott | which I assume is just another way of saying 'the sync isn't working' | 14:56 |
| mugsie | yeah, it means the info is not being pushed | 14:59 |
| andrewbogott | oh shit! I have a typo in my pool config, a wrong IP | 15:01 |
| andrewbogott | let's fix that — I bet that will help :) | 15:01 |
| andrewbogott | mugsie: ok, so I had two problems, both entirely self-inflicted: a wrong master IP and some busted firewall rules. My logs are now blissfully quiet. | 15:12 |
| andrewbogott | thank you for talking me through this! I would still be rearranging the pool config (and missing the typo) but for your help. | 15:13 |
| mugsie | andrewbogott: great! no problem at all, I do it all the time :D | 15:13 |
| *** salmankhan has joined #openstack-dns | 15:30 | |
| *** salmankhan has quit IRC | 15:34 | |
| *** dhellmann has quit IRC | 16:39 | |
| *** dhellmann has joined #openstack-dns | 16:39 | |
| *** aniketh has joined #openstack-dns | 16:41 | |
| aniketh | Hi all. | 16:44 |
| *** dhellmann has quit IRC | 16:44 | |
| *** dhellmann has joined #openstack-dns | 16:45 | |
| *** dhellmann has quit IRC | 16:54 | |
| *** dhellmann has joined #openstack-dns | 16:54 | |
| mugsie | aniketh: hey | 17:07 |
| aniketh | I am Aniketh Girish, a Junior year Computer Science Engineering student at the Amrita University, Kerala, India. I had initiated to contribute towards OpenStack a few months back. I told some time off since I was selected to participate in Google Summer of code with GNU Linux organization. | 17:07 |
| aniketh | For the last few months, I have been mainly focusing on implementing and learning about advanced internet protocols. My primary interest leans towards network security, with a particular interest in the Networking protocol , mainly DNS and HTTP and future internet architecture. | 17:07 |
| aniketh | It is about time for me to look for a promising research-oriented thesis project in my field of interest. Therefore, I wish to collaborate with the Designate and start working on a project as my thesis. I'm expected to get my project done by the end of 2019. So, actually I have lot of time to get familiar with the codebase, Therefore I like to know if there is a possibility for me to do my thesis project together with | 17:08 |
| aniketh | designate project. | 17:08 |
| mugsie | aniketh: sure, that would be great | 17:14 |
| mugsie | what would you like to do for the project> | 17:14 |
| mugsie | ? | 17:14 |
| aniketh | mugsie: actually, I'm not familiar with what designate would like to have. So it would be great if I could know what the community expects from this and could be considered as a thesis project for me too | 17:16 |
| mugsie | OK, we will be meeting in person next week, so we can have a discussion there | 17:17 |
| mugsie | if you have anything you would like to work on, let me know | 17:17 |
| aniketh | mugsie: meeting in person? 😅, Sorry I didn't get you. | 17:18 |
| mugsie | there is an openstack project teams gathering next week, so there will be people from the designate team in Denver. The people who are there can see if we csn find a section of the work we want done that could be a project | 17:19 |
| aniketh | Actually I'm in India. 😅 | 17:22 |
| mugsie | yes - but right now I cannot think of something that would be a thesis, and we need, so I am going to wait until denver to find something we can suggest for you | 17:25 |
| aniketh | mugsie: yes. I can wait. Since I have alot of time as well ;). | 17:26 |
| aniketh | I hope you can put across a word about me and my query about such a opportunity there and if possible get back to me :). | 17:27 |
| aniketh | I will ping you back next week as well :) | 17:27 |
| aniketh | When does the meeting gets done? So that I can ping you for something regarding this. | 17:28 |
| mugsie | we finish sept 14 | 17:29 |
| aniketh | mugsie: Cool. I will get to you by sept 15 :) | 17:33 |
| *** sapd1_ has quit IRC | 17:39 | |
| *** simonmcc has quit IRC | 19:10 | |
| *** rfreire has quit IRC | 19:48 | |
| *** aniketh has quit IRC | 21:05 | |
| *** Emine has quit IRC | 21:20 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!