| *** ccneill has quit IRC | 00:24 | |
| *** rsyed is now known as rsyed_away | 00:29 | |
| *** rudrajit has joined #openstack-dns | 00:35 | |
| *** james_li has quit IRC | 00:36 | |
| *** rudrajit_ has quit IRC | 00:38 | |
| *** stanzgy has joined #openstack-dns | 01:11 | |
| *** ducttape_ has joined #openstack-dns | 01:20 | |
| *** ducttape_ has quit IRC | 01:30 | |
| *** jasonsb has joined #openstack-dns | 01:33 | |
| *** rudrajit has quit IRC | 01:37 | |
| *** richm has quit IRC | 01:44 | |
| *** bpokorny has quit IRC | 01:51 | |
| *** stanzgy has quit IRC | 01:57 | |
| *** stanzgy has joined #openstack-dns | 02:01 | |
| *** ducttape_ has joined #openstack-dns | 02:03 | |
| *** penick has quit IRC | 02:19 | |
| *** ducttape_ has quit IRC | 02:43 | |
| *** ducttape_ has joined #openstack-dns | 02:43 | |
| *** boris-42 has quit IRC | 03:04 | |
| *** ducttape_ has quit IRC | 03:06 | |
| *** ducttape_ has joined #openstack-dns | 03:12 | |
| *** rudrajit has joined #openstack-dns | 03:13 | |
| *** ducttape_ has quit IRC | 03:26 | |
| *** ducttape_ has joined #openstack-dns | 03:41 | |
| *** chlong_ has quit IRC | 03:47 | |
| *** chlong_ has joined #openstack-dns | 03:49 | |
| *** ducttape_ has quit IRC | 03:53 | |
| *** ducttape_ has joined #openstack-dns | 03:56 | |
| *** rudrajit has quit IRC | 04:05 | |
| *** rudrajit has joined #openstack-dns | 04:12 | |
| *** ducttape_ has quit IRC | 04:24 | |
| *** ducttape_ has joined #openstack-dns | 04:29 | |
| *** ducttape_ has quit IRC | 04:37 | |
| *** johnbelamaric has quit IRC | 06:02 | |
| *** johnbelamaric has joined #openstack-dns | 06:02 | |
| *** kei_yama has quit IRC | 06:34 | |
| *** kei_yama has joined #openstack-dns | 06:42 | |
| *** fawadkhaliq has joined #openstack-dns | 06:50 | |
| *** jasonsb has quit IRC | 07:02 | |
| *** fawadkhaliq has quit IRC | 07:51 | |
| *** rudrajit has quit IRC | 08:27 | |
| *** en_austin has joined #openstack-dns | 08:36 | |
| *** en_austin has quit IRC | 08:47 | |
| *** boris-42 has joined #openstack-dns | 08:52 | |
| *** jschwarz has joined #openstack-dns | 08:52 | |
| *** en_austin has joined #openstack-dns | 08:56 | |
| en_austin | Kiall: do you succeeded in reading my bind logs I've sent you yesterday?) | 08:58 |
|---|---|---|
| *** fawadkhaliq has joined #openstack-dns | 09:17 | |
| *** fawadkhaliq has quit IRC | 09:29 | |
| *** evrardjp has joined #openstack-dns | 09:32 | |
| *** evrardjp has quit IRC | 10:27 | |
| *** stanzgy has quit IRC | 10:38 | |
| *** pcaruana has joined #openstack-dns | 10:42 | |
| eandersson | Looking at my logs today it looks like mdns is behind on the serial -_- | 10:47 |
| eandersson | ok so I think I know why it's failing | 10:55 |
| eandersson | I have two control nodes, ccn02 is sending notifications, ccn01 is sending the SOA | 10:55 |
| eandersson | when I disabled ccn01 | 10:55 |
| eandersson | ccn02 took over and sent both the NOTIFY and SOA | 10:56 |
| eandersson | and it worked | 10:56 |
| eandersson | Sounds like there is an issue with our setup. | 10:56 |
| *** kei_yama has quit IRC | 10:56 | |
| eandersson | nvm ccn02 is back to erroring out on updates | 10:57 |
| *** nyechiel_ has joined #openstack-dns | 10:59 | |
| en_austin | eandersson: what is "control node"? | 11:02 |
| eandersson | en_austin: Openstack Control Nodes | 11:03 |
| eandersson | where all the Openstack services are hosted | 11:03 |
| eandersson | e.g. nova, neutron, designate | 11:03 |
| en_austin | awh. I don't have such in my infrastructure | 11:03 |
| en_austin | only Heat, Keystone and Designate is running now | 11:03 |
| en_austin | so much custom | 11:03 |
| en_austin | very custom | 11:03 |
| en_austin | wow :) | 11:03 |
| eandersson | hah | 11:03 |
| *** jordanP has joined #openstack-dns | 11:25 | |
| *** fawadkhaliq has joined #openstack-dns | 11:27 | |
| eandersson | It looks like it is actually working, but replication is slow when it starts to error out. | 11:38 |
| *** en_austin has quit IRC | 12:35 | |
| *** nyechiel_ has quit IRC | 12:39 | |
| *** nyechiel has joined #openstack-dns | 12:39 | |
| *** fawadkhaliq has quit IRC | 12:42 | |
| *** fawadkhaliq has joined #openstack-dns | 12:43 | |
| *** johnbelamaric has quit IRC | 12:45 | |
| *** fawadkhaliq has quit IRC | 12:47 | |
| *** pcaruana has quit IRC | 12:55 | |
| *** km has quit IRC | 12:55 | |
| *** pcaruana has joined #openstack-dns | 13:09 | |
| *** ducttape_ has joined #openstack-dns | 13:17 | |
| *** evrardjp has joined #openstack-dns | 13:33 | |
| *** ducttape_ has quit IRC | 13:36 | |
| *** fawadkhaliq has joined #openstack-dns | 13:53 | |
| *** johnbelamaric has joined #openstack-dns | 14:20 | |
| *** fawadkhaliq has quit IRC | 14:20 | |
| *** rsyed_away is now known as rsyed | 14:31 | |
| *** mlavalle has joined #openstack-dns | 14:37 | |
| *** jschwarz_ has joined #openstack-dns | 14:37 | |
| *** jschwarz has quit IRC | 14:38 | |
| *** ducttape_ has joined #openstack-dns | 15:03 | |
| *** PsionTheory has joined #openstack-dns | 15:04 | |
| *** jmccrory has quit IRC | 15:12 | |
| *** jmccrory has joined #openstack-dns | 15:13 | |
| *** jschwarz__ has joined #openstack-dns | 15:13 | |
| *** jschwarz_ has quit IRC | 15:16 | |
| *** penick has joined #openstack-dns | 15:23 | |
| *** PsionTheory has quit IRC | 15:26 | |
| eandersson | Kiall: https://github.com/openstack/designate/blob/master/designate/service.py#L273 | 15:26 |
| eandersson | I brought this up before, but socket.timeout can never be reached | 15:27 |
| *** pglass has joined #openstack-dns | 15:34 | |
| *** penick has quit IRC | 15:36 | |
| *** evrardjp has quit IRC | 15:37 | |
| *** jasonsb has joined #openstack-dns | 15:59 | |
| elarson | eandersson: is that b/c the socket.timeout is a subclass of socket.error? | 16:02 |
| * elarson checks | 16:03 | |
| eandersson | Yea | 16:03 |
| eandersson | So if something throws an socket.timeout it will be caught by the socket.error | 16:03 |
| elarson | >>> import socket | 16:04 |
| elarson | >>> issubclass(socket.timeout, socket.error) | 16:04 |
| elarson | True | 16:04 |
| elarson | indeed! | 16:04 |
| *** jschwarz__ is now known as jschwarz | 16:04 | |
| elarson | eandersson: so we've seen that error in mdns, which gets stuck at some point. | 16:10 |
| elarson | so we should fix it! | 16:11 |
| elarson | maybe move that client.close to a finally | 16:11 |
| elarson | h/t rsyed | 16:11 |
| pglass | finally++ | 16:11 |
| rsyed | but back to eandersson's observation...the behavior of both those blocks are similar other than their error message. it wouldn't likely resolve the issue IMO | 16:21 |
| elarson | rsyed: yeah, I just know we do see an exception due to the error message, so that is something to check off the list | 16:22 |
| rsyed | yup, certainly | 16:23 |
| rsyed | not sure i would have ever recognized that one was a subclass of the other... | 16:23 |
| *** johnbelamaric has quit IRC | 16:26 | |
| eandersson | rsyed, yea, this is not related to my current issues, just spotted it again when browsing the code | 16:27 |
| *** johnbelamaric has joined #openstack-dns | 16:28 | |
| *** bpokorny has joined #openstack-dns | 16:31 | |
| *** james_li has joined #openstack-dns | 16:37 | |
| *** johnbelamaric has quit IRC | 16:38 | |
| *** johnbelamaric has joined #openstack-dns | 16:38 | |
| mlavalle | mugsie: neutron cient dns functionality merged: https://review.openstack.org/#/c/282588 | 17:01 |
| mlavalle | Kiall: ^^^ | 17:01 |
| *** richm has joined #openstack-dns | 17:01 | |
| openstackgerrit | Eric Larson proposed openstack/designate: Ensure the zone records quota is enforced https://review.openstack.org/284361 | 17:09 |
| elarson | pglass: ^^^ I think that fixes the bug you found | 17:09 |
| elarson | !m pglass | 17:09 |
| openstack | elarson: Error: "m" is not a valid command. | 17:09 |
| elarson | also, the previous tests used 3 to account for the NS and SOA records that are managed. I changed the quota test to exclude managed records since it is confusing | 17:10 |
| elarson | mugsie, Kiall, timsim: ^^^ | 17:10 |
| *** johnbelamaric has quit IRC | 17:12 | |
| *** penick has joined #openstack-dns | 17:20 | |
| timsim | Would have said it if I'd been in the meeting yesterday, but plz review https://review.openstack.org/#/c/283784/ it's some WIP worker model stuff. Before I go write unit tests and get it ready for +2, I'd love to know what people actually think about the way I've done it, and how they find it works. | 17:26 |
| *** jordanP has quit IRC | 17:35 | |
| *** jasonsb has quit IRC | 17:41 | |
| openstackgerrit | Merged openstack/designate: Ensure ZoneManager emits valid objects https://review.openstack.org/280163 | 17:43 |
| openstackgerrit | Merged openstack/designate: Update README to RST https://review.openstack.org/267089 | 17:44 |
| openstackgerrit | Merged openstack/designate: Updated from global requirements https://review.openstack.org/283980 | 17:44 |
| openstackgerrit | Merged openstack/designate: Set Ubuntu as primary Vagrantfile VM https://review.openstack.org/283620 | 17:44 |
| openstackgerrit | Merged openstack/designate: Set InvalidRecordSetName and InvalidRecordSetLocation as expected https://review.openstack.org/283547 | 17:44 |
| *** _ducttape_ has joined #openstack-dns | 17:46 | |
| *** ducttape_ has quit IRC | 17:49 | |
| *** nyechiel_ has joined #openstack-dns | 17:54 | |
| *** nyechiel has quit IRC | 17:55 | |
| *** jschwarz has quit IRC | 18:11 | |
| *** penick has quit IRC | 18:20 | |
| *** penick has joined #openstack-dns | 18:25 | |
| *** rudrajit has joined #openstack-dns | 18:28 | |
| *** rudrajit_ has joined #openstack-dns | 18:39 | |
| *** rudrajit has quit IRC | 18:41 | |
| *** rsyed is now known as rsyed_away | 18:49 | |
| *** penick has quit IRC | 19:09 | |
| *** rsyed_away is now known as rsyed | 19:11 | |
| *** _ducttape_ has quit IRC | 19:12 | |
| *** eandersson_ has joined #openstack-dns | 19:14 | |
| eandersson_ | Does the pool-manager cache need to be shared between nodes? e.g. when using memcached | 19:15 |
| federico3 | eandersson_: yes | 19:18 |
| federico3 | eandersson_: if you configure memcache servers they will be used by the pool managers, if you leave memcached_servers empty the pool managers should cache only locally in-memory | 19:20 |
| eandersson_ | ok, but if we have two control nodes, what happens if both of them point to the local memcached server? | 19:20 |
| federico3 | it's beneficial to them | 19:21 |
| eandersson_ | that would in theory work the same as caching only in local in-memory | 19:21 |
| eandersson_ | I am mostly wondering if the pool manager cache SHOULD be shared betweeen instances, or if that can cause unwanted behavior, like the pool-manager services becoming out-of-sync. | 19:22 |
| *** fawadkhaliq has joined #openstack-dns | 19:22 | |
| eandersson_ | like causing two mdns instances to report different serials | 19:23 |
| federico3 | the cache is meant to be shared across components: they will all update it and read from it | 19:23 |
| federico3 | and this will lead to less serial probing against your resolvers | 19:23 |
| elarson | eandersson: yeah, federico3 is right. for example, when node 1 is doing something and writes in the cache it is working on something, node 2 can recognize that and avoid duplicating the work | 19:24 |
| eandersson_ | kk, will need to fix that | 19:24 |
| federico3 | eandersson_: you should experience no change in terms of reliability, only a decrease in traffic from mdns to the resolvers and faster propagation time | 19:26 |
| openstackgerrit | Eric Larson proposed openstack/designate: Use lazy properties in central service https://review.openstack.org/284382 | 19:26 |
| *** ducttape_ has joined #openstack-dns | 19:27 | |
| eandersson_ | If I supply two memcached instances in the config, I assume everything gets replicated? | 19:30 |
| eandersson_ | Or would you need to put a vip in front of them or something | 19:30 |
| eandersson_ | I can't remember how that worked. | 19:30 |
| timsim | eandersson_: I believe memcached/memcached clients have an algorithm for keys so that if you supply multiple instances they know which one to go to. | 19:32 |
| *** nyechiel_ has quit IRC | 19:35 | |
| *** EricGonczer_ has joined #openstack-dns | 19:39 | |
| *** EricGonc_ has joined #openstack-dns | 19:41 | |
| *** EricGonczer_ has quit IRC | 19:45 | |
| *** EricGonc_ has quit IRC | 19:50 | |
| *** krotscheck is now known as krotscheck_dcm | 19:50 | |
| *** EricGonczer_ has joined #openstack-dns | 19:51 | |
| *** eandersson_ has quit IRC | 19:51 | |
| openstackgerrit | Eric Larson proposed openstack/designate: Use a single recordset validation method https://review.openstack.org/284372 | 19:52 |
| *** EricGonczer_ has quit IRC | 19:53 | |
| *** bpokorny has quit IRC | 20:06 | |
| *** ducttape_ has quit IRC | 20:09 | |
| *** fawadkhaliq has quit IRC | 20:13 | |
| *** fawadkhaliq has joined #openstack-dns | 20:13 | |
| *** penick has joined #openstack-dns | 20:15 | |
| *** ducttape_ has joined #openstack-dns | 20:16 | |
| *** bpokorny has joined #openstack-dns | 20:20 | |
| openstackgerrit | Rahman Syed proposed openstack/designate: Improve error handling for TCP connections https://review.openstack.org/284912 | 20:22 |
| rsyed | hey what's the etiquette for giving shoutouts? commit message? eandersson deserves props for ending our mdns nightmare :-) | 20:24 |
| pglass | rsyed: you can add reviewers in gerrit | 20:24 |
| pglass | also you can just ping them | 20:24 |
| pglass | in irc | 20:24 |
| timsim | We could send him a fruit basket | 20:25 |
| pglass | oh you mean like congratulations | 20:25 |
| rsyed | yup | 20:26 |
| timsim | Or some money | 20:26 |
| *** johnbelamaric has joined #openstack-dns | 20:30 | |
| openstackgerrit | Rahman Syed proposed openstack/designate: Improve error handling for TCP connections https://review.openstack.org/284912 | 20:36 |
| *** EricGonczer_ has joined #openstack-dns | 20:41 | |
| *** porunov has joined #openstack-dns | 20:48 | |
| openstackgerrit | Eric Larson proposed openstack/designate: Use lazy properties in services https://review.openstack.org/284382 | 20:48 |
| openstackgerrit | Rahman Syed proposed openstack/designate: Improve error handling for TCP connections https://review.openstack.org/284912 | 20:51 |
| *** ducttape_ has quit IRC | 20:52 | |
| *** ducttape_ has joined #openstack-dns | 20:53 | |
| *** EricGonc_ has joined #openstack-dns | 20:54 | |
| *** EricGonczer_ has quit IRC | 20:58 | |
| *** james_li_ has joined #openstack-dns | 20:59 | |
| *** james_li_ has quit IRC | 21:03 | |
| *** pglbutt has joined #openstack-dns | 21:03 | |
| *** james_li has quit IRC | 21:03 | |
| *** pglass_ has joined #openstack-dns | 21:03 | |
| *** james_li has joined #openstack-dns | 21:03 | |
| *** james_li has quit IRC | 21:03 | |
| *** james_li has joined #openstack-dns | 21:04 | |
| *** pglass has quit IRC | 21:04 | |
| *** pglbutt has quit IRC | 21:07 | |
| *** ducttape_ has quit IRC | 21:13 | |
| *** eandersson_ has joined #openstack-dns | 21:17 | |
| *** fawadkhaliq has quit IRC | 21:20 | |
| Kiall | rsyed / eandersson / timsim / elarson - that can't.. That just can't.. Heh. This problem needed a more detailed and involved fix after all the untraced reports we've seen :( | 21:22 |
| Kiall | Something so simple is just embarrassing :p | 21:22 |
| *** ducttape_ has joined #openstack-dns | 21:35 | |
| *** eandersson_ has quit IRC | 21:43 | |
| *** EricGonc_ has quit IRC | 21:44 | |
| *** rudrajit has joined #openstack-dns | 21:58 | |
| *** penick_ has joined #openstack-dns | 21:59 | |
| *** penick has quit IRC | 21:59 | |
| *** penick_ is now known as penick | 21:59 | |
| *** porunov has quit IRC | 22:01 | |
| *** rudrajit_ has quit IRC | 22:01 | |
| *** fawadkhaliq has joined #openstack-dns | 22:19 | |
| *** james_li_ has joined #openstack-dns | 22:19 | |
| *** james_li has quit IRC | 22:22 | |
| *** rudrajit_ has joined #openstack-dns | 22:39 | |
| *** pglass_ has quit IRC | 22:41 | |
| *** pglass has joined #openstack-dns | 22:41 | |
| *** rudrajit has quit IRC | 22:42 | |
| *** rsyed is now known as rsyed_away | 22:46 | |
| *** km has joined #openstack-dns | 23:17 | |
| *** ducttape_ has quit IRC | 23:24 | |
| openstackgerrit | Tim Simmons proposed openstack/designate: WIP Worker Model https://review.openstack.org/283784 | 23:26 |
| *** mlavalle has quit IRC | 23:26 | |
| *** kei_yama has joined #openstack-dns | 23:32 | |
| *** fawadkhaliq has quit IRC | 23:47 | |
| *** pglass has quit IRC | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!