17:05:10 <Kiall> #startmeeting designate
17:05:11 <openstack> Meeting started Wed May 28 17:05:10 2014 UTC and is due to finish in 60 minutes.  The chair is Kiall. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:05:12 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:05:15 <openstack> The meeting name has been set to 'designate'
17:05:23 <Kiall> Hey all! Who's about?
17:05:29 <vinod1> here
17:05:30 <betsy> o\
17:05:33 <eankutse> heya
17:05:34 <mugsie> o/
17:05:38 <Kiall> Sorry for starting late, head deep in git hell ;)
17:05:40 <timfreund> hi
17:05:42 <RaginBajin> hi
17:05:45 <jbratton> Kiall: I'm here to talk about anything Joe added to the agenda
17:06:01 <shakamunyi> hey there
17:06:36 <Kiall> #topic Review action items from last week
17:06:58 <Kiall> First two are richm, he's not here at the moment, so we'll come back to them
17:07:14 <Kiall> eankutse: open blueprint for regionalized mdns servers / to file blueprint around region-aware handing of NOTIFY's and AXFR's
17:07:25 <eankutse> done
17:07:34 <Kiall> Oh - Cool, I must have missed them
17:07:37 <Kiall> Links?
17:07:39 <eankutse> will post link later
17:07:43 <Kiall> K
17:07:59 <Kiall> last one was for me, re-add multi-region mdns to agenda once ^ is done..
17:08:15 <Kiall> It'll be on the agenda next week, once we have a chance to review the BP's
17:08:34 <Kiall> #topic mDNS Questions
17:08:54 <Kiall> Okay - So there's a bunch of questions re mDNS here: https://wiki.openstack.org/wiki/Meetings/Designate#Agenda
17:09:05 <eankutse> #Link https://blueprints.launchpad.net/designate/+spec/mdns-designate-mdns-region-awareness
17:09:12 <Kiall> eankutse, I'm guessing they were you?
17:09:15 <jbratton> those are probably mostly my fault
17:09:30 <Kiall> jbratton: oh, fair enough ;)
17:09:40 <eankutse> jbratton: good items :-)
17:09:45 <Kiall> Okay .. So, first up.. "How does a server start up from scratch look like?"
17:10:08 <eankutse> I made submission for first cut of service code
17:10:14 <eankutse> tho build is failing
17:10:36 <jbratton> yeah, does it do zone transfers for every zone from scratch? or if not, how does it know what was changed while it was down?
17:10:37 <eankutse> service starts up like all other designate services bt
17:10:41 <eankutse> but
17:10:54 <eankutse> also sits on tcp/udp port
17:11:06 <eankutse> from where it will talk to name servers
17:11:15 <Kiall> jbratton: so, I think there's probably 2 different things to handle in this.. Brand new/replacement server, and a server that was down for maintainance for short period
17:11:27 <jbratton> yeah, true, I see that was broken up in to two points
17:11:37 <Kiall> Oh - I missed that :)
17:11:50 <jbratton> so I guess the first thing is.. how does it get the list of zones for the initial run
17:11:59 <jbratton> building out the config file from them
17:12:15 <Kiall> The brand new/replacement server scenario?
17:12:19 <eankutse> Actually, these are issues that I am thinking through currently and would be updating the blueprint as I go. I welcome ideas on these as well.
17:12:21 <jbratton> yeah
17:12:26 <RaginBajin> Shouldn't that just be a lookup in the database for the respective tags.
17:12:46 <jbratton> I'm talking about from an operational standpoint.. I'm a server admin :)
17:13:50 <eankutse> jbratton: would you clarify a little bi?
17:14:02 <eankutse> so miniDNS is up
17:14:09 <eankutse> for the first time
17:14:15 <jbratton> sure, so I run a BIND nameserver, I start BIND, now I need to get a list of every single zone and add them to the config
17:14:30 <jbratton> knowing that it will be out of date almost immediately, but it is still a starting point
17:15:35 <jbratton> is this going to be part of a BIND plugin that still needs to be written?
17:15:44 <eankutse> jbratton: very good question. I need to think through that a bit. I'll update the blueprint and design docs as we go.
17:16:18 <mugsie> currently this is done with a "sync" command
17:16:23 <mugsie> (not sink)
17:16:30 <RaginBajin> My $0.02 would say that if you send designate a request with the IP/hostname of the BIND server you set up and it will send a Notify to that server
17:16:38 <mugsie> and can be done via the api by an admin
17:16:39 <eankutse> #action eankutse to get initial answers to jbrattons questions
17:16:47 <Kiall> mugsie: yea, it's pretty naive though
17:17:07 <mugsie> but that patter could be extended to target hosts / NSs
17:17:10 <mugsie> pattern*
17:17:38 <Kiall> Ideally, we need a way to monitor that all the expected zones are actually on the nameserver, and automatically re-create any missing zones.
17:17:40 <mugsie> which would cause our backend to try and recreate all domains on the host
17:17:44 <jbratton> cool, because once it gets the list, it can zone transfer all of them and get the current version
17:18:01 <mugsie> Kiall: that might be difficult on a new server install
17:18:04 <jbratton> Kiall: that would be great to catch anything added while it is starting up
17:18:24 <mugsie> exactlly - we need to do that in any case
17:18:30 <mugsie> (lost msgs etc)
17:18:33 <jbratton> yeah
17:19:03 <mugsie> but the inital "here is 1million domains" could be the sync
17:19:22 <jbratton> sure.. basically, I just need the zone names, because then it can zone transfer them all from mdns
17:19:51 <Kiall> Yea - It's more complicated that that though, unless we can freeze any zone create/deletes on the pool for the duration
17:20:31 <jbratton> yeah
17:20:35 <Kiall> e.g. the server get's a list of 1mil domains and starts chewing on them, midway, one of the pending zones is deleted
17:20:40 <mugsie> Kiall  - theory would those messages not just buuld up in the queue?
17:20:59 <mugsie> and then you would get an eventual consistancy?
17:21:05 <jbratton> well, at least with BIND, it starts up almost instantly when no zones are present, then it starts working on the zone transfers
17:21:19 <jbratton> so it can take requests to add new zones while it is also going through 1 million zone transfers
17:21:51 <Kiall> jbratton: Yea, i think the actual AXFR bit should be fine - or maybe we're talking about different things?
17:21:55 <jbratton> and if something is deleting in the middle.. it just won't transfer it
17:22:13 <Kiall> If a server fails, and a new one replaces it, it's BIND config is empty, we need to populate it
17:22:46 <jbratton> right, so we create the BIND config, start it, and then it will start up and become responsive while simultaneously issuing tons of zone transfers
17:23:16 <jbratton> it will respond with SERVFAIL for zones it hasn't loaded yet and it will respond authoritatively for zones it has already transferred
17:23:46 <jbratton> and if a zone is deleted while it is going through the list doing transfers, it just removes it from the list of pending transfers
17:23:56 <jbratton> using rndc addzone/delzone
17:24:48 <RaginBajin> jbratton: Are you thinking that there is a designate-mdns command that sits on the client to help facilitate this?
17:24:57 <Kiall> Yea - As graham mentioned, I think we can take a similar approach for actually populating the BIND config.. Allow create/delete messages to build up while we action a snapshot of the zones.. Then start processing the messages, and catching up
17:25:04 <rjrjr_> isn't this a problem for server pools, not mini-dns?
17:25:19 <jbratton> right, it isn't so much a mdns question
17:25:25 <Kiall> rjrjr_: I think it applies to both
17:25:33 <jbratton> RaginBajin: that would work too
17:25:40 <mugsie> yeah, it will just be more pronounced in server pools
17:25:42 <Kiall> if pools don't exist, we still need to get BIND/PowerDNS/NSD etc synced up
17:25:51 <jbratton> I leave it to you all to implement this stuff, I just know how it all runs on the nameserver :)
17:26:02 <Kiall> ;)
17:26:12 <eankutse> jbratton: you have great input ffrom the ops side of things
17:26:14 <Kiall> Yea - I think we need to give this one some though
17:26:18 <Kiall> thought*
17:26:26 <Kiall> (from an implementation POV)
17:26:30 <mugsie> jbratton: this is why it i s good to have an ops person shouting "that wont work!!!"
17:26:46 <eankutse> I'll talk to you as I design/implement mdns
17:26:46 <jbratton> that's why I'm here!
17:26:53 <eankutse> :-)
17:27:21 <vinod1> How about if zones are created on the backend (outside of designate), do they/will they be added to mdns?
17:27:45 <eankutse> #action eankutse to pick Jbratton's brain on ops side of things for minidns/pools
17:27:47 <Kiall> vinod1: So - if the BIND server has an existing zone, "import" it?
17:28:38 <vinod1> yes - essentially will "sync" be bidirectional or unidirectional?
17:28:55 <Kiall> Ahh.. Okay, I'd have thought unidirectional
17:29:23 <Kiall> we have no way to associate zones with tenant's etc if they can go both way
17:30:07 <Kiall> Okay, Let's park this for today?
17:30:10 <vinod1> How about if a designate created zone is deleted at the backend - the scenario being designate is down and the zone needs to be deleted asap
17:30:22 <Kiall> #action kiall to write our various initial load scenarios
17:30:58 <Kiall> vinod1: I believe we should re-create that zone when Designate is back online / notices.
17:31:03 <jbratton> vinod1: at least currently, we would break synchronization.. remove it once it is back up, then resync
17:31:08 <jbratton> or that
17:31:40 <Kiall> jbratton: yea, same thing really..  break synchronization, delete from designate, then there's nothing for designate to notice is missing
17:31:56 <jbratton> yeah.. it's a problem to solve no matter what implementation you have
17:32:12 <jbratton> I agree on the unidirectional thing
17:32:24 <Kiall> But from Designate's POV, I think it should re-create zones that "mysteriously" go missing from the backend DNS server
17:32:59 <Kiall> urgent zone deletion is probably the least common reason for a zone to go "poof" on a DNS server ;)
17:33:20 <eankutse> so that means, if anyone deletes something from backend (manually) they have to remove it from storage/central too if they don't want if resurrected
17:33:24 <jbratton> yeah, and zone deletion is almost never an emergency thing
17:33:57 <mugsie> imho opinion, people should not be touching the backend servers that designate manages
17:34:23 <jbratton> yeah, I agree, we never do that on our current implementation
17:34:24 <mugsie> so, in the small egde cases that they need to we should not make too many compramises on design
17:34:26 <Kiall> Well, not touching the list of zones
17:34:44 <mugsie> yeah - thatswhat I meant
17:34:57 <mugsie> I assume someone will be looking to make sure that they are still running
17:35:00 <mugsie> ;)
17:35:08 <Kiall> Okay - I'll write up the various initial load scenarios I can think of on an etherpad tomorrow and email them out, hopefully we can get some clearer ideas that way :)
17:35:46 <Kiall> That kinda covered the first 3 of the items jbratton added, last one was
17:35:49 <Kiall> "I noticed during the demo that listing records in a zone doesn't include NS records. That seems weird, because how do you modify them?"
17:36:13 <Kiall> We don't currently expose the (system defined) NS or SOA records in the API
17:36:44 <jbratton> makes sense from a corporate standpoint, but from a customer service standpoint, customers really like to white label nameservers
17:36:55 <jbratton> they want ns.mydomain.com instead
17:37:11 <Kiall> There was some talk in the past about changing that, but I don't think we ever settled on something?
17:37:22 <mugsie> no afik
17:37:49 <mugsie> i am not sure how we could do it so that customers don;t bork their zones
17:38:06 <jbratton> same as you do currently.. don't let them delete the last NS record
17:38:10 <Kiall> We can likely display them to the V2 API pretty easily, but allowing them to reband them would be more work :)
17:38:31 <Kiall> On the plus side, if we expose them, it's not a breaking API change later to allow them be edited
17:38:41 <jbratton> yeah
17:38:49 <mugsie> yeah ++ on showing them in REST API then
17:38:52 <jbratton> we couldn't implement it until they could be changed.. but that's something I can bug our guys about :)
17:38:57 <betsy> jbratton: Treat it as a regular record type?
17:39:18 <rjrjr_> what is the use case?
17:39:28 <jbratton> customers want ns.mydomain.com instead of ns.rackspace.com
17:39:37 <jbratton> or they want to delegate a subdomain
17:39:37 <Kiall> jbratton: yea, API changes are much harder to do though ;) So getting all the breaking changes done ASAP is important
17:39:52 <Kiall> delegate a subdomain works today BTW
17:39:57 <jbratton> oh, cool
17:40:06 <jbratton> then the main use case is white labeling
17:40:21 <Kiall> we just don't allow customers to change the NS records at the root of the zone - i.e. white label
17:41:17 <jbratton> so going back to server pools.. how would someone know which pool they were on?
17:41:19 <Kiall> #action kiall to file BP on exposing the NS (and SOA?) record in the V2 API
17:41:28 <jbratton> because they would want to know what NS records to use for their domain
17:41:44 <jbratton> or do you envision that being something the implementation should define
17:41:50 <Kiall> jbratton: today, you can call /v1/domains/$ID/servers and get the list back
17:42:01 <jbratton> okay, cool
17:42:40 <Kiall> (I would imagine that still get's kept even if the NS records are exposed in the normal records list, as it's easier to pick out for UI's etc)
17:43:01 <jbratton> yeah, definitely, in case you are making administrative changes to the pools
17:43:30 <vinod1> in v2 can we not get the same list as /v2/zones/<zoneid>/nameservers?
17:43:44 <Kiall> vinod1: yea, we can
17:44:32 <Kiall> Okay - Recap... BP to be filed for showing the NS (and SOA?) records in the v2 API - that avoids a breaking API change in the future if we allow customers to white label them. *** Etherpad to be setup with initial load/post-maint scenarios for discussion
17:44:57 <jbratton> thanks, that addresses the remainder of my concerns
17:45:01 <Kiall> I'll email the etherpad out tomorrow, and we can figure out where to go from there...
17:45:39 <Kiall> anything else on those Q's before we move on?
17:46:03 <Kiall> #topic Open Discussion
17:46:18 <Kiall> Okay - Any off agenda items?
17:46:27 <bbaby> i have one
17:46:43 <bbaby> If we are sharing backend BIND server from the existing dns infrastructure, In the mDNS —-> BIND communication, is the BIND server going to act as a slave for tenant Zone?
17:47:14 <Kiall> bbaby: heya
17:47:15 <rjrjr_> yes
17:47:28 <eankutse> mDNS is the master here
17:47:34 <Kiall> So, yes, with mDNS BIND will always be setup as a slave
17:47:57 <eankutse> the "stealth" masters will become slaves
17:48:03 <eankutse> for mDNS to update them
17:48:48 <Kiall> vinod1: that random gate failure - I've finally got some logs to debug it :)
17:48:56 <Kiall> (this was https://bugs.launchpad.net/designate/+bug/1321873 )
17:49:07 <Kiall> vinod1: but before I do, did you get anywhere debugging it?
17:49:27 <vinod1> no i did not make much headway on this bug
17:49:44 <eankutse> bbaby: so this brings up an issue where the BIND stealth master has to both a slave and master (to the other name servers)
17:49:47 <Kiall> Okay .. I'll dig into the logs I've got and see what's happening
17:50:27 <rjrjr_> eankutse, slaves can be masters for other slaves.
17:50:27 <Kiall> eankutse: Didn't we settle on avoiding that scenario where possible?
17:50:29 <timfreund> heh... I spent a few hours trying to recreate that gate failure.  I didn't realize it was a known issue, and I didn't want to say "works on my machine"
17:50:33 <bbaby> eankutse: sure
17:50:47 <eankutse> rjrjr: yes.
17:51:01 <Kiall> timfreund: lol live and learn :)
17:51:01 <eankutse> but I think it takes quite a bit of work to do
17:51:10 <eankutse> maybe?
17:51:13 <jbratton> rjrjr_: not really.. it requires views and notifies don't go out right away
17:51:22 <eankutse> k
17:51:32 <rjrjr_> jbratton: ?
17:51:48 <jbratton> if you define a zone as a slave, it won't send a notify out as far as I know
17:52:19 <eankutse> jbratton: would you clarify a bit?
17:52:20 <bbaby> jbratton: you mean to say that all tenant zone need to go under a specific view under BIND (or underlying DNS infrastructure) ?
17:52:22 <Kiall> jbratton: yea, PowerDNS is certainly that way.. I'm guessing you're talking about BIND being difficult to do that with too
17:52:25 <mugsie> nope - I think we covered that - thats why we have multiple hosts associated with serves
17:52:52 <Kiall> bbaby: No, there's two conversations being had at once I think :)
17:53:05 <jbratton> eankutse: if you want a BIND nameserver to be both a master and a slave for a zone, you have to define two views, the view where it is a slave and the view where it is the master
17:53:16 <jbratton> but the master view won't know when the slave view updates
17:53:19 <jbratton> so you would need to write scripts
17:53:26 <jbratton> it's a hack is what I'm trying to say :)
17:53:28 * mugsie 's brain hurts
17:53:35 <betsy> But I thought we had a way to make it work without that
17:53:43 <Kiall> betsy: we do
17:53:48 <jbratton> we aren't using any stealth masters anymore
17:53:50 <jbratton> with Designate
17:53:59 <betsy> Ok. That’s what I thought
17:53:59 <jbratton> mdns will be our stealth masters
17:54:00 <mugsie> nope, but we can define multiple hosts as one sever
17:54:05 <Kiall> mugsie: multiple hosts is about having N physical servers behind a single NS record name
17:54:46 <mugsie> yeah, which gets rid of the requirments (along with mdns only nofitying in the right region) for the 2nd teir of masters
17:55:18 <Kiall> Okay - Any other off agenda items? 3 mins left before Trove kick us out
17:55:30 <cp16net> :-D
17:55:31 <hub_cap> Kiall: :)
17:55:44 <timfreund> One quick admin question:  is there a project mailing list that I should subscribe to?  You mentioned mailing stuff out earlier.
17:55:44 <mugsie> hub_cap: miss taco mac?
17:55:53 <eankutse> timfreund: Hello :-)
17:55:56 <cweid_> too much taco mac...
17:55:57 <mugsie> timfreund: the openstack-dev list
17:55:59 <hub_cap> mugsie: TACO MAC!!!
17:56:09 <timfreund> Hey eankutse!
17:56:14 <Kiall> timfreund: for the occasional stuff like this it tends to just be direct email.. But we really should be using the openstack-dev list
17:56:26 <timfreund> OK, I'll subscribe there.  Thanks!
17:56:27 <Kiall> 99% of the emailed out stuff is related to meetups ups ;)
17:56:29 <mugsie> Kiall: i assumed you were sending it to the dev list ;)
17:56:36 <rjrjr_> i'd like to revisit N tier next discussion.
17:56:38 <Kiall> mugsie: yep - I will
17:56:49 <mugsie> rjrjr_: cool - lash it on the agenda
17:56:53 <Kiall> rjrjr_: Sure
17:56:55 <bbaby> Kiall: may be you have answered before. Is the designate packaged now (.deb/.rpm)?. I want to try it on poc environment
17:57:16 <Kiall> bbaby: the previous release is packaged in https://launchpad.net/~designate-ppa/+archive/havana
17:57:23 <Kiall> they haven't been updated for icehouse yet
17:57:23 <bbaby> thnx
17:57:47 <Kiall> Okay - That's time I think?
17:57:50 <mugsie> yu[
17:57:53 <mugsie> o/
17:57:54 <Kiall> Any last works?
17:57:56 <Kiall> words*
17:58:05 <Kiall> Guess not ;) Thanks all
17:58:06 <Kiall> #endmeeting