Monday, 2014-09-08

*** darshan104 has joined #openstack-dns00:33
*** darshan104 has quit IRC01:22
*** darshan104 has joined #openstack-dns01:24
*** darshan104 has quit IRC01:34
*** darshan104 has joined #openstack-dns01:35
*** EricGonczer_ has joined #openstack-dns01:38
*** darshan104 has quit IRC01:40
*** nosnos has joined #openstack-dns01:51
*** EricGonczer_ has quit IRC01:54
*** HenryG_ is now known as HenryG02:01
*** nkinder has joined #openstack-dns02:32
*** nkinder has quit IRC02:59
*** darshan104 has joined #openstack-dns03:34
*** darshan104 has quit IRC03:39
*** darshan104 has joined #openstack-dns04:35
*** darshan104 has quit IRC04:40
*** darshan104 has joined #openstack-dns04:51
*** nosnos has quit IRC05:57
*** nosnos_ has joined #openstack-dns06:00
*** darshan104 has quit IRC08:13
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Rollback master backend on all exceptions in multi backend  https://review.openstack.org/11971809:19
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Rollback master backend on all exceptions in multi backend  https://review.openstack.org/11971809:20
*** jordanP has joined #openstack-dns10:19
jordanPhi10:19
ekarlso-jordanP: hey10:20
jordanPis grahamhayes around here ?10:20
jordanPwhat's his usual nick name ?10:20
ekarlso-ping mugsie  :)10:20
jordanPmugsie, :) ping10:20
jordanPekarlso-, thanks for the info10:21
jordanPmugsie, i'll leave a message, ping me back if you have time : I am looking at https://github.com/openstack/designate/commit/47e92c4da5e8bdbec2016fbad7921656e9722e96 You moved the call to policy.init() to designate/service.py. But it seems that designate api and designate central doesn't inherit from a class defined in designate/service.py. Thus the call to policy.init() is never done. For instance, in a fresh devstack, I can"t launch designate centra10:25
jordanPl. Moreoever I get the following stack trace in the default nova notification handler of designate-sink : http://paste.openstack.org/show/108176/10:25
jordanPi am having launch, brb10:26
mugsiejordanP: hey10:30
jordanPmugsie, hey10:30
mugsieyeah - I found this issue with the agent last week10:30
openstackgerritEndre Karlson proposed a change to openstack/designate: Make use of SQLA Core in PowerDNS  https://review.openstack.org/11951610:30
mugsiebut, afik central / api are fine - they should be inheriting from designate/service10:30
mugsielet me dig into the central / api iussue - I will upload my ps for the agent10:31
mugsieyou have logs from the central startup?10:31
jordanPmugsie, central seems to inherit from wsgi.Service10:31
jordanPmugsie, the stack trace looked exactly as http://paste.openstack.org/show/108176/10:32
Kiall"central seems to inherit from wsgi.Service" huh no, that's can't be right?10:32
mugsiejordanP: https://github.com/openstack/designate/blob/master/designate/central/service.py#L7210:32
mugsiethat is from desingate/service10:32
KiallOh.. I guess he means the api inherits from wsgi.Service10:33
jordanPyeah sorry....10:33
KiallWhich it does, humm10:33
KiallWhy aren't we seeing this in the gate?10:33
jordanPi am 100% new to designate, I just want to give it a try10:33
mugsiethe api policy gets init'sd in the middleware afik10:33
KialljordanP: So, designate-central should launch fine - every commit causes a devstack install to be done, and some tests performed against it...  What error are you seeing?10:34
Kiallmugsie: no references to policy in the middleware.py file10:34
jordanPKiall, at the moment I am seeing this error http://paste.openstack.org/show/108176/  when designate sink receives a notification10:35
mugsieoh...10:35
mugsiethis might be a policy config file issue10:36
mugsiecan you paste the full output when you start the api?10:36
mugsie(redact any passwords etc)10:36
jordanPyep10:36
jordanPdesignate api logs : http://paste.openstack.org/show/108184/10:38
jordanPcloud@jordan-test-1:~/devstack$ ls -la /etc/designate/policy.json10:39
jordanP-rw-r--r-- 1 cloud cloud 2251 Aug 29 14:28 /etc/designate/policy.json10:39
jordanPmugsie, ^^10:40
mugsiejordanP: k looking now10:42
jordanPk10:42
Kiallyep.. waiting for a devstack run to complete to have a look myself10:47
jordanPit seems that sink inherits from service.Service, but this is the module defined in "from designate.openstack.common import service"10:49
jordanPand not designate/service.py10:49
KiallReally? That's probably a mistake.. Humm10:50
jordanPyeah it seems so, 90% sure10:50
jordanP:)10:50
KiallOkay.. Could you report a bug for the sink issue? I can work up a fix for that one now10:52
jordanPdesignate central is fine because it doesn't import designate.openstack.common import service10:52
jordanPKiall, I will submit a bug report in an hour or so10:53
jordanPi'll have launch with collegues before, otherwise i'll alone :(10:53
Kiall:D10:53
mugsiejordanP: just did a fresh devstack - central & api both seem to be working fine11:12
mugsiehavent tested sink11:12
Kiallmugsie: I've reproduced, api bails.. and Sink is def wrong too11:19
KiallIt only happens with ?all_tenants=1 - where the policy code is called...11:19
mugsiehum11:20
mugsiethat definitly used to work11:20
KialljordanP: I filed  bug 1366764 for the sink part of this issue, think I have a fix ready...11:21
uvirtbotLaunchpad bug 1366764 in designate "Policy is not initialized in designate-sink" [High,Triaged] https://launchpad.net/bugs/136676411:21
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Ensure designate-sink has policy initialized  https://review.openstack.org/11973511:22
*** mwagner_lap has quit IRC11:23
Kialland bug 1366766 for the API side, which is a more involved fix (at least, if we do it "correctly")11:25
uvirtbotLaunchpad bug 1366766 in designate "Policy is not initialized in designate-api" [High,Confirmed] https://launchpad.net/bugs/136676611:25
openstackgerritEndre Karlson proposed a change to openstack/designate: Make use of SQLA Core in PowerDNS  https://review.openstack.org/11951611:29
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Ensure designate-sink has policy initialized  https://review.openstack.org/11973511:37
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Ensure designate-api has policy initialized  https://review.openstack.org/11973711:37
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Ensure designate-api has policy initialized  https://review.openstack.org/11973711:38
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Ensure designate-api has policy initialized  https://review.openstack.org/11973711:39
Kiall(probably the last review for that ;))11:39
jordanPi am back11:48
jordanPKiall, thanks for the patch, I'll test this right now :)11:52
KialljordanP: sure, there's 2 patches.. 1 for sink and 1 for the api .. They should work :)11:53
*** mwagner_lap has joined #openstack-dns12:02
*** nosnos_ has quit IRC12:13
*** RaginBajin has joined #openstack-dns12:14
*** nosnos has joined #openstack-dns12:14
*** diga has joined #openstack-dns12:14
*** nosnos has quit IRC12:18
*** betsy has joined #openstack-dns12:25
betsyKiall: You around?12:26
KiallHeya12:26
Kiallbetsy: yep, I am12:26
betsygood morning12:26
betsySo, I wanted to ask you about the managed feature12:26
betsyI couldn’t figure out how it is suppose to work. There are about 5 fields for it in the database12:27
betsyIs it written up anywhere?12:27
KiallNo, and I think it may even be "broken" at the minute.. the API should be rejecting calls to update/delete managed=True stuff, but it's not12:28
betsyRight. We talked about that it’s broken and I looked into fixing it, but I’m not sure how it’s supposed to work12:29
betsyCould you open a bug for it and describe how it’s supposed to work?12:29
betsyWho can update it?12:29
betsyAdmins or only a certain project id?12:29
KiallSure.. I can file a bug, in the meantime.. The original intent was actually fairly simple - if managed=True, Nobody should be allowed edit it via the API as the resource is "managed internally" by Designate.12:32
betsyThen how does it get changed if nobody can edit it?12:34
betsyFor example, if the SOA is managed, it still has to be updated every time the serial number changes.12:34
betsyDoes it skip the policy file?12:35
betsyAnyway, I description would help12:35
mugsiebetsy: that would be designate changing it internally, not via the API12:36
KiallSo, the `if managed==True: raise YouCantDoThisException()` would live in the API layer, so internal updates for SOA etc would never hit that code12:37
Kiall(sorry - debugging/discussing about 3 issues at once ;) Hence the short replies :D)12:37
betsy:) No problem.12:37
betsyWhat about the other db fields? How do they fit it?12:37
betsyAnyway. You can write up the bug and I can ask any other questions about it later12:39
*** diga has quit IRC12:49
*** richm1 has joined #openstack-dns13:11
Kiallbetsy: so, we've discovered a fairly serious issue with the expose NS/SOA stuff.. While tracing another issue, I noticed the NS/SOA generation code was still in the powerdns / bind / ipa backends ..13:13
KiallI expected to find duplicates of the NS/SOA's in the DB.. But.. Didn't, it seems anything done with create/update/delete_recordset is effectively ignored by the backends as the methods aren't implemented ..13:13
KiallThings seemed to work, as the SOA/NS generation was still in place, and because the gate uses the V1 API, which does create/update/delete_record rather than create/update/delete_recordset13:14
KiallWorking through what we need to fix it now :(13:14
*** EricGonczer_ has joined #openstack-dns13:30
*** eankutse has joined #openstack-dns13:44
jordanPKiall, so I tested your patched, they seem to work, they got ride of the policy engine not being initialised. Now I get another error when designate-sink receives a notification from nova, the stack trace is here : http://paste.openstack.org/show/108289/13:50
jordanPcould it be related to the problem you mentionned half an hour ago ?13:50
KiallPossibly - It seems we have a pretty serious issue right now that went unnoticed :(13:51
Kiall(and debugging the policy issues you spotted led me to find it! Thanks ;))13:52
jordanPokay, that's good to know13:54
*** timsim has joined #openstack-dns13:59
*** nkinder has joined #openstack-dns14:00
openstackgerritKiall Mac Innes proposed a change to openstack/designate: WIP: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976814:04
Kiallmugsie: .. start of the fix, some tests still failing - haven't dug in yet14:04
mugsiek14:07
openstackgerritKiall Mac Innes proposed a change to openstack/designate: WIP: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976814:10
KiallTests fixed.. Never try to fix something "While you're in there anyway"14:11
mugsie:)14:11
mugsiethats just a good rule for life14:11
betsykiall: let me know if I can help14:28
*** betsy has quit IRC14:29
*** paul_glass has joined #openstack-dns14:33
*** paul_glass has quit IRC14:36
*** paul_glass has joined #openstack-dns14:37
openstackgerritKiall Mac Innes proposed a change to openstack/designate: WIP: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976815:01
*** paul_glass1 has joined #openstack-dns15:06
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976815:09
Kiallmugsie: .. https://review.openstack.org/#/c/119768/4/designate/resources/templates/bind9-zone.jinja215:09
*** paul_glass has quit IRC15:09
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976815:11
*** EricGonc_ has joined #openstack-dns15:11
KiallOkay.. bind+powerdns both work as expected etc now...15:11
*** EricGonczer_ has quit IRC15:11
*** vinod1 has joined #openstack-dns15:21
*** betsy has joined #openstack-dns15:25
*** EricGonczer_ has joined #openstack-dns15:36
*** EricGonc_ has quit IRC15:36
*** mikedillion has joined #openstack-dns15:40
*** mwagner_lap has quit IRC15:46
openstackgerritVinod Mangalpally proposed a change to openstack/designate: _find_recordsets returns active recordsets only  https://review.openstack.org/11950215:53
*** rmoe has quit IRC15:53
*** mikedillion has quit IRC16:04
*** EricGonc_ has joined #openstack-dns16:04
*** EricGonczer_ has quit IRC16:04
*** rmoe has joined #openstack-dns16:08
*** EricGonc_ has quit IRC16:09
*** mikedillion has joined #openstack-dns16:09
*** EricGonczer_ has joined #openstack-dns16:12
vinod1kiall/mugsie: Re https://review.openstack.org/#/c/119502/16:16
vinod1Mugsie mentioned adding a new parameter16:16
mugsievinod1: Kiall has decided he doesn't like the parameter, so an extra item in the criterion should work16:17
vinod1Adding another key/value pair to criterion too looks good16:17
mugsieyeah - the name should be domains_deleted - as it is about if the domain is deleted, not the recordset16:17
vinod1okay - will add a domains_deleted to the criterion16:19
mugsiecool - thanks vinod116:21
*** mikedillion has quit IRC16:28
*** mikedillion has joined #openstack-dns16:30
*** darshan104 has joined #openstack-dns16:39
*** jordanP has quit IRC16:42
openstackgerritVinod Mangalpally proposed a change to openstack/designate: _find_recordsets returns active recordsets only  https://review.openstack.org/11950217:03
Kiallvinod1: Discovered a pretty big issue earlier.. https://review.openstack.org/#/c/119768/ is a review to (hopefully) fix it17:15
Kiallvinod1: you guys know bind better, so the "workaround" I've put in for bind might have a better alternative...17:15
vinod1reviewing it now17:15
*** rjrjr has joined #openstack-dns17:18
*** EricGonc_ has joined #openstack-dns17:19
*** EricGonczer_ has quit IRC17:19
Kiallthanks17:21
*** eankutse has quit IRC17:31
*** mwagner_lap has joined #openstack-dns17:36
*** darshan104 has quit IRC17:47
*** rjrjr has quit IRC17:51
*** eankutse has joined #openstack-dns18:09
*** diga has joined #openstack-dns18:17
*** vinod1 has quit IRC18:33
*** darshan104 has joined #openstack-dns18:36
*** diga has quit IRC18:39
*** vinod1 has joined #openstack-dns18:41
*** vinod1 has quit IRC18:45
*** vinod1 has joined #openstack-dns18:46
*** mikedillion has quit IRC18:49
*** openstackgerrit has quit IRC18:51
Kiallopenstackgerrit - who said you could leave? ;)18:51
timsimslacker...18:52
Kiall;)18:53
*** mikedillion has joined #openstack-dns19:04
*** openstackgerrit has joined #openstack-dns19:06
*** ChanServ sets mode: +v openstackgerrit19:06
*** openstackgerrit has quit IRC19:06
*** openstackgerrit has joined #openstack-dns19:07
*** ChanServ sets mode: +v openstackgerrit19:07
openstackgerritBetsy Luzader proposed a change to openstack/designate:  Downgrade in 039 does not delete NS recordsets  https://review.openstack.org/11989019:34
*** zain has joined #openstack-dns19:37
openstackgerritA change was merged to openstack/designate: _find_recordsets returns active recordsets only  https://review.openstack.org/11950219:38
*** amcrn has joined #openstack-dns19:40
*** darshan104 has quit IRC20:18
betsyKiall: for https://review.openstack.org/#/c/119890/, and filtering on individual records — what would I look for, if the record data == the server name, delete?20:25
KiallI think you need to find the recordset ID, then `DELETE FROM records where recordset_id = <ID> AND managed = True`20:26
Kiall(or do a join and DELETE ... WHERE recordsets.name = <name> AND recordsets.type = <type> AND records.managed = True`20:26
Kiallre managed=True - They are, L152 shows the migration added records are getting managed=True20:27
betsyThat would work, except for one thing…20:27
betsyWhen I’m looking at mysql, all the managed are set to 0, not 120:28
KiallYea, zones added after that won't get managed = True.. Which sucks.. But we have no way to tell those apart from user added ones :(20:28
KiallIf you add zones before migration 39, they'll have it20:28
betsyNope, they don’t20:28
betsyI just redid this vagrant box with 3920:28
betsyThey’re set to managed=True in the code, but the db is still set to 020:29
KiallHumm, That's odd.. The code looks to be doing it20:29
Kiallif that's not working for some reason, then, we have no way to tell the difference :(20:30
betsyBad word! Bad word!20:31
betsyOh, never mind!20:32
betsyI just ran the upgrade again and it worked!20:32
betsyI scared myself20:32
* Kiall hides.. Not-deterministic migrations are always scary ;)20:33
betsyI’d been playing with the downgrade and upgrade too many times testing the bug I just submitted20:33
*** vinod1 has quit IRC20:33
betsySo, can I filter on the records being managed = true and type = ns and that’s it?20:33
KiallWe probably also want to add managed=True here, so future ones get marked too: https://github.com/openstack/designate/blob/master/designate/central/service.py#L41120:34
betsyOr do I still need to check for root delegation?20:34
KiallWell, we only add NS's at the root, so probably best to find just those20:34
* Kiall hates Jinja2's whitespace control20:34
betsyWould there be any that were managed = true that are not root?20:35
betsyOr is that not a good assumption?20:35
betsyAnd I’ll fix the managed = true in central for ns and soa20:36
KiallIn theory, there could be.. Either in the future we add some, or some sink plugin is creating them, etc etc - We wouldn't want to accidentally purge those :)20:36
*** darshan104 has joined #openstack-dns20:36
betsyThat’s true. Ok.20:36
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976820:39
Kiallvinod: Just fixed the whitespace issue in the bind9 template... ^20:39
betsyKiall: One more question20:40
betsyIf I delete the records instead of the recordsets, wouldn’t that leave empty NS recordsets in the db?20:40
Kiallvinod: logs at .. http://paste.openstack.org/show/108452/20:40
KiallYes, but that's a much better position than deleting customer data :) We could check if there are no more records in the RRSet, then delete it too if there are none20:41
betsyok20:42
Kiall(We have customers with NS records at the root, deleting them would be disastrous ;))20:42
betsySo back to the drawing board. :)20:45
*** HenryG has quit IRC20:48
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976820:55
Kiallvinod: also spotted your (2) part in the comment + fixed20:55
*** HenryG has joined #openstack-dns21:00
betsyKiall: With expose NS and SOA, you cannot update the root NS recordset, which means you can’t add a non-managed record to it21:05
*** amcrn has quit IRC21:05
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Support filtering on Zone/RecordSet status  https://review.openstack.org/11015721:05
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Support filtering on Zone/RecordSet status  https://review.openstack.org/11015721:07
*** mwagner_lap has quit IRC21:08
Kiallbetsy: Wasn't that just a temp restriction, until we fixed managed?21:08
betsyYeah, I guess it was.21:08
betsyOk. I’ll just ignore that for now and change the db by hand, so I can test my fix21:09
*** arborism has joined #openstack-dns21:10
*** arborism is now known as amcrn21:11
*** EricGonc_ has quit IRC21:20
*** eankutse has quit IRC21:29
*** darshan104 has quit IRC21:35
*** darshan104 has joined #openstack-dns21:37
*** mikedillion has quit IRC21:46
*** zain has quit IRC21:47
*** zain1 has joined #openstack-dns21:47
*** vinod1 has joined #openstack-dns21:48
*** zain1 has quit IRC21:53
*** betsy has quit IRC22:03
*** timsim has quit IRC22:09
*** darshan104 has quit IRC22:15
*** darshan104 has joined #openstack-dns22:15
*** mwagner_lap has joined #openstack-dns22:24
*** amcrn_ has joined #openstack-dns22:30
*** paul_glass1 has quit IRC22:32
*** amcrn has quit IRC22:32
*** zain has joined #openstack-dns22:38
*** vinod1 has quit IRC22:49
*** eankutse has joined #openstack-dns22:52
*** zain has quit IRC22:54
openstackgerritA change was merged to openstack/designate: Rollback master backend on all exceptions in multi backend  https://review.openstack.org/11971822:59
*** amcrn_ is now known as amcrn23:01
*** eankutse has quit IRC23:21
*** eankutse has joined #openstack-dns23:21
*** betsy has joined #openstack-dns23:35
*** nkinder has quit IRC23:42
*** eankutse has quit IRC23:45
*** richm1 has quit IRC23:47
*** eankutse has joined #openstack-dns23:48
*** betsy has quit IRC23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!