*** zigo has quit IRC | 00:07 | |
*** rmoe has quit IRC | 00:07 | |
*** jmcbride has joined #openstack-dns | 00:12 | |
*** zigo has joined #openstack-dns | 00:13 | |
*** rmoe has joined #openstack-dns | 00:24 | |
*** EricGonczer_ has joined #openstack-dns | 01:41 | |
*** tryggvil has quit IRC | 01:42 | |
*** richm has quit IRC | 01:53 | |
*** nosnos has joined #openstack-dns | 02:02 | |
*** EricGonczer_ has quit IRC | 02:46 | |
*** nosnos has quit IRC | 03:26 | |
*** nosnos has joined #openstack-dns | 03:27 | |
*** nosnos has quit IRC | 03:32 | |
*** nosnos has joined #openstack-dns | 04:16 | |
*** k4n0 has joined #openstack-dns | 05:50 | |
*** ryanpetrello has quit IRC | 05:55 | |
*** ryanpetrello has joined #openstack-dns | 05:55 | |
*** amcrn has quit IRC | 07:20 | |
*** bauruine_ has quit IRC | 07:46 | |
*** jordanP has joined #openstack-dns | 07:46 | |
*** openstackgerrit has quit IRC | 08:34 | |
*** openstackgerrit has joined #openstack-dns | 08:34 | |
*** ChanServ sets mode: +v openstackgerrit | 08:34 | |
*** viktors has joined #openstack-dns | 09:07 | |
*** mugsie has quit IRC | 09:27 | |
*** mugsie has joined #openstack-dns | 09:29 | |
*** tryggvil has joined #openstack-dns | 09:30 | |
*** Wiscontiy has joined #openstack-dns | 09:42 | |
Kiall | rjrjr: I'm personally happy to merge any Pools code that doesn't break existing code - even if the code isn't "usable" without more code.. e.g. your code shouldn't have to wait in the review queue too long :) | 09:51 |
---|---|---|
rjrjr | kiall: okay. give me about 1 hour. i have the pool manager database worked out. just going through the code now fixing it up. | 09:52 |
Kiall | rjrjr: cool :) I have a few hours later set aside to do reviews | 09:54 |
Kiall | Also .. Why are you awake? lol | 09:54 |
Kiall | 3am! | 09:54 |
*** Wiscontiy has quit IRC | 09:58 | |
*** nosnos has quit IRC | 10:02 | |
*** nosnos has joined #openstack-dns | 10:03 | |
*** nosnos has quit IRC | 10:08 | |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add Server object validations https://review.openstack.org/129625 | 10:08 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Implement a DesignateObject Registry https://review.openstack.org/129599 | 10:08 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add basic validation functionality to Designate Objects https://review.openstack.org/127846 | 10:08 |
rjrjr | i have my day job and this is my night job. 8^) | 10:12 |
openstackgerrit | Ron Rickard proposed a change to openstack/designate: Pool Manager Service Changes https://review.openstack.org/128300 | 10:32 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add Server object validations https://review.openstack.org/129625 | 10:34 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add basic validation functionality to Designate Objects https://review.openstack.org/127846 | 10:34 |
Kiall | rjrjr: Q - PoolManagerBackend and Bind9PoolManagerBackend etc feel a little "weird" to me, is the intent to always keep both kinds of backends in place, or will PoolManagerBackend's become the only kind of backends? | 10:40 |
rjrjr | I think these become the only backends once pool manager is working. | 10:44 |
rjrjr | so, PoolManagerBackend becomes Backend and Bind9PoolManagerBackend becomes Bind9Backend. | 10:46 |
Kiall | Okay - I'll have a proper look later today, that was the only thing that immediately jumped out at me at a glance :) | 10:46 |
rjrjr | i saw we did something similar with PowerDNSMDNSBackend. | 10:47 |
rjrjr | except I needed another base class to encapsulate registering the configuration options. | 10:48 |
Kiall | Kinda :) PowerDNSMDNSBackend extends the standard Backend class, while this adds a whole new hierarchy that doesn't pass through the standard Backend class :) | 10:49 |
Kiall | Maybe Backend -> PoolBackend -> Bind9PoolBackend? | 10:49 |
Kiall | (reduces the boilerplate duplication and still offers a place for the shared config logic..) | 10:50 |
Kiall | Anyway .. As I said, I haven't done a proper review .. Just a quick glance, that might cause more pain ;) | 10:50 |
rjrjr | i didn't want a bunch of empty methods. 8^) 90% of the methods in Backend are not needed going forward. | 10:50 |
Kiall | Yep - PoolBackend would stub them out, so the "real" backends don't need to implement them :) | 10:51 |
Kiall | Anyway.. brb! | 10:51 |
rjrjr | i can do Backend -> PoolBackend -> Bind9PoolBackend | 11:00 |
rjrjr | zuul/gerrit/jenkins is taking its good old time again. i want to make sure this passes testing so i can go to bed. lol | 11:01 |
Kiall | hah.. you can always run them locally :) `tox -r -epy27,pep8` would give you a clean run the same as the gate | 11:07 |
ekarlso | what's the diff between the backend > poolbackend ? | 11:09 |
*** alokj has joined #openstack-dns | 11:10 | |
rjrjr | i did flake8 and tox -e py27 -r. i'll run pep8 too and head to bed. | 11:12 |
Kiall | flake8 and pep8 are the same really.. 2 names for the same check | 11:13 |
Kiall | pep8 is there because the gate used to do the actual pep8 tool, and it stuck when people moved to flake8 | 11:14 |
*** mwagner_lap has quit IRC | 11:14 | |
ekarlso | rjrjr: is poolbackend a newer kind of thing ? | 11:15 |
rjrjr | i've had my tests pass locally and then they fail on the review. mostly this happens with the import stuff at the top of the files. | 11:15 |
rjrjr | ekarlso: poolbackend is backend stripped of 90% of the methods we don't need anymore (tsigkeys, recordset, record, server CRUD). additionally, i use it to get the backend configurations for pool manager to instantiate the server (currently pool_server) objects. | 11:17 |
ekarlso | ah ok | 11:18 |
Kiall | rjrjr: usually that's caused by having out of date requirements, tox `-r` causes it to reinstall all deps, which helps avoid it :) | 11:18 |
rjrjr | i had this problem just last week and i always run a tox -e py27 -r and flake8 right before i push the code for review. | 11:19 |
Kiall | Oh.. Weird.. | 11:19 |
rjrjr | i'm just lucky i guess. lol | 11:20 |
rjrjr | the exact problem was i had the different imports mushed together without blank lines. it passed locally. it failed on review. | 11:21 |
rjrjr | i don't remember the exact file, but it was from oslo.XXX followed by a from designate.XXX without a blank line between or something like that. | 11:22 |
ekarlso | https://bugs.launchpad.net/designate/+bug/1383670 < I added that if anyone has opinions : ) | 11:25 |
uvirtbot | Launchpad bug 1383670 in designate "Migrate to Alembic for Migrations" [Wishlist,New] | 11:25 |
rjrjr | well, the tox tests passed, so i'm going to take the chance and go to bed. i'll be back on in 5-6 hours. | 11:26 |
*** k4n0 has quit IRC | 11:37 | |
*** k4n0 has joined #openstack-dns | 11:38 | |
*** k4n0 has quit IRC | 11:46 | |
*** k4n0 has joined #openstack-dns | 11:46 | |
*** ttrumm has joined #openstack-dns | 11:48 | |
*** EricGonczer_ has joined #openstack-dns | 12:03 | |
*** mwagner_lap has joined #openstack-dns | 12:11 | |
*** EricGonczer_ has quit IRC | 12:12 | |
*** paul_glass has joined #openstack-dns | 12:14 | |
*** k4n0 has quit IRC | 12:28 | |
*** ttrumm has quit IRC | 12:30 | |
*** richm has joined #openstack-dns | 13:06 | |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Implement a DesignateObject Registry https://review.openstack.org/129599 | 13:09 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add basic validation functionality to DesignateObjects https://review.openstack.org/127846 | 13:09 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Support Nested/Recursive Object Validations https://review.openstack.org/129895 | 13:09 |
Kiall | I love when changes end up easier than you expect :D https://review.openstack.org/#/c/129895/ is a perfect example of one of those things I thought would be a PITA ;) | 13:11 |
*** EricGonczer_ has joined #openstack-dns | 13:26 | |
*** jmcbride has quit IRC | 13:37 | |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add Server object validations https://review.openstack.org/129625 | 13:47 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add Domain object validations https://review.openstack.org/129909 | 13:47 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add Server object validations https://review.openstack.org/129625 | 13:48 |
openstackgerrit | Kiall Mac Innes proposed a change to openstack/designate: Add Domain object validations https://review.openstack.org/129909 | 13:48 |
*** paul_glass has quit IRC | 13:49 | |
*** paul_glass has joined #openstack-dns | 13:59 | |
*** rmoe has quit IRC | 14:19 | |
*** rmoe has joined #openstack-dns | 14:24 | |
*** jmcbride has joined #openstack-dns | 14:28 | |
*** timsim has joined #openstack-dns | 14:31 | |
*** alokj has quit IRC | 14:41 | |
*** rmoe has quit IRC | 14:41 | |
*** rmoe has joined #openstack-dns | 14:43 | |
*** vinod1 has joined #openstack-dns | 14:49 | |
*** jmcbride has quit IRC | 14:59 | |
*** jmcbride has joined #openstack-dns | 15:00 | |
*** paul_glass has quit IRC | 15:03 | |
openstackgerrit | A change was merged to openstack/designate: Reduce SQLAlchemy loglevel during tests https://review.openstack.org/128278 | 15:03 |
*** vinod1 has quit IRC | 15:04 | |
*** EricGonc_ has joined #openstack-dns | 15:06 | |
*** EricGonczer_ has quit IRC | 15:06 | |
*** timsim has quit IRC | 15:09 | |
*** timsim has joined #openstack-dns | 15:12 | |
*** vinod1 has joined #openstack-dns | 15:12 | |
*** vinod1 has quit IRC | 15:16 | |
*** jmcbride has quit IRC | 15:17 | |
*** jmcbride1 has joined #openstack-dns | 15:18 | |
*** vinod1 has joined #openstack-dns | 15:21 | |
*** vinod1 has quit IRC | 15:41 | |
*** vinod1 has joined #openstack-dns | 15:44 | |
*** rmoe has quit IRC | 15:47 | |
*** paul_glass has joined #openstack-dns | 15:56 | |
*** viktors is now known as viktors|afk | 15:57 | |
*** tryggvil has quit IRC | 15:58 | |
Kiall | Humm.. The 4 pools patches weigh in at about 4k lines of code.. I only managed to set aside an hour to review them, that's not enough -_- | 16:02 |
*** tryggvil has joined #openstack-dns | 16:03 | |
*** rmoe has joined #openstack-dns | 16:09 | |
Kiall | betsy: about? | 16:11 |
betsy | yep | 16:12 |
Kiall | Heya - Looking at your Server Pools Storage change.. | 16:13 |
betsy | got a question? | 16:13 |
Kiall | I'm still reviewing, but I think we might want to extend that review to actually remove the old "servers" stuff all together | 16:14 |
betsy | ok. I was gonna do that in a separate patch | 16:14 |
betsy | Trying to keep them smaller. The code work needs to be done, too, before all the old server stuff is removed | 16:15 |
Kiall | For me anyway, Ideally, a single migration handles the new tables, data move, and deletion of the old tables.. | 16:15 |
betsy | ok | 16:15 |
Kiall | Yea... | 16:15 |
betsy | I can do that | 16:15 |
Kiall | Anyway - Still reviewing :) | 16:15 |
betsy | Oh here’s what I ran into with that... | 16:15 |
Kiall | Was just curious if you had though it it.. and it seems you have ;) | 16:15 |
betsy | To make the change to the domain tables, I need the pool_id from the pool table | 16:16 |
Kiall | Ah.. My draft comment has a suggestion for that | 16:16 |
betsy | Oh cool. | 16:16 |
Kiall | let me publish that comment real quick | 16:16 |
betsy | Yeah. I was thinking it needed to be separate patches because of that | 16:16 |
Kiall | Done.. | 16:17 |
Kiall | The suggestion basically boils down to pre-choosing the default pool's UUID | 16:17 |
Kiall | That allows you to set the well-known UUID as the default value of the "default_pool_id" config option, and the migration can read from the config to get the ID it uses, allowing end users to "choose" a different one if they wish.. Similar to what Keystone a while back for a similar migration | 16:18 |
betsy | Ok. Can I read the config file from this to get the default pool name? | 16:19 |
Kiall | Yep - You should just be able to import the CONF object like any other place | 16:19 |
betsy | So, take all of the create the default pool out of central/service.py | 16:20 |
Kiall | I hadn't got that far :D | 16:20 |
betsy | ok | 16:20 |
Kiall | But, pre-creating it let's us move stuff (e.g. servers) around and update every domain with pool_id = <UUID> during the migration | 16:21 |
Kiall | rather than slowly as stuff get's updated over time (which could be years :D) | 16:21 |
betsy | Yeah, I was planning on doing it in two separate migrations becuase of the pool_id, but this solves that problem, as long as I can read the conf file to get the default pool name. The spec just has it being created in the central service | 16:23 |
Kiall | Really? I missed that, I would have said something about that :D | 16:24 |
betsy | Well, unless I misread the spec. :) | 16:24 |
Kiall | (specifically, updating all N thousand domains pool_id's to the just-created pool's ID really needs to be done in the migration) | 16:24 |
betsy | Right. I was just going to have it as a separate migration. After this one was in place, and the defaut pool was created, then the subsequent migration would read that pool_id and update all the domains’ pool_ids | 16:26 |
Kiall | (In case you can't tell - I'm trying to see how we can get stuff merged without duplicating efforts or breaking things between merges!) | 16:26 |
betsy | But if we can do it in one fell swoop, I’m fine with that | 16:26 |
*** jordanP has quit IRC | 16:26 | |
betsy | Well, I don’t think this patch would break anything because there’s no code change | 16:26 |
Kiall | and - your review kinds sits at at the core - so likely needs to go in first ;) | 16:26 |
betsy | More work needs to be done after this | 16:27 |
betsy | But if you want it done all in one patch, I’m fine with that too | 16:27 |
Kiall | Yea - I *personally* think it's better in this case to handle removing servers in the same review, as the pools code you have in place seems to provide everything necessary to make that happen, and we get the advantage of an "atomic" 1 migration to remove the old and add the new :) | 16:28 |
betsy | Cool. I make those changes | 16:30 |
*** OpenStack_ has joined #openstack-dns | 16:31 | |
Kiall | Some more comments up BTW - But looks great, the sooner we get this merged, the sooner others can build on it :) | 16:31 |
*** vinod1 has quit IRC | 16:31 | |
*** OpenStack_ has left #openstack-dns | 16:32 | |
Kiall | betsy: actually, looking at the servers code that would have to be removed, I'm not sure about removing servers stuff straight away.. More to that than I expected! | 16:39 |
betsy | Yeah | 16:40 |
Kiall | But... I think the other comments around creating the default pool etc still apply? | 16:40 |
betsy | Sure. I’ll go ahead and do that and then do the server changes in a separate patch | 16:40 |
betsy | Does that work? | 16:40 |
betsy | So all the migration stuff is done at once | 16:40 |
betsy | Well, except for dropping the server table, which would be done later | 16:41 |
Kiall | Yep.. We'll probably still need another migration to move the servers data, but yea, makes sense | 16:41 |
betsy | ok | 16:41 |
Kiall | timsim: about? Looking at your DB indexing patch, I think we can reduce the # of index's created by removing some of the "duplicates" e.g.: | 16:45 |
Kiall | ['zone_id_tenant', zones_table.c.id, zones_table.c.tenant_id], | 16:45 |
Kiall | ['zone_id_tenant_deleted', zones_table.c.id, zones_table.c.tenant_id, | 16:45 |
Kiall | zones_table.c.deleted], | 16:45 |
Kiall | the second index there covers off the first index, so the first index won't improve read performance, but it will reduce write performance | 16:46 |
Kiall | An index will be chosen based on the # of matches to the left hand side, e.g. a query with `WHERE id = 1 AND tenant_id = 2` can be satisfied by either of those indexes, and a query on `WHERE id = 1 AND deleted = 0` won't be satisfied by either index | 16:48 |
Kiall | (though, ID is bad example ;) Since it will produce exactly 1 hit anyway :P) | 16:48 |
timsim | Kiall: Yeah I didn't want to present those as a legit answer. I did a barbaric analysis of some queries and blindly created those. There are certainly some duplicates there, and some that I'm missing I'm sure. | 16:49 |
Kiall | :D | 16:49 |
timsim | I'm trying to find time to do a more in depth query analysis and go over some of the stuff with one of our DBAs so I can get a more definitive set of indices to create | 16:50 |
Kiall | Yea, DB indexing can be a bit of a black art sometimes.. lol | 16:51 |
Kiall | http://www.percona.com/doc/percona-toolkit/2.1/pt-duplicate-key-checker.html <-- Likely a good way to remove "duplicates" quicky | 16:51 |
timsim | I have almost no experience with it so I was just kind of flailing around in the dark. | 16:51 |
timsim | Nice, I'll definitely use that. | 16:52 |
Kiall | percona's toolkit is GREAT.. If you set the MySQL slow query log to 0 seconds, then point pt-query-digest at it, you can get some really useful pointers | 16:53 |
Kiall | Oh.. there's a new PT tool I didn't know of.. http://www.percona.com/doc/percona-toolkit/2.2/pt-index-usage.html | 16:55 |
Kiall | "This tool connects to a MySQL database server, reads through a query log, and uses EXPLAIN to ask MySQL how it will use each query. When it is finished, it prints out a report on indexes that the queries didn’t use." | 16:55 |
timsim | That's good advice, I'll do that. I was going to turn on query logging and do some tests and see what queries single API requests/MiniDNS ops execute. | 16:55 |
timsim | That looks useful. | 16:55 |
Kiall | Yea, everytime I go looking for something MySQL related.. Percona has the answer ready to use ;) | 16:56 |
timsim | I hadn't thought to look. I'll be looking at all those tools. | 16:57 |
*** mwagner_lap has quit IRC | 16:58 | |
*** paul_glass has quit IRC | 17:07 | |
*** zigo has quit IRC | 17:08 | |
*** paul_glass has joined #openstack-dns | 17:08 | |
*** zigo has joined #openstack-dns | 17:08 | |
*** paul_glass has quit IRC | 17:08 | |
*** vinod1 has joined #openstack-dns | 17:11 | |
*** skyler_ has joined #openstack-dns | 17:12 | |
*** paul_glass has joined #openstack-dns | 17:13 | |
*** paul_glass has quit IRC | 17:14 | |
vinod1 | Kiall, while you are up reviewing code - could you also review https://review.openstack.org/#/c/125868/ | 17:15 |
vinod1 | I updated the code so that it works both with the current setup and going forward with the pool managers | 17:16 |
vinod1 | Also the object code has reached a point (for me) where a document would be helpful to understand what is going on :-) | 17:17 |
*** jmcbride1 has quit IRC | 17:23 | |
*** jmcbride has joined #openstack-dns | 17:26 | |
*** ryanpetrello has quit IRC | 17:27 | |
*** amcrn has joined #openstack-dns | 17:27 | |
Kiall | vinod1: yea, I thought the same! There's still more to go to actually make use of it and replace the existing validation throughout | 17:29 |
Kiall | On the plus side, it's uncovering dodgy code and tests already ;) | 17:30 |
*** jmcbride has quit IRC | 17:31 | |
*** ryanpetrello has joined #openstack-dns | 17:31 | |
Kiall | I'll try write something up this evening, for now though, time to head home before the rain starts! | 17:32 |
*** zigo has quit IRC | 17:41 | |
*** zigo has joined #openstack-dns | 17:41 | |
*** mwagner_lap has joined #openstack-dns | 17:42 | |
rjrjr | why did the code I submitted 7 hours ago never get reviewed by gated? | 17:44 |
rjrjr | reviewed = tested | 17:46 |
rjrjr | gated = zuul/jenkins | 17:50 |
*** jmcbride has joined #openstack-dns | 17:54 | |
vinod1 | rjrjr: Looks like it just got done - Oct 21, 2014 12:53 PM | 17:56 |
vinod1 | the other designate patchsets too are sitting there for the last 5 hrs | 17:56 |
rjrjr | yeah, i ran a 'recheck no bug'. glad that worked! | 17:58 |
rjrjr | is there a URL for the zuul console to look at the status of the checks? we run these apps inhouse and i can look at the status to see what zuul is working on. | 17:59 |
vinod1 | http://status.openstack.org/zuul/ | 18:01 |
vinod1 | You can set a filter to designate | 18:01 |
vinod1 | All the waiting tests are currently waiting for the devstack tests to complete | 18:02 |
*** jmcbride has quit IRC | 18:16 | |
*** paul_glass has joined #openstack-dns | 18:24 | |
*** timsim has quit IRC | 18:32 | |
*** timsim has joined #openstack-dns | 18:36 | |
*** jmcbride has joined #openstack-dns | 18:44 | |
*** jmcbride has quit IRC | 18:46 | |
*** jmcbride has joined #openstack-dns | 18:46 | |
*** paul_glass has quit IRC | 19:05 | |
*** paul_glass has joined #openstack-dns | 19:09 | |
Kiall | rjrjr: looks like openstack infra had some sort of failure earlier, queues have been backed up for nearly 24 hours | 19:28 |
Kiall | with 800 VMs doing testing.. lol | 19:28 |
Kiall | And I dislike when our internal zuul doesn't start on a change immediately ;) | 19:28 |
*** amcrn has quit IRC | 19:54 | |
*** jmcbride has quit IRC | 20:00 | |
*** jmcbride has joined #openstack-dns | 20:01 | |
*** vinod1 has quit IRC | 20:02 | |
*** jmcbride has quit IRC | 20:05 | |
*** paul_glass has quit IRC | 20:06 | |
*** RaginBaj_ has quit IRC | 20:12 | |
*** tryggvil has quit IRC | 20:28 | |
*** drawesome has joined #openstack-dns | 20:29 | |
*** jmcbride has joined #openstack-dns | 20:29 | |
*** paul_glass has joined #openstack-dns | 20:44 | |
*** paul_glass1 has joined #openstack-dns | 20:49 | |
*** paul_glass has quit IRC | 20:50 | |
*** EricGonc_ has quit IRC | 21:08 | |
*** timsim has quit IRC | 21:13 | |
*** paul_glass1 has quit IRC | 21:38 | |
*** tryggvil has joined #openstack-dns | 21:40 | |
*** amcrn has joined #openstack-dns | 21:54 | |
*** vinod1 has joined #openstack-dns | 22:01 | |
*** vinod1 has quit IRC | 22:11 | |
*** EricGonczer_ has joined #openstack-dns | 22:22 | |
*** vinod1 has joined #openstack-dns | 22:22 | |
*** EricGonczer_ has quit IRC | 22:29 | |
*** jmcbride has quit IRC | 22:37 | |
*** vinod1 has quit IRC | 22:40 | |
*** bauruine has quit IRC | 23:07 | |
*** bauruine has joined #openstack-dns | 23:08 | |
*** skyler_ has quit IRC | 23:16 | |
*** EricGonczer_ has joined #openstack-dns | 23:44 | |
*** EricGonczer_ has quit IRC | 23:49 | |
*** rmoe has quit IRC | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!