Thursday, 2018-04-19

« Wednesday, 2018-04-18 Index Friday, 2018-04-20 »
*** kbyrne has quit IRC01:23
*** kbyrne has joined #openstack-dns01:23
*** AlexeyAbashkin has joined #openstack-dns01:39
*** AlexeyAbashkin has quit IRC01:43
*** Leo_m has quit IRC01:54
*** masuberu has joined #openstack-dns02:27
*** daidv has joined #openstack-dns02:30
*** masber has quit IRC02:31
*** caowei has joined #openstack-dns02:53
*** daidv has quit IRC03:11
*** masuberu has quit IRC03:23
*** masuberu has joined #openstack-dns03:33
*** masber has joined #openstack-dns03:35
*** masuberu has quit IRC03:38
*** itlinux has joined #openstack-dns06:02
*** itlinux has quit IRC06:09
*** itlinux has joined #openstack-dns06:12
*** pcaruana has joined #openstack-dns06:21
*** itlinux has quit IRC06:32
*** itlinux has joined #openstack-dns06:37
*** daidv has joined #openstack-dns06:53
*** daidv has quit IRC06:55
*** itlinux has quit IRC07:02
*** AlexeyAbashkin has joined #openstack-dns07:26
*** pcaruana has quit IRC07:45
*** pcaruana has joined #openstack-dns07:46
*** daidv has joined #openstack-dns08:08
*** trungnv has joined #openstack-dns08:08
*** trungnv has quit IRC08:15
*** trungnv has joined #openstack-dns08:16
*** itlinux has joined #openstack-dns08:27
openstackgerritIvan Kolodyazhny proposed openstack/designate-dashboard master: Use Horizon UT framework  https://review.openstack.org/56252508:56
*** abalutoiu has joined #openstack-dns09:07
*** itlinux has quit IRC09:09
*** trungnv has quit IRC10:22
*** caowei has quit IRC10:38
openstackgerritJens Harbott (frickler) proposed openstack/designate-dashboard master: Fix lower-constraints.txt  https://review.openstack.org/56254510:47
*** rfreire has joined #openstack-dns10:49
rfreiremugsie, hello there Graham!10:49
rfreiremugsie, well. You are the reason I am staying another night in Vancouver :-P10:49
rfreireOne of your presentations is the _last one_ on thursday!10:49
mugsieis it?11:02
* mugsie hasn't looked yet :)11:02
mugsieThu 24, 5:30pm - 6:10pm - wow it is the very last slot11:08
rfreiremugsie, ah yes it is!!!11:10
openstackgerritJens Harbott (frickler) proposed openstack/designate-dashboard master: Use Horizon UT framework  https://review.openstack.org/56252511:10
rfreiremugsie, no moar flites to Brazil after noon, so... Well, I will have to indulge myself another night over there11:10
rfreire(to the hate of my wife!)11:10
mugsie:)11:10
rfreiremugsie, I'll buy you a beer over there. /me owes that.11:11
mugsieI am not flying back for a few days after, so it is OK for me :)11:11
rfreirehahaha great! ;-D11:11
mugsierfreire: I will always take a beer11:11
rfreiremugsie, deal. I'll introduce some friends of mine which are tagging along too.11:12
rfreiremugsie, hey Graham i was asked something else here, putting Designate in HA >_<11:12
rfreireHow different/hard is doing that?11:12
rfreireI have looked https://docs.openstack.org/designate/latest/admin/ha.html and that looks somewhat... vague11:13
rfreireNot up to my desig-fu ;-/11:13
mugsieit is pretty easy11:13
mugsieeverything scales horizontally11:13
rfreireTellmemoar!11:13
mugsiejust boot more of each service, and connect to the same Rabbit / MySQL11:13
rfreireno way11:14
mugsieapart from designate producer11:14
mugsiewhich needs a co-ordination backed11:14
rfreireproducer -> the node that will run the rndc stuffs right?11:14
mugsieno - it co-ordinates tasks11:14
mugsiethe worker does the rndc stuff11:15
mugsiehttps://docs.openstack.org/designate/latest/admin/ha.html#designate-producer has links to the tooz docs11:15
mugsieright now, that is memcached, redis or zookeeper11:15
mugsiezookeeper is best, but can be complex to set up, redis is OK and easy enough, memcached is really easy, but not great reliability11:16
rfreirehrnmmm. Think I'll resort to memcached as it is a staple in RHOSP scenarios11:17
mugsieyeah11:17
mugsiethere is work on etc3 happening at the moment, but that will be a rocky thing11:17
rfreireugh hehe11:17
rfreireAlrighto11:17
rfreirePackstack burning CPU here, will give it a try in a few minutes.11:18
rfreireHUGE thanks Graham! ;-)11:18
mugsienp :)11:18
mugsieif you see ways we can improve the HA docs, please file a bug with details, or send up a patch :)11:18
rfreiremugsie, well. I'm that ol' sysadmin guy11:19
rfreirewho scripts things11:19
rfreiredo this this this that, configure this this this that, run, running done11:19
rfreireI can surely and happilly share this doc (as I did with my designate basic install)11:19
rfreirefor what is worth, it was a uphill battle to have RHT to update the (BROKEN) Designate config guide11:20
mugsiecool - any extra details on how people use it is welcome :)11:20
mugsierfreire: oh? what was broken?11:20
rfreirenot broken11:20
rfreire_HEAVILY BROKEN_11:20
rfreireI came here to whine for help because I followed the guide step by step and I  ended with a non-functional designate :-P11:21
rfreire(so did my custy)11:21
rfreiremugsie, fixed: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html/dns-as-a-service_guide/index11:22
rfreireSingle-page: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/dns-as-a-service_guide/index11:22
mugsieneed to make sure the upstream docs are up to date as wellk11:24
mugsieas well*11:24
rfreiremugsie, for what is worth: My custys use case: They have every component in HA (whenever possible), 3 nodes. F5 load balances the requests.11:24
mugsiebnemec: do you know who could get us a diff of the changes ?11:24
mugsierfreire: sounds like a sane setup11:24
rfreiremugsie, well, this one works11:24
rfreire;-)11:24
mugsiewas our default setup in HPE11:24
rfreiremugsie, you left hpe after microfocus spinoff?11:25
rfreireI saw somewhere that you are in Verizon, I think?11:25
mugsieyeah, in Verizon now - was part of the HPE team that went to Suse before the spinoff11:25
rfreireLots of HPE guys leaving;  hired two ex HPers down here in Brazil in the last two months11:26
rfreireCarlos Schimidt11:26
rfreireand Clesio Takahashi11:26
rfreireClesio used to work with the CMP11:26
rfreireand Carlos was a openstack guy11:26
mugsieyeah - it is unfortunate - HP was a great company before all the spin offs11:27
rfreireword11:27
mugsieah, I have seen Carlos's blog before11:28
rfreirewe nicknamed him 'piano carrier'11:28
* mugsie was part of the HP Public Cloud team back in the day :)11:28
rfreireHa!11:28
rfreireI stole Carlos from HPE11:28
mugsiewe ran designate in production back in 201311:29
rfreirethere was a issue in one of my acctys where some BL660c randomly rebooted with UMCE (unexpected machine check exception)11:29
rfreireand HPE sent Carlos to check their side11:29
rfreirebut instead of looking to hardware (which was the culprit in the end), Carlos was reviewing the OS and the Openstack config11:29
rfreireI FUMED with that11:29
rfreireBut I noticed, hnm, this Carlos guy is great!11:30
rfreireInvited him to take part of a selection and is now a consultant, implementing openstack11:30
rfreireAnd he will never again sniff my OpenStack environments haha.11:30
mugsie:D11:31
rfreireAlrighto packstack's done, time to fiddle with this HA thingy11:32
*** mugsie has quit IRC11:42
*** mugsie has joined #openstack-dns11:42
*** mugsie has quit IRC11:42
*** mugsie has joined #openstack-dns11:42
rfreiremugsie, Graham11:55
rfreiredo you have a HA configurated designate.conf and/or pools.yaml to share?11:55
mugsienot right here ...11:55
* rfreire puzzled on how to setup the pool_target when having more than one11:56
rfreireshould I just use localhost:5354 for each node?11:56
rfreireThe scenario here is designate1, designate2, designate 3 and other machine running DNS11:57
mugsieoh, no - that should have 3 entries - one for each node11:57
rfreireso:11:57
rfreire    - type: bind911:57
rfreire      description: BIND9 Server 111:57
rfreire    - type: bind911:57
rfreire      description: BIND9 Server 211:57
rfreireRinse and repeat?11:57
rfreire[...] ?11:57
mugsieso bind is on each node as well?11:58
rfreireno, single bind server11:58
rfreireall designates pointing to a single bind server11:58
mugsieoh, then in the target, there is a masters section, and in there you add the 2 other designate nodes11:58
rfreirehrnm righto11:58
rfreiremetries11:58
rfreiremugsie, for what is worth11:59
rfreireby defining the yml file11:59
rfreireshould I still configure:12:00
rfreire--12:00
rfreirecrudini --set /etc/designate/designate.conf pool_target:$DEFAULT_TARGET_ID type bind912:00
rfreirecrudini --set /etc/designate/designate.conf pool_target:$DEFAULT_TARGET_ID masters $LOCAL_SERVER_IP:535412:00
rfreirecrudini --set /etc/designate/designate.conf pool_target:$DEFAULT_TARGET_ID options "host: $LOCAL_SERVER_IP, port: 53, rndc_host: $DNS_SERVER_IP, rndc_port: 953, rndc_config_file: /etc/rndc.conf, rndc_key_file: /etc/rndc.key"12:00
rfreire--12:00
rfreire?12:00
rfreirein designate.conf ?12:00
mugsiehttp://paste.openstack.org/show/719552/12:00
* rfreire mira12:00
rfreireawww yea12:00
mugsieno the config in the designate.conf file is never used12:00
rfreireah fine.12:00
rfreireSame for pool: poolid?12:01
mugsieyup12:02
*** thiagolib_ has joined #openstack-dns12:30
*** AlexeyAbashkin has quit IRC12:34
*** AlexeyAbashkin has joined #openstack-dns12:36
rfreireo SELinux... y u make my life miserable /o\12:49
rfreiremugsie, something is going on here12:54
rfreiremugsie, When creating a zone:12:54
rfreire--12:54
rfreire2018-04-19 12:54:09.780 25064 INFO designate.worker.tasks.zone [-] Attempting CREATE on zone internal.rf01.co.12:54
rfreire2018-04-19 12:54:09.976 25064 INFO designate.worker.tasks.zone [-] Failed to CREATE zone internal.rf01.co. on target fa90190d-514f-4c3a-9701-660c658173ae on attempt 1, Error: int() argument must be a string or a number, not 'NoneType'.12:54
rfreire2018-04-19 12:54:25.166 25064 INFO designate.backend.impl_bind9 [-] RNDC call failure: Unexpected error while running command.12:54
rfreire--12:55
rfreireThre's this fail.12:55
rfreireThen, in the subsequent errors I get:12:55
rfreire---12:55
rfreireCommand: sudo designate-rootwrap /etc/designate/rootwrap.conf rndc -s 172.25.250.133 -p 953 -k /etc/designate/rndc.key addzone internal.rf01.co  { type slave; masters { 192.168.100.140 port 5354; 192.168.100.211 port 5354; 192.168.100.130 port 5354;}; file "slave.internal.rf01.co.3538e863-13c7-46c9-9632-7da6bab1a6fa"; };12:55
rfreireExit code: 112:55
rfreireStdout: u''12:55
rfreireStderr: u"rndc: 'addzone' failed: already exists\n"12:55
rfreire---12:55
rfreireSo it seems that the zone _was_ successfully created in the 1st step12:56
rfreirebut something tickled worker12:56
rfreireAny idea?12:56
openstackgerritJens Harbott (frickler) proposed openstack/designate-dashboard master: Use Horizon UT framework  https://review.openstack.org/56252513:02
*** McClymontS has joined #openstack-dns13:03
*** McClymontS has quit IRC13:05
mugsierfreire: oh, weird13:13
rfreiremugsie, works with sink method13:15
rfreire¯\_(ツ)_/¯13:15
rfreire[root@desigha10-cont1 ~(keystone_admin)]# openstack zone list13:16
rfreire+--------------------------------------+-----------------+---------+------------+--------+--------+13:16
rfreire| id                                   | name            | type    |     serial | status | action |13:16
rfreire+--------------------------------------+-----------------+---------+------------+--------+--------+13:16
rfreire| bf218b8d-1c48-4821-b997-247d68383c5a | mugsie.rf01.co. | PRIMARY | 1524143647 | ACTIVE | NONE   |13:16
rfreire+--------------------------------------+-----------------+---------+------------+--------+--------+13:16
rfreire--13:16
rfreireI think that the zone name plays some role in it haha13:16
mugsieno way13:16
rfreireI have changed from worker -> sink.13:17
rfreiresink works.13:17
mugsiesink?13:17
mugsieor pool manager?13:17
rfreiretheres the worker13:17
rfreireah13:17
rfreireyea13:17
rfreireI think it is pool13:17
rfreireI disabled worker13:17
mugsieyeah, thats pool manager then13:18
rfreireoh wait, I think that I have mistaken pool with sink ; i have disabled the sink work instead13:18
* rfreire tries again without disabling sink service13:18
rfreireme--13:18
rfreirerfreire and his Dory fish memory13:18
rfreire....13:18
*** AlexeyAbashkin has quit IRC13:23
rfreiremugsie, unrelated to sink service. Something with Newton. Reverting to pool.13:24
mugsieok - unfortunately I can't fix newton :/13:25
*** AlexeyAbashkin has joined #openstack-dns13:25
rfreireyea; frozen song to that13:26
rfreirehave something that works and wont lose sweat on it13:26
rfreiremugsie, question; is producer necessary in Pool mode?13:29
rfreireor is it something specific to Worker?13:30
mugsieit is for the worker13:31
mugsiezone manager is needed with pool manager13:31
*** AlexeyAbashkin has quit IRC13:52
*** AlexeyAbashkin has joined #openstack-dns13:57
*** McClymontS has joined #openstack-dns14:15
rfreiremugsie, <ahem>14:17
rfreireDo you have any Redis url for coordination backend?14:17
* rfreire trying redis://172.25.250.72:6379?sentinel=desigha10-cont1.interna.rf01.co 14:17
mugsiebackend_url = redis://<sentinel host>:<sentinel port>?sentinel=<master name>14:21
mugsieshould do ity14:21
mugsiethe tooz docs are really lackikng14:22
*** AlexeyAbashkin has quit IRC14:23
*** AlexeyAbashkin has joined #openstack-dns14:25
rfreireok my error is something different here. I just found that there's something named redis-sentinel14:26
rfreirewhich I dont have configured heh14:26
* rfreire trying to set it up14:26
*** McClymontS has quit IRC14:28
bnemecmugsie: A diff of the downstream docs?  I'm not sure.14:35
bnemecThey aren't public and I'm not sure what the policy on upstreaming them is.  I can check though.14:35
bnemecI'm not sure how useful they will be to you anyway since they're still based on the pool-manager arch.14:35
mugsiebnemec: yeah - I just want to see if we need to update the upstream docs, and a diff is an easy way to check :)14:36
*** v12aml has quit IRC14:37
*** v12aml has joined #openstack-dns14:38
bnemecmugsie: Hopefully I will have some feedback on that soon.  My TripleO patches have mostly merged so I'm setting up a baremetal test environment locally to play around with it some more.14:39
*** AlexeyAbashkin has quit IRC14:40
*** AlexeyAbashkin has joined #openstack-dns14:42
rfreirebnemec, hi Ben o/14:45
bnemecrfreire: o/14:46
*** AlexeyAbashkin has quit IRC14:54
*** AlexeyAbashkin has joined #openstack-dns14:56
*** rfreire is now known as rfreire|afk14:58
*** AlexeyAbashkin has quit IRC15:04
*** AlexeyAbashkin has joined #openstack-dns15:08
*** v12aml has quit IRC15:12
*** v12aml has joined #openstack-dns15:13
*** rfreire|afk is now known as rfreire15:26
rfreiremugsie, alright man I have a few questions here:15:35
rfreire1. I am using the Pool architecture (because worker failed).15:35
rfreire2. As you are well aware, I am trying to make it work in HA.15:35
rfreire3. In https://docs.openstack.org/designate/latest/admin/ha.html#designate-producer I see that the coordination back-end is something bound to Producer.15:36
rfreire-- Questions --15:36
rfreireA. Does HA works in Pool architecture?15:37
rfreireB. Do I need to setup any Tooz coordination backend in Pool architecture?15:37
rfreire<EOQ>15:37
mugsieA) HA works for pool manager, but in a Active, Passive Failover method15:38
*** PsionTheory has joined #openstack-dns15:38
mugsieif you want HA pool manager, yes, you need a tooz co-ordination15:38
mugsiehowever, you can just run one process15:38
rfreireThat's fine, I'm okay for active/passive15:39
rfreire<mugsie> however, you can just run one process15:39
rfreireDoes that mean that I should leave the node sleeping, with services stopped?15:39
rfreire*the other nodes15:39
mugsiejust the pool manager process15:39
rfreirehrnm nice.15:39
rfreire<mugsie> if you want HA pool manager, yes, you need a tooz co-ordination15:39
mugsieeverything else still does active active15:40
rfreireIs coordination backend_url redis://172.25.250.72:26379?sentinel=mymaster enough?15:40
rfreireor do I need something else configured?15:40
mugsiethat should be it15:40
rfreireWell, think I'm done then :-)15:40
rfreireso in the standby nodes I will need the pool manager proccess stopped15:40
rfreireeverything else left running15:40
rfreireACK/nak?15:40
mugsieyup, that sounds right15:42
rfreireyay!15:42
rfreirephew learnt lots of new things today15:42
rfreireO_o15:42
rfreireTime to automate the tasks15:43
rfreireand ensure they are all good and valid15:43
rfreire;-)15:43
mugsie:)15:44
*** AlexeyAbashkin has quit IRC15:55
*** AlexeyAbashkin has joined #openstack-dns15:56
*** pcaruana has quit IRC16:03
*** AlexeyAbashkin has quit IRC16:11
*** AlexeyAbashkin has joined #openstack-dns16:14
*** itlinux has joined #openstack-dns16:19
*** Leo_m has joined #openstack-dns16:49
*** Leo_m has quit IRC16:54
*** AlexeyAbashkin has quit IRC16:57
*** Leo_m has joined #openstack-dns17:02
*** itlinux has quit IRC17:02
rfreiremugsie, hey Graham, still there?17:33
rfreiremugsie, just a small issue here; IDK for whatever reason it is trying to send notifies to 127.0.0.117:33
rfreire2018-04-19 17:32:32.580 30523 INFO designate.mdns.notify [req-62d5e82d-218c-48ff-bb65-612fa745ab68 2ddb80cadbe344a39ae609948a55bd5c e2eb2a38370b48cf9cae39dda9ec8db4 - - -] Sending 'NOTIFY' for 'internal.rf01.co.' to '127.0.0.1:53'.17:34
rfreire2018-04-19 17:33:02.585 30523 WARNING designate.mdns.notify [req-62d5e82d-218c-48ff-bb65-612fa745ab68 2ddb80cadbe344a39ae609948a55bd5c e2eb2a38370b48cf9cae39dda9ec8db4 - - -] Got Timeout while trying to send 'NOTIFY' for 'internal.rf01.co.' to '127.0.0.1:53'. Timeout='30' seconds. Retry='5'17:34
*** thiagolib_ has quit IRC17:49
rfreiremugsie, Dumped the database here and found no *127* in the log18:09
rfreire>_<18:09
rfreirewheres this coming from!18:10
rfreireHA!18:16
rfreire--18:16
rfreire    def __init__(self, target):18:16
rfreire        super(AgentPoolBackend, self).__init__(target)18:16
rfreire        self.host = self.options.get('host', '127.0.0.1')18:16
rfreire        self.port = int(self.options.get('port', DEFAULT_AGENT_PORT))18:16
rfreire        self.timeout = CONF['service:pool_manager'].poll_timeout18:16
rfreire        self.retry_interval = CONF['service:pool_manager'].poll_retry_interval18:16
rfreire        self.max_retries = CONF['service:pool_manager'].poll_max_retries18:16
rfreire        # FIXME: the agent retries creating zones without any interval18:16
rfreire--18:16
rfreire¬¬18:16
*** KeithMnemonic1 has quit IRC18:22
*** KeithMnemonic1 has joined #openstack-dns18:22
*** diman has joined #openstack-dns18:37
eanderssonI ran into the same one rfreire a long time ago :D18:41
rfreireeandersson, hi hi Gunnar! o/18:41
rfreireHow did you tackle that?18:41
rfreireI'm still banging head against wall here18:41
rfreire>_<18:41
eanderssonWhich version of Designate is this?18:41
rfreiretnis is Neutron18:42
rfreire3.something18:42
rfreireops18:42
rfreireNeuton18:42
rfreireops18:42
rfreireNEWTON18:42
rfreirephew18:42
eandersson:D18:42
rfreire:-P18:42
eanderssonDid you do the pool configuration?18:42
*** diman has quit IRC18:42
* rfreire makes a pastebin18:42
eanderssonIt's step ten here https://docs.openstack.org/designate/latest/install/install-rdo.html18:43
rfreireeandersson, http://paste.openstack.org/show/719576/18:43
rfreire--18:43
* rfreire looks18:43
rfreireeandersson, ah looksp pretty much in line for what I have done18:43
rfreire:|18:43
eanderssonMising one thing18:44
eandersson      options:18:44
eandersson        host: 127.0.0.118:44
eandersson        port: 5318:44
rfreireoh18:44
rfreireoh18:44
rfreireoh18:44
* rfreire reverts source codes to original state18:45
rfreire<grins>18:45
eanderssonhehe18:45
openstackgerritMerged openstack/designate-dashboard master: Fix lower-constraints.txt  https://review.openstack.org/56254518:48
*** AlexeyAbashkin has joined #openstack-dns18:49
rfreireeandersson, reverted stuff to original18:52
rfreireand reloaded config from yml18:52
rfreireeandersson, http://paste.openstack.org/show/719577/18:53
rfreirefingers crossed18:53
* rfreire allocates a float18:53
eandersson!!18:53
openstackeandersson: Error: "!" is not a valid command.18:53
openstackgerritMerged openstack/designate-dashboard master: Use Horizon UT framework  https://review.openstack.org/56252518:54
rfreire[root@arp10-cont1 designate(keystone_admin)]# openstack recordset list 9a545be0-da12-4487-bf23-7f46738b4de918:56
rfreire+--------------------------------------+------------------------+------+------------------------------------------------------------------------------+--------+--------+18:56
rfreire| id                                   | name                   | type | records                                                                      | status | action |18:56
rfreire+--------------------------------------+------------------------+------+------------------------------------------------------------------------------+--------+--------+18:56
rfreire| 9366b710-64ed-41a9-bfa4-75ab240b0525 | internal.rf01.co.      | NS   | adns.interna.rf01.co.                                                        | ACTIVE | NONE   |18:56
rfreire| ca8b3a8c-297a-4673-a242-c30b87e3defe | internal.rf01.co.      | SOA  | adns.interna.rf01.co. admin.internal.rf01.co. 1524164113 3514 600 86400 3600 | ACTIVE | NONE   |18:56
rfreire| f6196dc3-2b75-44c2-8b1e-165c2900eada | fafa.internal.rf01.co. | A    | 172.25.250.131                                                               | ACTIVE | NONE   |18:56
rfreire+--------------------------------------+------------------------+------+------------------------------------------------------------------------------+--------+--------+18:56
rfreireAWWWWWWWWW YEAAAAAA eandersson++18:56
rfreireYou going to OpenSTack summit too???18:56
eanderssonAwesome!18:56
rfreireAwesome YOU18:56
eanderssonI have to skip this one, as I can't renew my passport in time18:57
rfreireoooh boy18:57
rfreireWas eager to pay you a well deserved beer too18:57
eanderssonhaha - next one! :D18:57
rfreireLooking forward18:57
*** thiagolib_ has joined #openstack-dns18:57
rfreireA HUGE THANK YOU for you Erik18:57
eanderssonnp friend18:57
* rfreire bows18:57
*** AlexeyAbashkin has quit IRC18:59
*** rfreire is now known as rfreire|afk19:27
*** itlinux has joined #openstack-dns20:36
*** itlinux has quit IRC20:41
*** thiagolib_ has quit IRC21:37
*** diman has joined #openstack-dns22:38
*** diman has quit IRC22:42
« Wednesday, 2018-04-18 Index Friday, 2018-04-20 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!