Monday, 2020-03-23

*** yamamoto has joined #openstack-lbaas01:14
*** sapd1_x has joined #openstack-lbaas01:40
johnsomThe nightly image build is broken due to a DIB issue. The svc-map element needs pyyaml for it's extra.d 02- script and included it in the package-installs.yaml, however the package-install element doesn't fire until pre-install.d which is after extra.d01:53
johnsomAh, opps, it is 10- not 02-01:54
johnsomBut still, same ordering issue01:54
*** yamamoto has quit IRC02:38
*** yamamoto has joined #openstack-lbaas02:40
openstackgerritMerged openstack/octavia master: Fix padding logic for UDP health daemon  https://review.opendev.org/71424003:16
*** psachin has joined #openstack-lbaas03:24
*** hongbin has quit IRC04:20
*** ramishra has joined #openstack-lbaas04:28
*** yamamoto has quit IRC04:32
*** yamamoto has joined #openstack-lbaas04:44
*** vishalmanchanda has joined #openstack-lbaas04:58
*** yamamoto has quit IRC05:24
*** yamamoto has joined #openstack-lbaas05:43
*** yamamoto has quit IRC05:45
*** yamamoto has joined #openstack-lbaas05:49
*** yamamoto has quit IRC05:58
*** yamamoto has joined #openstack-lbaas06:27
*** yamamoto has quit IRC06:27
*** yamamoto has joined #openstack-lbaas06:27
*** yamamoto has quit IRC06:28
*** gcheresh has joined #openstack-lbaas07:01
*** yamamoto has joined #openstack-lbaas07:01
*** lxkong has quit IRC07:42
*** yamamoto has quit IRC07:48
*** yamamoto has joined #openstack-lbaas07:53
*** gcheresh has quit IRC07:56
*** gcheresh has joined #openstack-lbaas07:58
*** yamamoto has quit IRC08:03
*** maciejjozefczyk has joined #openstack-lbaas08:04
*** ccamposr has joined #openstack-lbaas08:09
*** tkajinam has quit IRC08:15
openstackgerritAnn Taraday proposed openstack/octavia master: Jobboard based controller  https://review.opendev.org/64740608:20
*** ataraday_ has joined #openstack-lbaas08:21
*** gthiemon1e has joined #openstack-lbaas08:38
*** gthiemonge has quit IRC08:39
*** rpittau|afk is now known as rpittau08:52
*** gthiemon1e is now known as gthiemonge08:57
*** numans has joined #openstack-lbaas09:22
*** ramishra has quit IRC09:33
*** ramishra has joined #openstack-lbaas09:40
openstackgerritAnn Taraday proposed openstack/octavia master: Jobboard based controller  https://review.opendev.org/64740609:45
*** sapd1_x has quit IRC11:23
*** rpittau is now known as rpittau|bbl11:47
*** gthiemonge has quit IRC12:08
*** gthiemon1e has joined #openstack-lbaas12:08
*** gcheresh has quit IRC12:15
*** gcheresh has joined #openstack-lbaas12:32
*** psachin has quit IRC12:42
*** sapd1_x has joined #openstack-lbaas12:56
*** rpittau|bbl is now known as rpittau13:08
*** sapd1_x has quit IRC13:35
*** TrevorV has joined #openstack-lbaas13:38
*** gthiemon1e has quit IRC13:49
*** gthiemonge has joined #openstack-lbaas14:00
*** gcheresh has quit IRC14:02
*** gcheresh has joined #openstack-lbaas14:08
*** hongbin has joined #openstack-lbaas14:19
*** Trevor_V has joined #openstack-lbaas14:23
*** TrevorV has quit IRC14:27
*** ccamposr__ has joined #openstack-lbaas14:31
*** ccamposr has quit IRC14:34
*** TrevorV has joined #openstack-lbaas14:34
*** Trevor_V has quit IRC14:37
*** gcheresh has quit IRC14:49
openstackgerritMichael Johnson proposed openstack/octavia master: Refactor the failover flows  https://review.opendev.org/70531715:31
*** gcheresh has joined #openstack-lbaas15:37
*** KeithMnemonic has joined #openstack-lbaas15:42
*** ccamposr__ has quit IRC15:59
*** ccamposr__ has joined #openstack-lbaas16:00
rm_workdaily plug for https://review.opendev.org/#/q/status:open+project:openstack/octavia+branch:master+topic:az-tweaks16:19
rm_workthough looks like johnsom is not going to approve it any time soon, and carlos already +2'd, not sure who else would actually look so maybe I should just give up, lol16:19
rm_workalso https://review.opendev.org/#/c/589180/ is good to go16:19
rm_workI guess I owe a triage pass on the priority list16:20
johnsomI didn't say I *wasn't* going to approve, but priorities...16:20
openstackgerritMichael Johnson proposed openstack/octavia master: Exclude acpid on Red Hat family DIB builds  https://review.opendev.org/71448716:27
openstackgerritAdam Harwell proposed openstack/octavia-tempest-plugin master: Drop queens support, and make rocky non-voting  https://review.opendev.org/71448816:27
rm_worklol16:40
rm_workokay :D16:40
*** sapd1_x has joined #openstack-lbaas16:58
*** sapd1_x has quit IRC17:03
*** rpittau is now known as rpittau|afk17:08
*** ccamposr has joined #openstack-lbaas17:18
*** ccamposr__ has quit IRC17:20
*** gregwork has joined #openstack-lbaas17:24
*** TrevorV has quit IRC17:45
dawzonIs there a way to locally generate unit test reports like you see on zuul?18:11
*** gcheresh has quit IRC18:16
*** hongbin has quit IRC18:22
johnsomdawzon Yes, but it's a few steps18:25
*** maciejjozefczyk has quit IRC18:28
johnsomstestr last --subunit > /tmp/subunit18:28
johnsomsubunit2html subunit_stream test_results.html18:28
*** maciejjozefczyk has joined #openstack-lbaas18:29
johnsomI think you need the venv active for the py36 test run for the commands to work18:29
johnsomYou may need to install os-testr too for the 2html command to work18:30
rm_workwtf I am so confused18:34
* johnsom Not dazed?18:34
rm_workamp agent appears to be sending UDP health packets... but tcpdump is showing me nothing on the amp... and the HM service certainly isn't getting them18:34
rm_workis udp / udpv6 the same? the traffic i am looking for would be ipv618:35
johnsomrmwork try "ip6 proto 17" for the filter18:38
johnsomrm_work ^^^18:38
rm_workkk18:39
rm_workhmmm18:40
rm_worknot working18:40
rm_workfff weird18:40
johnsomThen it is probably getting dropped in the kernel18:40
johnsomLike a missing or bad route18:40
rm_workhmmmm18:41
rm_workhow would i test that18:43
johnsomtraceroute618:43
johnsomI would start there18:44
rm_workhmm18:44
johnsomIt may not work depending on your network config18:44
johnsomI usually manually inspect the routing table(s)18:44
rm_workdon't know how to interpret this output T_T18:44
johnsom"*" mean nothing responded18:45
rm_workyeah but in this case it's literally one line and it looks like18:45
rm_work 1  5c3c1668.amphorae.service.stg.ows.oath.cloud (2001:4998:58:a880::b)  3105.222 ms !H  3105.182 ms !H  3105.168 ms !H18:45
johnsomAh, the !H is a code, let me see18:45
johnsom!H == host unreachable18:46
openstackjohnsom: Error: "H" is not a valid command.18:46
johnsom !H == host unreachable18:46
rm_workhmmm18:46
rm_workk18:46
rm_workmaybe some ACLs got messed up18:46
johnsomSo, maybe a problem or maybe just firewalled off18:46
rm_worki'll look into it, thanks18:46
johnsomMy guess is it is a missing / bad route18:47
johnsomOk, lunch time18:47
rm_workyeah18:58
rm_workwill look, i have a working and a non-working one side-by-side18:59
*** maciejjozefczyk has quit IRC19:12
*** ataraday_ has quit IRC19:13
*** laerling2 has quit IRC19:18
*** hongbin has joined #openstack-lbaas19:18
*** laerlingSAP has joined #openstack-lbaas19:20
*** laerlingSAP has quit IRC19:23
*** maciejjozefczyk has joined #openstack-lbaas19:23
*** laerlingSAP has joined #openstack-lbaas19:26
*** gcheresh has joined #openstack-lbaas19:34
dawzonI can't quite figure out this test: https://github.com/openstack/octavia/blob/2cf00d8c85692c571e23b221da7885dc0f179d14/octavia/tests/unit/common/jinja/haproxy/combined_listeners/test_jinja_cfg.py#L1117 It uses the protocol TERMINATED_HTTPS but it doesn't specify some of the other TLS options for its sample listener.  Should ciphers be specified on this config?19:35
rm_worksome tests assume defaults19:37
rm_workyou probably want to make a new test specifically for your stuff and then specify explicitly the things you're testing19:38
*** vishalmanchanda has quit IRC19:46
dawzonI'm probably going about this the wrong way... I was under the impression that explicitly defining a cipherstring in the haproxy config would the the default going forward so existing tests should be updated.  Should I be treating it as more of a special case instead? (i.e. leaving the existing tests as is and making new ones for rendering configurations with cipher strings)19:50
*** gcheresh has quit IRC19:57
*** maciejjozefczyk has quit IRC20:07
johnsomI thought we were going to have a "None" option for existing deployed LBs that would be the current configuration. Then, for newly created listeners it would pick up the new default from the configuration file.20:07
*** maciejjozefczyk has joined #openstack-lbaas20:07
*** maciejjozefczyk has quit IRC20:14
rm_workit's possible you'll have to edit existing tests, yes20:20
rm_worki was just saying, that if not specified, it's just assuming defaults -- so it's pretty normal for not every test to specify every option explicitly20:20
rm_workjohnsom: ugh, so you use: socket.inet_ntop(interface_af, socket.inet_pton(interface_af, attr[1]))20:21
rm_workjohnsom: can we do this in ... something else? like ... maybe ipaddress ?20:21
rm_workon OSX, inet_pton doesn't get along with address_family=1020:21
rm_work(ipv6 related?)20:21
rm_work`rtnl_api.get_addr(address=ip_address)` returns family=10 for ipv6 addresses20:24
rm_workso, `socket.inet_pton(interface_af, attr[1])` or rendered as `socket.inet_pton(10, 'IFA_ADDRESS')` explodes on OSX20:25
johnsomWhat???  It is a simple byte order method...20:25
rm_workerr sorry20:25
rm_worknot IFA_ADDRESS20:25
rm_workit's the IPv6 address20:25
johnsomMaybe best to get rid of macosx?20:25
rm_worklol20:25
rm_workwhat is this doing exactly, just verifying an ipv6 address?20:26
rm_workwe have code doing that elsewhere that is compatible20:26
haleybnetaddr.valid_ipv6(address) :)20:28
rm_workright :D20:28
rm_workbut like20:28
rm_worki don't understand why it does something so different here20:28
openstackgerritMerged openstack/octavia master: Network Delta calculations should respect AZs  https://review.opendev.org/70516520:28
openstackgerritMerged openstack/octavia master: Allow AZ to override valid_vip_networks config  https://review.opendev.org/69952120:28
rm_workjust above that it normalizes the address directly20:29
rm_workwith basically the same code20:29
rm_workbut anyway, ALL of this address normalization looks way complicated compared to what we do elsewhere with the ipaddress lib20:29
rm_workthe "normalized" format you seem to want is `ipaddress.ip_address(ip).compressed`20:31
rm_worki feel like 90% of this code could be clipped and replaced with that line20:32
rm_workbut i also feel like i have no idea what this code is all actually trying to accomplish20:33
rm_workso i'm probably missing something20:33
* haleyb doesn't even have the context, was just commenting blindly20:36
*** gcheresh has joined #openstack-lbaas20:38
rm_workyeah ok this would work fine, except now i have to disentangle the crazy mocking that was already being done with ip_address :D20:42
*** gcheresh has quit IRC20:46
johnsomrm_work Wait, I didn't add that inet_pton, that was there aready....20:47
rm_worki think you just added a test using it20:47
rm_workmaybe20:47
johnsomI did add a lot of missing test coverage for this stuff20:47
rm_workanyway, i got it20:48
johnsomI see it's in another file too, maybe I added that one. Checking20:48
johnsomOh, I moved that. Yeah, probably added missing test coverage. Let me see20:48
johnsomYeah, so compressed is not the same as what I am doing in network_utils.20:50
johnsominet_ntop and pton are bit order conversions.20:50
johnsomnetwork byte order to whatever the python host byte order is20:51
johnsomhttps://tools.ietf.org/html/draft-newman-network-byte-order-0120:51
rm_workhmm20:53
johnsomexploded might work however. let me see when they added that20:53
* johnsom wonders though, doesn't that module still use the builtin ntop/pton methods???? Hopes it doesn't ignore byte order20:54
rm_workerr20:54
rm_workhttps://review.opendev.org/#/c/705317/19/octavia/amphorae/backends/utils/network_utils.py@6220:54
rm_workso THAT is caring about byte order?20:55
rm_workalso: it seems to be using compressed20:55
rm_worklooking at what it's generating otherwise20:55
rm_worknormalized_addr = '2001:db8::55'20:56
rm_work(which is the one we're looking for, generated above via socket.inet_ntop / socket.inet_pton20:56
rm_work)20:56
johnsomIs it going to compressed? hmmm, it's been so long since that method was created. But it makes sense I may have used compressed to save a few bytes (however mysql is dumb and still allocates the same no matter what.20:57
rm_worklet me check the output of some commands on a linux system20:57
*** gcheresh has joined #openstack-lbaas20:58
rm_workyeah20:59
rm_work>>> socket.inet_ntop(socket.AF_INET6, socket.inet_pton(socket.AF_INET6, '2001:db8::55'))20:59
rm_work'2001:db8::55'20:59
rm_work>>> socket.inet_ntop(socket.AF_INET6, socket.inet_pton(socket.AF_INET6, '2001:0db8:0000:0000:0000:0000:0000:0055'))21:00
rm_work'2001:db8::55'21:00
rm_workand if we replaced both usages of socket.inet_* ... then it'd be guaranteed to match?21:01
rm_workso byte order be damned?21:01
johnsomAh, so there it is just matching and not storing, so yeah, using compresses should be ok.21:01
johnsomI have a vaque nagging that there was a problem with that library, but now we are native it probably is fixed.21:02
johnsomThe native ipaddress must be doing the byte order correct, or world + dog would be broken in python land21:03
*** gcheresh has quit IRC21:04
rm_workLOL no reason to mock that at all21:07
rm_workok then21:07
rm_workthat was weird :D removing mocks == win21:07
rm_workremoving low level socket code == win21:08
johnsomYeah, that patch is becoming even more cleanup!21:09
openstackgerritAdam Harwell proposed openstack/octavia master: Refactor the failover flows  https://review.opendev.org/70531721:42
dawzonjohnsom Here's what I was trying to say: Does it make sense to insert the ciphers parameter into the configs for all the existing unit tests?  Or is  it better to let them be21:49
rm_worki mean, it depends on what you envision the haproxy configs looking like21:59
rm_workdon't write the code around what you want to do with tests21:59
rm_work... i say as I just literally did that <_<21:59
rm_workbut anyway, if you plan to have default args generate haproxy configs with default cipher lists, then update the tests to reflect that22:00
dawzonAh, ok22:01
johnsomdawzon I think the right answer is to have tests that cover the different cases. So, some will likely stay the same, some may need to be added.22:01
dawzonAlright, I'm sure it'll be easier to discuss when I actually upload the patch :)22:08
rm_workusually yeah :D22:45
rm_workI always try to just upload something first since it's easier to point at code in a CR than to try to hand-wave at people until they understand (though I often try anyway)22:46
*** tkajinam has joined #openstack-lbaas22:57
*** lxkong has joined #openstack-lbaas23:03
sorrisonrm_work: Got openstacksdk support for AZs here https://review.opendev.org/#/c/714345/. (once I learn how to spell availability consistently). Almost got the horizon stuff done too23:08
*** gthiemonge has quit IRC23:39
*** gthiemonge has joined #openstack-lbaas23:47

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!