| *** limao has quit IRC | 00:04 | |
| *** mattmceuen has quit IRC | 00:13 | |
| *** hongbin has quit IRC | 00:22 | |
| *** tonanhngo has quit IRC | 00:44 | |
| *** tonanhngo has joined #openstack-kuryr | 00:44 | |
| *** neiljerram has quit IRC | 00:45 | |
| *** limao has joined #openstack-kuryr | 00:46 | |
| *** dougbtv__ has joined #openstack-kuryr | 01:18 | |
| *** dougbtv has quit IRC | 01:19 | |
| *** yedongcan has joined #openstack-kuryr | 01:25 | |
| *** yedongcan1 has joined #openstack-kuryr | 01:37 | |
| *** yedongcan has quit IRC | 01:41 | |
| *** yedongcan has joined #openstack-kuryr | 02:13 | |
| *** yedongcan1 has quit IRC | 02:15 | |
| *** hongbin has joined #openstack-kuryr | 03:02 | |
| *** hongbin has quit IRC | 03:20 | |
| *** hongbin has joined #openstack-kuryr | 03:21 | |
| *** portdirect_away is now known as portdirect | 03:55 | |
| openstackgerrit | Jaivish Kothari(janonymous) proposed openstack/kuryr-kubernetes: Bump tox min version to 2.3.1 https://review.openstack.org/424972 | 04:21 |
|---|---|---|
| *** hongbin has quit IRC | 04:23 | |
| *** hongbin has joined #openstack-kuryr | 04:41 | |
| *** hongbin has quit IRC | 04:53 | |
| *** vikasc has quit IRC | 05:20 | |
| *** janki has joined #openstack-kuryr | 05:32 | |
| *** tonanhngo has quit IRC | 05:36 | |
| *** tonanhngo has joined #openstack-kuryr | 05:37 | |
| *** tonanhngo has quit IRC | 05:41 | |
| *** jchhatbar has joined #openstack-kuryr | 05:50 | |
| *** janki has quit IRC | 05:51 | |
| *** janki has joined #openstack-kuryr | 06:02 | |
| *** jchhatbar has quit IRC | 06:04 | |
| *** irenab_ has joined #openstack-kuryr | 06:11 | |
| *** irenab_ has quit IRC | 06:12 | |
| *** vikasc has joined #openstack-kuryr | 06:23 | |
| *** jchhatbar has joined #openstack-kuryr | 06:26 | |
| *** janki has quit IRC | 06:29 | |
| irenab | limao: ltomasbo : hi guys | 06:42 |
| limao | Hi irenab | 06:42 |
| irenab | limao: I just saw your reply regarding nested case limitations | 06:43 |
| irenab | limao: Maybe we need to record this on kuryr side | 06:43 |
| irenab | looks like the unexpected limitations can be an issue for kuryr consumers | 06:44 |
| limao | irenab: Do you mean this patch? https://review.openstack.org/#/c/422641/ | 06:44 |
| irenab | limao: related to this discussion ]http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2016-11-22.log.html | 06:45 |
| irenab | (15:29) | 06:46 |
| limao | irenab: oh, get it | 06:46 |
| irenab | its more about deployment limitations | 06:46 |
| ltomasbo | I tried that myself | 06:47 |
| ltomasbo | and if memory works, there was a problem with ovs-hybrid | 06:47 |
| ltomasbo | but it was working with ovs-firewall | 06:47 |
| limao | irenab: how about add this limitation in kuryr-libnetwork/README.rst ? | 06:47 |
| *** jchhatbar is now known as janki | 06:47 | |
| irenab | ltomasbo: do you have in mind the options that work? It will be great to add this in README, as ‘known limitations’ | 06:48 |
| irenab | to help kuryr users | 06:48 |
| ltomasbo | for the kubernetes devstack conf | 06:48 |
| ltomasbo | it is already there that you require ovs-firewall | 06:48 |
| ltomasbo | then there should be no problem | 06:49 |
| limao | How to try out nested-containers locally | 06:49 |
| limao | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | 06:49 |
| limao | 1. Installing OpenStack running devstack with the desired local.conf file but | 06:49 |
| limao | including the next to make use of OVS-firewall and enabling Trunk Ports:: | 06:49 |
| irenab | ltomasbo: what aboutdifferent vlans for ports on same network? | 06:49 |
| ltomasbo | and if the ovs-hybrid was used, i think the only problem was if the ports have the same mac, not the same subnet | 06:49 |
| ltomasbo | but I can re-try this just in case | 06:49 |
| ltomasbo | anyway, that is a trunk-port limitation, not a kuryr one | 06:50 |
| ltomasbo | yes, it does not matter if they are in the same network | 06:50 |
| irenab | ltomasbo: limao : the point is if there are more requirments/limitations, we better document it based on theway kuryr uses neutron trunk service | 06:50 |
| ltomasbo | yep, it makes sense | 06:51 |
| irenab | ltomasbo: since kuryr won’t work properly, and kuryr cuonsumers won’t be happy :-) | 06:51 |
| limao | irenab: ltomasbo: agree to add that in Limitations section | 06:51 |
| irenab | thanks a lot | 06:52 |
| irenab | limao: I can push an update, just want t make sure what are the revealed limitations | 07:11 |
| limao | irenab： Thanks， Kuryr nested-containers only work with neutron OVS-firewall(iptables_hybrid firewall will not work). This limitation is in Neutron Trunk Port by design. | 07:12 |
| irenab | limao: what about noop firewal driver? | 07:13 |
| limao | irenab: also works, but we lost the sg in that case | 07:13 |
| irenab | limao: correct, but if for some reason it s the deployment choice… Maybe port security is disabled | 07:15 |
| limao | irenab: Oh, yeah, that's make sense | 07:15 |
| irenab | ok, will sum-on and push README update asap | 07:16 |
| limao | irenab: Thanks for this | 07:16 |
| *** yamamoto has quit IRC | 07:17 | |
| limao | irenab: maybe add the link which jlibosva pasted | 07:17 |
| limao | jlibosvalimao: "Obviously this solution is not compliant with iptables firewall." from https://github.com/openstack/neutron/blob/master/doc/source/devref/openvswitch_agent.rst#tackling-the-network-trunking-use-case | 07:17 |
| irenab | limao: yes, I will point to the neutron not to duplicate | 07:17 |
| *** pcaruana has joined #openstack-kuryr | 07:28 | |
| ltomasbo | do you know that it will work with noop firewall driver? | 07:43 |
| limao | ltomasbo: I remembered that I tried noop firewall driver at that time. Container can ping each other in that case(vm nested), but I did not do more test. | 07:46 |
| ltomasbo | ok, that should work | 07:48 |
| limao | ltomasbo: The problem is mac flapping on linux bridge(in my memory), the fw driver without linux bridge should can work(ovs and noop, do not have lb) | 07:48 |
| ltomasbo | my concern is about if that would work a) if they are on the same network; b) if the ports have the same mac | 07:48 |
| ltomasbo | ahh, ok, didn't realized noop has no lb | 07:49 |
| limao | ltomasbo: Yeah, I think if it do not have lb, should be ok | 07:50 |
| ltomasbo | then is should be ok, yes | 07:50 |
| limao | ltomasbo: ;-) | 07:50 |
| ltomasbo | another limitation to include is the QoS | 07:50 |
| ltomasbo | it does not work together with trunk ports | 07:50 |
| ltomasbo | https://bugs.launchpad.net/neutron/+bug/1639186 | 07:51 |
| openstack | Launchpad bug 1639186 in neutron "qos max bandwidth rules not working for neutron trunk ports" [Low,Confirmed] - Assigned to Luis Tomas Bolivar (ltomasbo) | 07:51 |
| ltomasbo | irenab, ^^ | 07:51 |
| irenab | ltomasbo: on sub ports? | 07:52 |
| ltomasbo | yep | 07:52 |
| ltomasbo | the problem is how trunk-ovs and br-int are linked | 07:53 |
| ltomasbo | you cannot apply QoS (as of today) on patch-ports | 07:53 |
| ltomasbo | IIRC you cannot even apply it on parent port | 07:53 |
| ltomasbo | for the same reason | 07:53 |
| openstackgerrit | Berezovsky Irena proposed openstack/kuryr-libnetwork: Add nested-containers limitations https://review.openstack.org/425040 | 08:00 |
| *** yamamoto has joined #openstack-kuryr | 08:00 | |
| irenab | ltomasbo: tried not to enter into neutron details too much, more focus on the impact on kuryr users | 08:00 |
| ltomasbo | ok, but to me that is a limitation to kuryr users | 08:01 |
| ltomasbo | as they may use QoS for side-by-side deployments | 08:01 |
| ltomasbo | but not for nested | 08:01 |
| *** yamamoto has quit IRC | 08:02 | |
| *** yamamoto has joined #openstack-kuryr | 08:03 | |
| *** saneax-_-|AFK is now known as saneax | 08:06 | |
| *** pksingh has joined #openstack-kuryr | 08:15 | |
| *** gsagie has joined #openstack-kuryr | 08:29 | |
| *** vikasc has quit IRC | 08:33 | |
| *** vikasc has joined #openstack-kuryr | 08:51 | |
| *** garyloug has joined #openstack-kuryr | 08:52 | |
| *** pmannidi has quit IRC | 09:14 | |
| *** portdirect is now known as portdirect_brb | 09:22 | |
| *** limao has quit IRC | 09:31 | |
| ltomasbo | ping yedongcan | 09:32 |
| *** portdirect_brb is now known as portdirect | 09:32 | |
| yedongcan | ltomasbo: hi | 09:32 |
| ltomasbo | just saw your reply to https://review.openstack.org/#/c/420610/ | 09:33 |
| ltomasbo | just curious, before, if show_extension failed, we have an error and raised the exception | 09:33 |
| ltomasbo | and the proposed change removes that call, but if adding tag fails, we just log a warning and continue | 09:34 |
| yedongcan | ltomasho: yes. | 09:35 |
| ltomasbo | what happen if tag neutron extension is not supported | 09:35 |
| ltomasbo | before, we got an error and exception, and now it will continue with just a warning | 09:35 |
| yedongcan | now, we will give an warning for users and at the end we will not add the tag. | 09:36 |
| *** devvesa has joined #openstack-kuryr | 09:36 | |
| yedongcan | before we give the warning message, we still get the exception from neutron client | 09:37 |
| ltomasbo | it may happen then that the subnet gets deleted when the container is removed? | 09:37 |
| ltomasbo | as there is no tag indicating it was already existing? | 09:37 |
| *** yamamoto has quit IRC | 09:38 | |
| yedongcan | ltomasbo: I think not, we are not using tag indicate the subnet is existing Neutron subnet or created by Kuryr. | 09:45 |
| ltomasbo | yedongcan, true | 09:48 |
| ltomasbo | that is for networks | 09:48 |
| yedongcan | ltomasbo: thanks, :) | 09:50 |
| openstackgerrit | Dongcan Ye proposed openstack/kuryr-libnetwork: Removes subnetpool_id tag for Neutron existing subnet https://review.openstack.org/419735 | 09:51 |
| *** dougbtv__ has quit IRC | 10:15 | |
| *** neiljerram has joined #openstack-kuryr | 10:16 | |
| *** openstackgerrit has quit IRC | 10:17 | |
| irenab | janonymous: hi | 10:21 |
| *** openstackgerrit has joined #openstack-kuryr | 10:24 | |
| openstackgerrit | Dongcan Ye proposed openstack/kuryr-libnetwork: Remove subnetpool_id tag for Neutron existing subnet https://review.openstack.org/419735 | 10:24 |
| *** yamamoto has joined #openstack-kuryr | 10:25 | |
| *** yamamoto has quit IRC | 10:26 | |
| *** portdirect is now known as portdirect_away | 10:27 | |
| *** dougbtv__ has joined #openstack-kuryr | 10:31 | |
| janonymous | irenab: hey | 10:34 |
| irenab | janonymous: I wanted to verify the tls patch to seehow we can progress with it. I have a question regarding deployment | 10:36 |
| janonymous | irenab:sure | 10:37 |
| irenab | I am deploying devstack all-in-one based on https://review.openstack.org/#/c/410609/6 | 10:37 |
| irenab | any specific option to enable in local.conf? | 10:37 |
| janonymous | no | 10:38 |
| irenab | what are the addiotnal steps after devstack is done? | 10:38 |
| janonymous | Please check 2 things | 10:38 |
| janonymous | 1) kuryr.conf have 3 things enabled: enable_ssl , ssl_* options | 10:39 |
| janonymous | 2) kuryr.json file should have valid paths in /usr/... path where kuryr.json is placed | 10:39 |
| irenab | janonymous: how did you generate the ssl file? | 10:41 |
| janonymous | After that we need to create 3 files for local test of ssl ( cert, ca, key) ... which i created but not 100% sure i did it correctly | 10:41 |
| janonymous | i followed limao's commads as below: | 10:41 |
| janonymous | openssl genrsa -out ca.pem 2048; | 10:41 |
| janonymous | openssl req -new -x509 -nodes -days 1000 -key ca.pem -out ca-cert.pem; | 10:41 |
| janonymous | openssl req -newkey rsa:2048 -days 1000 -nodes -keyout key.pem -out server-req.pem; | 10:41 |
| janonymous | openssl x509 -req -in server-req.pem -days 1000 -CA ca-cert.pem -CAkey ca.pem -set_serial 01 -out cert.pem; | 10:41 |
| irenab | janonymous: and once this done, restart kuryr-libnetwork? | 10:42 |
| janonymous | but please check on net, because i tried several other links also | 10:42 |
| janonymous | irenab: yeah, both docker and libnetwork would be good | 10:42 |
| irenab | janonymous: great, thanks for sharing the details | 10:42 |
| janonymous | irenab: when all things are done, to verify everythin run command mentioned by limao in comments | 10:43 |
| janonymous | curl --cacert /var/lib/kuryr/certs/cert.pem -XPOST https://127.0.0.1:23750/Plugin.Activate | 10:43 |
| janonymous | this would ensure that on kuryr side everything is working find | 10:43 |
| janonymous | *fine | 10:43 |
| janonymous | and certificates are generated properly | 10:43 |
| irenab | ok | 10:44 |
| irenab | will let you know once done with the bring up | 10:44 |
| janonymous | irenab: let me know if you face any issue till these steps :) | 10:44 |
| irenab | to verify the issue you ran into | 10:44 |
| janonymous | sure, as per investigation point of view there was a prblem with docker to verify self signed ceritificates. but anyway please take your time | 10:45 |
| janonymous | i will be around | 10:45 |
| irenab | janonymous: any chance you have kuryr.con and kuryr.jsonsmaples to post? | 10:46 |
| janonymous | yes, | 10:47 |
| janonymous | i will paste it in a min | 10:47 |
| *** vikasc has quit IRC | 10:48 | |
| *** yedongcan has left #openstack-kuryr | 10:50 | |
| janonymous | irenab:http://paste.openstack.org/show/596416/ | 10:56 |
| janonymous | i pasted all the configs and certs ^^ | 10:56 |
| *** yamamoto has joined #openstack-kuryr | 11:18 | |
| *** yamamoto has quit IRC | 11:18 | |
| *** vikasc has joined #openstack-kuryr | 11:20 | |
| *** pksingh has quit IRC | 11:28 | |
| *** devvesa has quit IRC | 11:32 | |
| *** yamamoto has joined #openstack-kuryr | 11:33 | |
| *** yamamoto has quit IRC | 11:34 | |
| irenab | janonymous: hi | 11:34 |
| janonymous | irenab:hi | 11:34 |
| irenab | janonymous: I followed this link http://tech.paulcz.net/2016/01/secure-docker-with-tls/ to generate TLC certificates | 11:34 |
| irenab | seems to be working for me, but I added settings to the docker service creation | 11:35 |
| irenab | sudo /usr/bin/docker daemon -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375 --tlsverify --tlscacert=/var/lib/kuryr/certs/ca.pem --tlscert=/var/lib/kuryr/certs/cert.pem --tlskey=/var/lib/kuryr/certs/key.pem --cluster-store etcd://localhost:5001 | 11:36 |
| janonymous | what did you added in CN field | 11:36 |
| irenab | what is the CN field? | 11:37 |
| irenab | janonymous: except for the curl --cacert /var/lib/kuryr/certs/cert.pem -XPOST https://127.0.0.1:23750/Plugin.Activate, didn’t try anything yet | 11:38 |
| janonymous | it takes the HOST in certificate gen | 11:38 |
| janonymous | ohh cool | 11:38 |
| janonymous | irenab: doest that work? | 11:38 |
| irenab | curl --cacert /var/lib/kuryr/certs/cert.pem -XPOST https://127.0.0.1:23750/Plugin.Activate | 11:39 |
| irenab | { | 11:39 |
| irenab | "Implements": [ | 11:39 |
| irenab | "NetworkDriver", | 11:39 |
| irenab | "IpamDriver" | 11:39 |
| irenab | ] | 11:39 |
| irenab | } | 11:39 |
| janonymous | cool | 11:40 |
| janonymous | now try docker network create | 11:40 |
| janonymous | docker network create --driver kuryr --ipam-driver kuryr \ | 11:41 |
| janonymous | --subnet 10.10.0.0/16 --gateway=10.10.0.1 test_net | 11:41 |
| janonymous | 785f8c1b5ae480c4ebcb54c1c48ab875754e4680d915b270279e4f6a1aa52283 | 11:41 |
| irenab | janonymous: but following the link, thereis also openssl.cnf file | 11:41 |
| irenab | kuryr crashed | 11:45 |
| janonymous | :D | 11:45 |
| irenab | http://paste.openstack.org/show/596425/ | 11:45 |
| *** devvesa has joined #openstack-kuryr | 11:48 | |
| janonymous | irenab: strange | 11:53 |
| janonymous | irenab: what about docker log | 11:53 |
| janonymous | Also can you run sudo /usr/bin/docker daemon -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375 --tlsverify --tlscacert=/var/lib/kuryr/certs/ca.pem --tlscert=/var/lib/kuryr/certs/cert.pem --tlskey=/var/lib/kuryr/certs/key.pem --cluster-store etcd://localhost:5001 with -D flag | 11:54 |
| irenab | DEBU[0010] Calling POST /v1.25/networks/create | 11:54 |
| irenab | DEBU[0010] form data: {"Attachable":false,"CheckDuplicate":true,"Driver":"kuryr","EnableIPv6":false,"IPAM":{"Config":[{"Gateway":"10.10.0.1","Subnet":"10.10.0.0/16"}],"Driver":"kuryr","Options":{}},"Internal":false,"Labels":{},"Name":"test_net","Options":{}} | 11:54 |
| irenab | WARN[0010] Unable to connect to plugin: 127.0.0.1:23750/Plugin.Activate: Post http://127.0.0.1:23750/Plugin.Activate: read tcp 127.0.0.1:39702->127.0.0.1:23750: read: connection reset by peer, retrying in 1s | 11:54 |
| irenab | WARN[0011] Unable to connect to plugin: 127.0.0.1:23750/Plugin.Activate: Post http://127.0.0.1:23750/Plugin.Activate: read tcp 127.0.0.1:39704->127.0.0.1:23750: read: connection reset by peer, retrying in 2s | 11:54 |
| irenab | WARN[0013] Unable to connect to plugin: 127.0.0.1:23750/Plugin.Activate: Post http://127.0.0.1:23750/Plugin.Activate: read tcp 127.0.0.1:39716->127.0.0.1:23750: read: connection reset by peer, retrying in 4s | 11:54 |
| irenab | WARN[0017] Unable to connect to plugin: 127.0.0.1:23750/Plugin.Activate: Post http://127.0.0.1:23750/Plugin.Activate: read tcp 127.0.0.1:39728->127.0.0.1:23750: read: connection reset by peer, retrying in 8s | 11:54 |
| irenab | ERRO[0025] Handler for POST /v1.25/networks/create returned error: legacy plugin: Post http://127.0.0.1:23750/Plugin.Activate: read tcp 127.0.0.1:39760->127.0.0.1:23750: read: connection reset by peer | 11:54 |
| irenab | janlet me pastebin this | 11:54 |
| janonymous | It seems your ssl is working :O | 11:54 |
| irenab | http://paste.openstack.org/show/596427/ | 11:54 |
| janonymous | your problem is of keep_alive ..i guess | 11:55 |
| janonymous | BUT | 11:55 |
| janonymous | http ? | 11:56 |
| janonymous | WARN[0010] Unable to connect to plugin: 127.0.0.1:23750/Plugin.Activate: Post http://127.0.0.1:23750/Plugin.Activate: read tcp 127.0.0.1:39702->127.0.0.1:23750: read: connection reset by peer, retrying in 1s has http:// not https:// | 11:56 |
| janonymous | is there 2files kuryr.json, kuryr.spec ? | 11:57 |
| janonymous | irenab: ^^ | 11:59 |
| irenab | checking | 12:00 |
| *** portdirect_away is now known as portdirect | 12:02 | |
| irenab | yes | 12:02 |
| irenab | under /opt/stack/kuryr-libnetwork | 12:02 |
| janonymous | remove spec one, and check json one contents once that link in https in it | 12:03 |
| *** tonanhngo has joined #openstack-kuryr | 12:03 | |
| irenab | there are 2 kuryr.spec | 12:03 |
| janonymous | No, in /usr/lib/ path.. | 12:04 |
| janonymous | on /usr/lib/docker/plugins/kuryr/ path | 12:05 |
| *** tonanhngo has quit IRC | 12:08 | |
| irenab | Error response from daemon: legacy plugin: plugin not found | 12:11 |
| irenab | janonymous: looks like docker cannot locate the kuryr | 12:13 |
| apuimedo | irenab: are you testing the ssl patch? | 12:14 |
| janonymous | Docker checks kuryr.json in /usr/lib/docker/plugins/kuryr/ , can you recheck and restart kuryr, docker services | 12:14 |
| irenab | apuimedo: yes | 12:14 |
| apuimedo | there are other places where it looks for json/spec files | 12:14 |
| janonymous | apuimedo: o/ | 12:14 |
| apuimedo | and they have a precedence order | 12:14 |
| apuimedo | so probably you have an old .spec file elsewhere | 12:14 |
| apuimedo | that takes precedence | 12:14 |
| apuimedo | I think one possibility is on /etc | 12:15 |
| janonymous | /run/docker/plugins , | 12:15 |
| janonymous | /etc/docker/plugins | 12:15 |
| irenab | shall I remove all of them? | 12:16 |
| janonymous | irenab: pls check for .spec if it exists on these locations, remove it | 12:17 |
| irenab | janonymous: renamed all of them, hope this counts :-) | 12:18 |
| janonymous | irenab:In /usr/lib/docker/plugins/kuryr/ there should exist kuryr.json file with content having https://127.0.0.1:23750 in it | 12:18 |
| janonymous | great! | 12:18 |
| janonymous | restart and try again pls | 12:19 |
| irenab | janonymous: I have it here /opt/stack/kuryr-libnetwork/contrib/tls/kuryr.json | 12:19 |
| irenab | will copy | 12:19 |
| irenab | yes! | 12:21 |
| irenab | working | 12:21 |
| janonymous | Also you could set InsecureSkipVerify: true | 12:21 |
| irenab | janonymous: I think probably the instructions in REAME should be impoved | 12:22 |
| apuimedo | ;-) | 12:22 |
| irenab | janonymous: what is the issue you were concerned about? | 12:22 |
| janonymous | now try THE docker network create command mentioned above | 12:22 |
| janonymous | :D | 12:22 |
| irenab | janonymous: just did | 12:22 |
| janonymous | irenab: docker logs? does that work? | 12:23 |
| irenab | ait amin, need to create container | 12:23 |
| janonymous | is your network create working? | 12:24 |
| irenab | yes | 12:26 |
| irenab | also create container | 12:27 |
| irenab | docker seems to be happy | 12:28 |
| janonymous | :/ | 12:28 |
| janonymous | i am not happy! What i did wrong | 12:28 |
| janonymous | can you please paste docker logs and kuryr-logs | 12:28 |
| irenab | janonymous: http://paste.openstack.org/show/596434/ | 12:29 |
| apuimedo | ivc_: please see irenab's comment to https://review.openstack.org/#/c/423903/ (in case my +2 shadowed it) | 12:29 |
| apuimedo | janonymous: you are not happy because it works? | 12:30 |
| apuimedo | :-D | 12:30 |
| irenab | janonymous: try to follow the cert creation process in the link I posted, maybe there is something there you missed | 12:30 |
| irenab | http://tech.paulcz.net/2016/01/secure-docker-with-tls/ | 12:30 |
| irenab | posting kuryr logs in a sec | 12:30 |
| janonymous | apuimedo: :P ,i tried 4 types of creating ssl certs in my entire day! | 12:31 |
| janonymous | irenab:thanks! | 12:31 |
| openstackgerrit | Merged openstack/kuryr-libnetwork: trivial: nosetest .xml removal as testr is used now https://review.openstack.org/355812 | 12:32 |
| openstackgerrit | Merged openstack/kuryr-libnetwork: Typo fix: happend => happened https://review.openstack.org/423995 | 12:32 |
| apuimedo | janonymous: :-) | 12:33 |
| irenab | http://paste.openstack.org/show/596436/ | 12:34 |
| apuimedo | gerrit is slow like a snail parade | 12:34 |
| irenab | apuimedo: maybe it got flu | 12:35 |
| apuimedo | don't tell me about it | 12:35 |
| irenab | janonymous: what exactly didn’t work or you? | 12:35 |
| apuimedo | I'm fucking sick of being sick!!! When I thought we were all cured, different kind of coughing and belly ache kicks in... | 12:36 |
| apuimedo | I am so done with this | 12:36 |
| *** neiljerram has quit IRC | 12:36 | |
| irenab | apuimedo: you need to rest | 12:37 |
| janonymous | irenab:ssl certs genertions for sure! , i will try with your links tomorrow | 12:37 |
| apuimedo | can't, gotta take care of the children | 12:37 |
| apuimedo | :-) | 12:37 |
| irenab | janonymous: hope I could help. Add comment about instruvtions, but otherwise it seems ok | 12:38 |
| irenab | apuimedo: yes, on this front there is no days off to take … | 12:38 |
| janonymous | apuimed: take care , get well sson | 12:38 |
| *** neiljerram has joined #openstack-kuryr | 12:39 | |
| janonymous | irenab:thanks for looking in that | 12:39 |
| janonymous | irenab: one last question, are you sure it kuryr server is on https :P | 12:39 |
| irenab | janonymous: it seems so in the logs | 12:40 |
| apuimedo | thanks | 12:40 |
| apuimedo | irenab: make a curl | 12:40 |
| apuimedo | it will surely enough tell you that your cert is crap | 12:41 |
| apuimedo | :-) | 12:41 |
| irenab | http://paste.openstack.org/show/596438/ | 12:41 |
| irenab | apuimedo: already did | 12:41 |
| janonymous | :D | 12:41 |
| apuimedo | cool | 12:41 |
| irenab | curl --cacert /var/lib/kuryr/certs/cert.pem -XPOST https://127.0.0.1:23750/Plugin.Activate | 12:41 |
| irenab | { | 12:42 |
| irenab | "Implements": [ | 12:42 |
| janonymous | irenab: thanks lot! really appreciate it | 12:42 |
| irenab | "NetworkDriver", | 12:42 |
| *** vikasc has quit IRC | 12:42 | |
| irenab | "IpamDriver" | 12:42 |
| irenab | ] | 12:42 |
| irenab | } | 12:42 |
| irenab | janonymous: you welcome | 12:42 |
| janonymous | irenab: whole patch was correct, but i spent whole day to test it and that too improperly :/ | 12:43 |
| janonymous | limao, irenab, apuimedo: thanks! :) | 12:43 |
| irenab | apuimedo: can you please spend few mins explaining about kuryr.spec and kuryr.json, when it is used and why there can be number of them? | 12:43 |
| *** neiljerram has quit IRC | 12:43 | |
| openstackgerrit | Merged openstack/fuxi: Updated from global requirements https://review.openstack.org/419929 | 12:43 |
| janonymous | irenab: https://docs.docker.com/engine/extend/plugin_api/ | 12:43 |
| irenab | why there can be number of them? | 12:45 |
| *** neiljerram has joined #openstack-kuryr | 12:45 | |
| janonymous | but i personally think that there should be one convention in kuryr | 12:45 |
| irenab | for the same plugin | 12:45 |
| janonymous | json one is top priority | 12:45 |
| irenab | so they are expected to be under /run/docker/plugins? | 12:47 |
| janonymous | under any 3 locations | 12:47 |
| irenab | ok, got it | 12:47 |
| apuimedo | irenab: this is just to find different network providers | 12:47 |
| apuimedo | you can have several network remote drivers at the same time | 12:47 |
| apuimedo | irenab: it's a bit like with systemd | 12:48 |
| irenab | apuimedo: this make sense, but I was not sure why there are several kuryr.json and kury.spec files | 12:48 |
| apuimedo | /usr/lib/systemd/system has drivers installed by the system | 12:48 |
| apuimedo | but there's also /etc location for overriding locally | 12:48 |
| irenab | but as long as the location matters, it should not be a problem | 12:48 |
| apuimedo | irenab: that's leftovers from devstack runs I suppose | 12:49 |
| irenab | apuimedo: so for the devstack, what would be the main one? | 12:49 |
| * apuimedo checks | 12:51 | |
| apuimedo | /etc | 12:51 |
| irenab | for somereason I didn’t have them at all under the path mention in the docker page | 12:52 |
| apuimedo | wrong | 12:52 |
| apuimedo | /usr/lib/docker/plugins/kuryr | 12:52 |
| irenab | distro agnostic? | 12:52 |
| apuimedo | yes | 12:53 |
| irenab | thanks | 12:53 |
| * janonymous calling it a day now! :) | 12:54 | |
| irenab | janonymous: enjoy the rest of the day/evening | 12:55 |
| ivc_ | apuimedo https://review.openstack.org/#/c/423908/1 | 12:55 |
| ivc_ | irenab, apuimedo, regarding 'missing bp' in https://review.openstack.org/#/c/423903 | 12:56 |
| ivc_ | i dont think we need to reference bp there | 12:56 |
| ivc_ | first of all it does not implement a bp and has nothing to do with it. it's just a minor improvement | 12:56 |
| irenab | ivc_: it is very convenient to see all realted patches from the launchpad bp page | 12:56 |
| ivc_ | irenab the problem with that bp is it covers everything in kuryr-k8s | 12:57 |
| ivc_ | irenab do you want _every_ patch in kuryr-k8s to have the reference to that bp? | 12:57 |
| irenab | ivc_: checking the launshpad now, I do not see the bp at all ... | 12:59 |
| irenab | launchpad | 12:59 |
| ivc_ | irenab what bp are you looking for? | 13:00 |
| ivc_ | https://blueprints.launchpad.net/openstack/?searchtext=kuryr-k8s-integration ? | 13:00 |
| *** garyloug has quit IRC | 13:01 | |
| irenab | we actually referenced bp at kuryr and not kury-kuberntes | 13:01 |
| *** vikasc has joined #openstack-kuryr | 13:01 | |
| irenab | what a mess .. | 13:02 |
| apuimedo | :P | 13:02 |
| apuimedo | it can be changed | 13:02 |
| irenab | ivc_: my point is that it is quite convenient for tracing the relevant patches if there is some linking to the launchpad bp or bug that this patch refers to | 13:03 |
| ivc_ | i don't thing there's anything on lpad that refers to this patch | 13:03 |
| irenab | unless patches are trivial, it should be linked to something. What is trivial can be arguable ;-) | 13:03 |
| openstackgerrit | Ilya Chukhnakov proposed openstack/kuryr-kubernetes: OVO model for K8s Services support https://review.openstack.org/423908 | 13:05 |
| irenab | ivc_: did you check that devref is still valid? | 13:05 |
| irenab | apuimedo: shall we just create new bp under kuryr-k8s? | 13:07 |
| ivc_ | irenab why would it not? | 13:09 |
| ivc_ | devref is quite abstract on this topic: 'Thread group maps to an unique K8s resource' and does not specify if it should be uid or selfLink | 13:10 |
| apuimedo | irenab: it can just be moved IIRC | 13:10 |
| irenab | ivc_: good | 13:11 |
| ivc_ | irenab you are a bureaucrat xD you want me to create a bug for https://review.openstack.org/#/c/422946/ ? | 13:13 |
| irenab | ivc_: just hate changes not related to anything | 13:14 |
| irenab | in this case, I believe it really fixes a bug | 13:14 |
| irenab | ivc_: but taking you criticism very hard and going to have a lunch :-) | 13:15 |
| apuimedo | :-D | 13:16 |
| ivc_ | apuimedo, https://review.openstack.org/#/c/422910/ either you replied with incomplete copy/paste or i don't understand what you mean there | 13:16 |
| apuimedo | irenab: without you, kuryr would be far too chaotic | 13:16 |
| ivc_ | irenab <3 | 13:16 |
| apuimedo | ivc_: you're not considering the option of my fevered brain malfunctioning | 13:16 |
| apuimedo | xD | 13:16 |
| ivc_ | xD | 13:17 |
| irenab | :-) | 13:17 |
| ivc_ | apuimedo so, can you decrypt that? xD | 13:18 |
| apuimedo | yes | 13:18 |
| apuimedo | basically I was tlaking about another alternative | 13:18 |
| apuimedo | based on keeping a map | 13:19 |
| apuimedo | that is keyed by the event hash | 13:19 |
| apuimedo | *event json hash | 13:19 |
| ivc_ | and? | 13:19 |
| apuimedo | and as value it has a reference to the handler (now that I don't have fever, I see it could be just a set) | 13:19 |
| apuimedo | when you get a new event, if the hash of the event is in the queue, you ignore the event | 13:20 |
| ivc_ | problem is | 13:20 |
| apuimedo | (the hashing would be only of certain relevant parts of the json) | 13:20 |
| ivc_ | you wont get hash-hits ever | 13:20 |
| ivc_ | or | 13:20 |
| ivc_ | oh | 13:20 |
| ivc_ | so not just plain hash(json) | 13:20 |
| ivc_ | imo that would be overly complicated | 13:21 |
| ivc_ | and will require special-casing to extract 'relevant' parts | 13:21 |
| ivc_ | i've got a better solution actually | 13:21 |
| ivc_ | instead of doing it on handler/pipeline layer we can do it as part of k8sclient | 13:22 |
| apuimedo | ivc_: did you consider what we did in the PoC? Just check if it has already kuryr annotations | 13:23 |
| ivc_ | resourceVersion patch can save resourceVersion on 'annotate' conflict and 'watch' will skip everything until it sees that version | 13:23 |
| apuimedo | so we at least filter our own MODIFIED events | 13:23 |
| ivc_ | apuimedo i don't want to filter those | 13:23 |
| *** jchhatbar has joined #openstack-kuryr | 13:24 | |
| apuimedo | which then? | 13:25 |
| apuimedo | maybe I missed something when reading the commit msg | 13:25 |
| ivc_ | so the current 'skip stale' patch is just a temporary bandaid. the k8sclient patch will require more work/testing (tho its still quite trivial) and some thinking about ha case | 13:25 |
| ivc_ | apuimedo k8s itself creates a bunch of events | 13:26 |
| *** janki has quit IRC | 13:26 | |
| ivc_ | try 'watch'ing with curl on namespace/pods endpoint and create a new pod | 13:26 |
| ivc_ | without kuryr there are several events fired | 13:27 |
| apuimedo | ivc_: for scheduling and such | 13:27 |
| apuimedo | yeah | 13:27 |
| *** v1k0d3n has quit IRC | 13:27 | |
| ivc_ | so K8s emits events A,B,C and neither of them have kuryr details | 13:27 |
| ivc_ | we process A, add annotations and trigger event D | 13:27 |
| ivc_ | but we still have B and C which from our point are no different from A | 13:28 |
| ivc_ | and have no kuryr-related info | 13:28 |
| *** v1k0d3n has joined #openstack-kuryr | 13:29 | |
| ivc_ | we do need to skip everything before D and it can be done in k8sclient, but the proposed timed 'skip stale' is just a simple solution and easiest one to implement and we need that | 13:29 |
| apuimedo | yeah | 13:30 |
| ivc_ | its bandaid :) | 13:30 |
| apuimedo | I know | 13:30 |
| apuimedo | It was more so users can put their own kuryr annotations | 13:30 |
| apuimedo | and have that not repeated by kuryr-controller | 13:31 |
| apuimedo | (for ports, so you can specify them) | 13:31 |
| apuimedo | but it had the (not processing our own modified events as a bonus) | 13:31 |
| ivc_ | you mean hashing? | 13:31 |
| apuimedo | no, the check for annotations | 13:32 |
| apuimedo | hashing is a new though I had | 13:32 |
| apuimedo | :P | 13:32 |
| apuimedo | not part of the PoC | 13:32 |
| apuimedo | as you well know | 13:32 |
| ivc_ | uhm can you clarify 'check for annotations' | 13:32 |
| ivc_ | like https://github.com/openstack/kuryr-kubernetes/blob/master/kuryr_kubernetes/controller/handlers/vif.py#L56-L58 ? | 13:33 |
| apuimedo | ivc_: almost. But in this case we'd still have two green threads polling neutron for active | 13:36 |
| apuimedo | which "kills the Neutron" | 13:36 |
| apuimedo | (dark knight reference, sorry) | 13:36 |
| ivc_ | it only polls if active!=True | 13:37 |
| ivc_ | and what are 'two' green threads? | 13:37 |
| apuimedo | what I mean is | 13:38 |
| apuimedo | if we receive the event for our modified (due to our annotate PATCH req), the new handler will go and also poll Neutron.... | 13:40 |
| apuimedo | Damn... | 13:40 |
| apuimedo | I can't read today | 13:40 |
| apuimedo | I see | 13:40 |
| apuimedo | the one that sets the vif finishes there | 13:40 |
| apuimedo | the one after the annotation is put is the one that polls neutron | 13:40 |
| ivc_ | :P | 13:40 |
| apuimedo | but if there was any other event while the polling is happening, wouldn't we end up polling from multiple event handlers | 13:41 |
| ivc_ | nope | 13:41 |
| ivc_ | remember, we have a queue | 13:41 |
| ivc_ | its an actor | 13:41 |
| ivc_ | sort of | 13:41 |
| apuimedo | by cid, yes | 13:42 |
| ivc_ | no concurrent events for the same object | 13:42 |
| apuimedo | but with the cahnge... | 13:42 |
| apuimedo | let me see to what you were changing it | 13:42 |
| apuimedo | I have no recollection of it now | 13:42 |
| ivc_ | with the change we just drop everything but the latest event | 13:43 |
| apuimedo | ah, you change it to self link | 13:43 |
| irenab | apuimedo: ivc_ : anything I can help? | 13:43 |
| *** vikasc has quit IRC | 13:43 | |
| apuimedo | that sounds like my concern is bollocks | 13:43 |
| apuimedo | good | 13:43 |
| ivc_ | the 'selfLink' change is different patch (and its 0-impact as far as we are concerned) | 13:44 |
| apuimedo | ivc_: I know | 13:44 |
| apuimedo | that's what I just said :-) | 13:44 |
| apuimedo | today my brain is almost 55% | 13:44 |
| apuimedo | :-) | 13:44 |
| ivc_ | cool! | 13:44 |
| apuimedo | up from when I did the reviews | 13:45 |
| apuimedo | ivc_: so, on to the new proposal you have for this | 13:45 |
| ivc_ | yes? | 13:46 |
| apuimedo | being that we only process one at a time, won't we just be doing the if and elif https://github.com/openstack/kuryr-kubernetes/blob/master/kuryr_kubernetes/controller/handlers/vif.py#L58-L71 | 13:46 |
| apuimedo | for those? | 13:46 |
| apuimedo | we finish event A | 13:46 |
| apuimedo | then we go to event B | 13:46 |
| apuimedo | oh | 13:46 |
| apuimedo | damn | 13:46 |
| ivc_ | :P | 13:46 |
| ivc_ | xD | 13:46 |
| apuimedo | fucking slow brain | 13:46 |
| apuimedo | this sucks | 13:46 |
| apuimedo | so what was the solution that you favor from the list you put? | 13:47 |
| ivc_ | in the comments for that patch? the 'feedback' one, sort of | 13:47 |
| ivc_ | except it does not require any interaction with Async | 13:48 |
| ivc_ | we just change k8sclient to skip everything after 'annotate' | 13:48 |
| apuimedo | I meant to ask about the 'handle it in the k8s client' | 13:48 |
| apuimedo | how would that change look like? | 13:48 |
| apuimedo | a selector ? | 13:49 |
| ivc_ | after 'resourceVersion' patch in k8sclient we have the 'feedback' now on version conflicts | 13:49 |
| ivc_ | so if there is a version conflict, we resolve it and store that 'resourceVersion' in k8sclient resource->resourceVersion mapping | 13:50 |
| ivc_ | and 'watch' will be changed to check that mapping and skip everything unless it sees the resourceVersion from the mapping | 13:50 |
| ivc_ | thats the implementation | 13:50 |
| *** garyloug has joined #openstack-kuryr | 13:51 | |
| ivc_ | the logic is - after we 'annotate' we need to skip everything before the event triggered by our own annotation | 13:51 |
| ivc_ | quite simple and effective actually | 13:51 |
| ivc_ | i just need to check how it works considering 'resourceVersion' should be treated as opaque string and not a int-sequence | 13:52 |
| apuimedo | ivc_: I think it's safe to treat it as an int | 13:53 |
| ivc_ | its not | 13:53 |
| ivc_ | google says not to | 13:54 |
| apuimedo | the only other thing I can think is that they'll change it to etcd timestamps | 13:54 |
| apuimedo | and we could then move to use that | 13:54 |
| ivc_ | they are very specific about it | 13:54 |
| apuimedo | I know | 13:54 |
| apuimedo | but still, they internally allow you to establish an order | 13:54 |
| apuimedo | the fact that they tell you not to use ordering on client side is a bit vexing | 13:55 |
| ivc_ | i do not think we need the ordering | 13:55 |
| *** vikasc has joined #openstack-kuryr | 13:55 | |
| apuimedo | well, IIRC the watch guarantees order | 13:55 |
| apuimedo | so you could match the string | 13:55 |
| ivc_ | yes | 13:55 |
| apuimedo | and ignore all the events | 13:55 |
| ivc_ | but there's one problem there | 13:56 |
| apuimedo | until you see the annotation | 13:56 |
| ivc_ | 'watch' can restart | 13:56 |
| apuimedo | I know :P | 13:56 |
| apuimedo | when it does, you are stuck with checking from the last resourceversion | 13:56 |
| ivc_ | yup and thats the only place where int/ordering would help | 13:57 |
| ivc_ | but i'd like to solve it with google's constraints in place if possible | 13:57 |
| ivc_ | its an interesting problem :) | 13:58 |
| ivc_ | and that would be a clean solution without 'but's | 13:58 |
| apuimedo | alright | 13:59 |
| ivc_ | but that requires time and i'm kinda low on that valuable resource now :/ hence the bandaid :P | 13:59 |
| apuimedo | got it | 14:00 |
| * apuimedo -> lunch | 14:01 | |
| *** tonanhngo has joined #openstack-kuryr | 14:07 | |
| openstackgerrit | Merged openstack/kuryr-kubernetes: OVO model for K8s Services support https://review.openstack.org/423908 | 14:22 |
| *** hongbin has joined #openstack-kuryr | 14:55 | |
| *** ivc_ has quit IRC | 15:00 | |
| *** ivc_ has joined #openstack-kuryr | 15:01 | |
| *** dougbtv__ is now known as dougbtv | 15:02 | |
| *** tonanhngo has quit IRC | 15:08 | |
| *** tonanhngo has joined #openstack-kuryr | 15:09 | |
| *** tonanhngo has quit IRC | 15:13 | |
| *** saneax is now known as saneax-_-|AFK | 15:15 | |
| *** devvesa has quit IRC | 15:28 | |
| *** devvesa has joined #openstack-kuryr | 15:44 | |
| *** jchhatbar has quit IRC | 15:55 | |
| *** vikasc has quit IRC | 15:56 | |
| *** gsagie has quit IRC | 16:00 | |
| *** tonanhngo has joined #openstack-kuryr | 17:07 | |
| openstackgerrit | Antoni Segura Puimedon proposed openstack/kuryr-libnetwork: Add nested-containers limitations https://review.openstack.org/425040 | 17:22 |
| *** garyloug has quit IRC | 17:25 | |
| openstackgerrit | Antoni Segura Puimedon proposed openstack/kuryr-libnetwork: README: fix nested container rendering https://review.openstack.org/425309 | 17:30 |
| openstackgerrit | Merged openstack/fuxi: Fix logging format https://review.openstack.org/423570 | 17:45 |
| *** garyloug has joined #openstack-kuryr | 17:51 | |
| *** tonanhngo has quit IRC | 17:53 | |
| *** garyloug has quit IRC | 18:26 | |
| *** tonanhngo has joined #openstack-kuryr | 18:53 | |
| *** pcaruana has quit IRC | 18:53 | |
| *** tonanhngo has quit IRC | 18:57 | |
| *** saneax-_-|AFK has quit IRC | 20:46 | |
| *** yamamoto has joined #openstack-kuryr | 21:27 | |
| *** yamamoto has quit IRC | 21:36 | |
| *** v1k0d3n has quit IRC | 21:59 | |
| *** v1k0d3n has joined #openstack-kuryr | 22:01 | |
| *** yamamoto has joined #openstack-kuryr | 22:22 | |
| openstackgerrit | Hongbin Lu proposed openstack/kuryr-libnetwork: Support creating network with pool uuid https://review.openstack.org/424889 | 22:54 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!