| *** hongbin has quit IRC | 00:36 | |
| *** mordred has quit IRC | 00:49 | |
| *** rcernin has quit IRC | 01:39 | |
| *** rcernin has joined #openstack-containers | 01:39 | |
| *** mgariepy has quit IRC | 03:24 | |
| *** mgariepy has joined #openstack-containers | 03:35 | |
| *** ykarel|away has joined #openstack-containers | 03:49 | |
| *** ykarel|away is now known as ykarel | 03:56 | |
| *** udesale has joined #openstack-containers | 04:15 | |
| *** janki has joined #openstack-containers | 04:32 | |
| *** ivve has joined #openstack-containers | 04:39 | |
| *** ykarel has quit IRC | 05:24 | |
| *** ykarel has joined #openstack-containers | 05:40 | |
| *** ricolin has joined #openstack-containers | 06:13 | |
| *** pcaruana has joined #openstack-containers | 07:22 | |
| *** ramishra has joined #openstack-containers | 07:30 | |
| *** ykarel is now known as ykarel|lunch | 07:44 | |
| *** rcernin has quit IRC | 08:13 | |
| *** ykarel|lunch is now known as ykarel | 08:36 | |
| *** chhagarw has joined #openstack-containers | 08:55 | |
| *** FlorianFa has joined #openstack-containers | 10:14 | |
| *** shrasool has joined #openstack-containers | 10:24 | |
| *** ianychoi has quit IRC | 10:25 | |
| *** ign0tus has joined #openstack-containers | 10:43 | |
| *** ramishra has quit IRC | 10:43 | |
| *** salmankhan has joined #openstack-containers | 10:49 | |
| *** salmankhan1 has joined #openstack-containers | 10:52 | |
| *** salmankhan has quit IRC | 10:54 | |
| *** salmankhan1 is now known as salmankhan | 10:54 | |
| *** udesale has quit IRC | 11:15 | |
| *** ianychoi has joined #openstack-containers | 11:31 | |
| *** janki has quit IRC | 11:43 | |
| *** dims has quit IRC | 11:45 | |
| *** ramishra has joined #openstack-containers | 12:17 | |
| *** shrasool has quit IRC | 13:14 | |
| *** shrasool has joined #openstack-containers | 13:35 | |
| *** udesale has joined #openstack-containers | 13:44 | |
| *** pcaruana has quit IRC | 13:50 | |
| *** zul has joined #openstack-containers | 14:06 | |
| *** dims has joined #openstack-containers | 14:14 | |
| *** shrasool has quit IRC | 14:15 | |
| *** shrasool has joined #openstack-containers | 14:16 | |
| *** pcaruana has joined #openstack-containers | 14:25 | |
| *** FlorianFa has quit IRC | 14:33 | |
| *** ykarel is now known as ykarel|away | 14:34 | |
| *** FlorianFa has joined #openstack-containers | 14:46 | |
| *** ykarel|away has quit IRC | 14:47 | |
| *** hongbin has joined #openstack-containers | 14:52 | |
| *** shrasool has quit IRC | 15:27 | |
| *** shrasool has joined #openstack-containers | 15:38 | |
| *** rpittau has joined #openstack-containers | 15:43 | |
| *** ykarel|away has joined #openstack-containers | 16:11 | |
| *** shrasool has quit IRC | 16:23 | |
| *** ricolin has quit IRC | 16:31 | |
| *** ramishra has quit IRC | 16:32 | |
| *** ign0tus has quit IRC | 16:41 | |
| *** udesale has quit IRC | 16:48 | |
| openstackgerrit | Merged openstack/magnum stable/queens: Update heat-container-agent version tag https://review.openstack.org/619356 | 17:15 |
|---|---|---|
| *** shrasool has joined #openstack-containers | 17:16 | |
| *** chhagarw has quit IRC | 17:25 | |
| *** shrasool has quit IRC | 17:56 | |
| *** ykarel|away has quit IRC | 18:09 | |
| *** salmankhan has quit IRC | 18:32 | |
| *** rpittau has quit IRC | 18:46 | |
| *** lpetrut has joined #openstack-containers | 18:46 | |
| mnaser | flwang: have you noticed/seen docker just randomly stopping in magnum? | 19:13 |
| mnaser | like im seeing the docker process randomly stopping and the cluster acting all weird | 19:13 |
| mnaser | seen this happening both on master and minion | 19:13 |
| mnaser | docker-containerd-current[888]: time="2018-11-22T19:31:52.118293419Z" level=info msg="stopping containerd after receiving terminated" | 19:32 |
| *** shrasool has joined #openstack-containers | 19:52 | |
| flwang | mnaser: no, never see that | 20:32 |
| flwang | is there any more information from kubelet log? | 20:33 |
| mnaser | flwang: kubelet has no idea, it just sees docker disappear | 20:33 |
| mnaser | and docker process disappears | 20:33 |
| mnaser | i tried upgrading the atomic host and going to try again | 20:33 |
| mnaser | this happens when running sonobuoy | 20:33 |
| flwang | i assume you're using the default docker version in fedora atomic 27? | 20:34 |
| mnaser | flwang: yeah, but its an older image, so maybe that it | 20:34 |
| flwang | what do you mean older? | 20:35 |
| flwang | what's the version of your fedora atomic 27? | 20:35 |
| mnaser | flwang: i think this one is pretty old. 2018-02-01 | 20:38 |
| mnaser | (Feb) | 20:38 |
| mnaser | upgraded the host now and trying again | 20:38 |
| mnaser | looks much better | 20:39 |
| flwang | mnaser: cool | 20:39 |
| mnaser | flwang: sorry for noise, i'll update our images | 20:39 |
| mnaser | too bad fedora doesnt ship .raw images so we have to store converted ones to deploy via osa (boo) | 20:39 |
| flwang | mnaser: no problem, any time | 20:39 |
| flwang | mnaser: from magnum side, we're working with fedora/coreos atomic to figure out the requirements we need | 20:40 |
| flwang | for long term | 20:40 |
| mnaser | flwang: awesome. it's a great platform for exactly the stuff that we're doing! | 20:41 |
| flwang | mnaser: pls return any feedback and we're happy to address | 20:41 |
| mnaser | flwang: absolutely. i've been trying to get more people to know magnum is a really good option | 20:42 |
| mnaser | as you saw in the talk some people thought that it doesn't provide you with the tooling that you need out of the box :) | 20:42 |
| mnaser | wanna kill that fud | 20:42 |
| flwang | mnaser: Yep, i can feel that from the session. thank you. | 20:42 |
| flwang | there are a lot of tools can help to deploy k8s, but magnum is the good one if you want to integrate with openstack | 20:43 |
| flwang | i think that is key for most of the services in openstack ecosystem | 20:43 |
| mnaser | flwang: yep, also, i have seen some issues recently with flannel not properly going up cause its stuck waiting for etcd | 20:47 |
| mnaser | i havent debugged much more, it happens from time to time | 20:47 |
| mnaser | i will try to investigate more | 20:48 |
| flwang | mnaser: why not go for calico? | 20:53 |
| mnaser | flwang: dunno. it was the default? :p | 20:53 |
| mnaser | flwang: did you use calico when you ran the conformance tests? | 20:53 |
| flwang | mnaser: yep | 20:53 |
| flwang | i use calico | 20:54 |
| mnaser | it's failing for me right now using flannel | 20:54 |
| flwang | and in Catalyst Cloud, we use calico as default, we don't 'support' flannel | 20:54 |
| mnaser | Nov 22 20:52:35.022: INFO: Failed to get response from guestbook. err: the server is currently unable to handle the request (get services frontend), response: | 20:54 |
| mnaser | any reasons behind picking one vs the other? | 20:54 |
| flwang | mnaser: we need network policy | 20:54 |
| flwang | for better security | 20:54 |
| mnaser | flwang: ack, it looks like the default is flannel.. i think? let me see | 20:55 |
| flwang | yep, default is flannel | 20:55 |
| mnaser | ill try to hack on this a little bit just to make sure it works properly with flannel | 20:56 |
| mnaser | i wonder what gke runs, or maybe something like kuryr | 20:56 |
| flwang | mnaser: gke supports both | 21:04 |
| flwang | by default is flannel | 21:04 |
| flwang | and if you want to enable network policy, then calico | 21:04 |
| flwang | mnaser: ^ | 21:04 |
| mnaser | flwang: gotcha. is network policy required for conformance? | 21:04 |
| flwang | mnaser: no | 21:06 |
| flwang | with flannel, it should work as well | 21:06 |
| mnaser | flwang: ok, thanks, i'll try to figure out why things are not working right now :\ | 21:06 |
| flwang | mnaser: no worries, let me know if there is any question i can help | 21:06 |
| *** ivve has quit IRC | 21:12 | |
| *** lpetrut has quit IRC | 21:23 | |
| mnaser | flwang: ugh, found it.. a reboot of a vm broke it. https://github.com/openstack/magnum/blob/f0dec728e78bcb3851b1a484b73bfe567b3c1fc9/magnum/drivers/common/templates/swarm/fragments/network-service.sh#L61-L63 | 21:31 |
| mnaser | or rather https://github.com/openstack/magnum/blob/9375dc2ae51c8aed39ba57984bc8cfe07ab070e4/magnum/drivers/common/templates/kubernetes/fragments/flannel-service.sh#L25-L27 | 21:32 |
| flwang | haha, you added it | 21:35 |
| mnaser | flwang: well i fixed it because it wasn't working at all :) | 21:37 |
| mnaser | now it doesn't work on reboot | 21:37 |
| flwang | so you mean after reboot the vm, the ipstables lost the rule? | 21:38 |
| strigazi | mnaser: https://gitlab.cern.ch/cloud-infrastructure/magnum/commit/52a69ff2d63f1cb22332711254bf682b4f022bc6 | 21:44 |
| mnaser | strigazi: oh thats cool, mind if i cherry pick and push that upstream? | 21:45 |
| mnaser | unless thats there and i cant see it :D | 21:45 |
| strigazi | mnaser: not there yet | 21:45 |
| strigazi | mnaser: shoot | 21:45 |
| strigazi | mnaser: L1TF and spectre/meltdown reboots campaigns revealed those | 21:46 |
| mnaser | strigazi: must have been a fun time | 21:46 |
| strigazi | it was too much fun | 21:47 |
| openstackgerrit | Mohammed Naser proposed openstack/magnum master: Add iptables -P FORWARD ACCEPT unit https://review.openstack.org/619643 | 21:51 |
| mnaser | strigazi: that's an initial go at it | 21:51 |
| strigazi | mnaser: it is good for now. we need to drop in another patch the rule from the other unit. | 21:52 |
| flwang | strigazi: did you finish the rebase of https://review.openstack.org/#/c/561858/ | 21:53 |
| strigazi | mnaser: I'll do tmr the storyboard, reno dance tmr | 21:53 |
| mnaser | strigazi: ok thank you, i already actaully filed a bug | 21:53 |
| flwang | strigazi: i rebased i t locally, but got issues | 21:53 |
| mnaser | https://storyboard.openstack.org/#!/story/2004416 | 21:53 |
| mnaser | so if you wanna reuse that and you only have reno dance left :) | 21:53 |
| flwang | seems those bash scripts are executed before the heat-params file | 21:53 |
| * mnaser goes back to breaking stuff | 21:53 | |
| strigazi | flwang: I was trying to but then the network didn't let me. FIrst thing tmr. leave a comment for the issues you saw. | 21:54 |
| flwang | strigazi: ok | 21:55 |
| strigazi | flwang: i'm simplifying the patch to see less issues. | 21:55 |
| strigazi | flwang: plus the agent needs more things. openssh and openssl pkgs | 21:56 |
| flwang | strigazi: ok, should i stop review it now until you rebase it? | 21:56 |
| strigazi | yes, for review can you take a look to NGs, we can merge it | 21:57 |
| strigazi | and the jim's keypair patch | 21:57 |
| strigazi | I added you as reviewer | 21:57 |
| strigazi | https://review.openstack.org/#/c/590443/ https://review.openstack.org/#/c/607363/ | 21:58 |
| * strigazi is going to bed and prays the network gods will be happy tmr. | 22:00 | |
| *** shrasool has quit IRC | 22:03 | |
| *** rcernin has joined #openstack-containers | 22:05 | |
| flwang | strigazi: ok, got it | 22:13 |
| *** shrasool has joined #openstack-containers | 22:33 | |
| *** shrasool has quit IRC | 22:39 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!