*** openstack has joined #kata-dev | 09:52 | |
*** ChanServ sets mode: +o openstack | 09:52 | |
*** sgarzare has quit IRC | 11:02 | |
*** ailan has joined #kata-dev | 11:05 | |
*** hashar has joined #kata-dev | 12:02 | |
*** sgarzare has joined #kata-dev | 12:02 | |
*** sgarzare has quit IRC | 12:09 | |
*** devimc has joined #kata-dev | 12:13 | |
*** sameo has quit IRC | 12:15 | |
*** sameo has joined #kata-dev | 12:16 | |
kata-irc-bot | <fidencio> @james.o.hunt, @graham.whaley, are you guys (or someone else in this timezone) familiar with shimv2? | 12:38 |
---|---|---|
kata-irc-bot | <fidencio> I'm clearly facing a weird error, where configPath for shimv2 is "" ... making it always fallback to the default config path, which may break totally depending on how the distro packages it | 12:39 |
kata-irc-bot | <graham.whaley> hi @fidencio - sparse week in the UK this week :slightly_smiling_face: I saw your query, but I don't know a definitive answer - you'll need maybe @archana.m.shinde or @xu for that. afaik though, I thought shimv2 was just for containerd, and not any crio, thus no docker.... but, I could be wrong | 12:39 |
kata-irc-bot | <graham.whaley> @fidencio - better describe your stackup here, and hope somebody knows... | 12:40 |
kata-irc-bot | <fidencio> it's used in CRI-O and OpenShift | 12:41 |
kata-irc-bot | <fidencio> So, https://github.com/kata-containers/runtime/blob/master/containerd-shim-v2/create.go#L151 seems quite fishy | 12:41 |
kata-irc-bot | <fidencio> why configPath would come from the `ociSpec.Annotations`? | 12:42 |
kata-irc-bot | <fidencio> ociSpec.Annotations looks like: | 12:43 |
kata-irc-bot | <fidencio> ```Apr 14 12:25:02 ci-ln-x7v7s2t-002ac-cf4th-worker-centralus3-jm9mn crio[1360]: time="2020-04-14T12:25:02.703981407Z" level=warning msg="FIDENCIO | create | ociSpec.Annotations: map[app:hello-openshift io.container.manager:cri-o io.kubernetes.container.name:POD | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.Annotations:{\"k8s.ovn.org/pod-networks\":\"{\\\"default\\\":{\\\"ip_addresses\\\":[\\\"10.129.2.21/23\\\"],\\\"mac_address\\\":\\\"de:9c:60:81:02:16\\\",\\\"gateway_ips\\\":[\\\"10.129.2.1\\\"],\\\"ip_address\\\":\\\"10.129.2.21/23\\\",\\\"gateway_ip\\\":\\\"10.129.2.1\\\"}}\",\"k8s.v1.cni.cncf.io/network-status\":\"[{\\n \\\"name\\\": \\\"ovn-kubernetes\\\",\\n \\\"interface\\\": \\\"eth0\\\",\\n \\\"ips\\\": [\\n | 12:43 |
kata-irc-bot | \\\"10.129.2.21\\\"\\n ],\\n \\\"mac\\\": \\\"de:9c:60:81:02:16\\\",\\n \\\"default\\\": true,\\n \\\"dns\\\": {}\\n}]\",\"k8s.v1.cni.cncf.io/networks-status\":\"[{\\n \\\"name\\\": \\\"ovn-kubernetes\\\",\\n \\\"interface\\\": \\\"eth0\\\",\\n \\\"ips\\\": [\\n \\\"10.129.2.21\\\"\\n ],\\n \\\"mac\\\": \\\"de:9c:60:81:02:16\\\",\\n \\\"default\\\": true,\\n \\\"dns\\\": | 12:43 |
kata-irc-bot | {}\\n}]\",\"kubernetes.io/config.seen\":\"2020-04-14T12:24:57.128717256Z\",\"kubernetes.io/config.source\":\"api\"} | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.CNIResult:{\"cniVersion\":\"0.4.0\",\"interfaces\":[{\"name\":\"c71cc5637475c40\",\"mac\":\"f6:b4:d8:ff:40:a0\"},{\"name\":\"eth0\",\"mac\":\"de:9c:60:81:02:16\",\"sandbox\":\"/var/run/netns/f125e35b-8602-4744-a219-c420b3340a9f\"}],\"ips\":[{\"version\":\"4\",\"interface\":1,\"address\":\"10.129.2.21/23\",\"gateway\":\"10.129.2.1\"}],\"dns\":{}} | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.CgroupParent:kubepods-besteffort-podd027e718_e6cf_48c1_b26e_812cb267b6be.slice io.kubernetes.cri-o.ContainerID:c71cc5637475c40c75cc3531ee30ffb13bac2f02c5e80a3d25f5e3148e7b239a io.kubernetes.cri-o.ContainerName:k8s_POD_example-vm-23_default_d027e718-e6cf-48c1-b26e-812cb267b6be_0 io.kubernetes.cri-o.ContainerType:sandbox io.kubernetes.cri-o.Created:2020-04-14T12:25:00.333691905Z io.kubernetes.cri-o.HostName:example-vm-23 | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.HostNetwork:false io.kubernetes.cri-o.HostnamePath:/var/run/containers/storage/overlay-containers/c71cc5637475c40c75cc3531ee30ffb13bac2f02c5e80a3d25f5e3148e7b239a/userdata/hostname io.kubernetes.cri-o.IP.0:10.129.2.21 io.kubernetes.cri-o.KubeName:example-vm-23 | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.Labels:{\"app\":\"hello-openshift\",\"io.kubernetes.container.name\":\"POD\",\"io.kubernetes.pod.name\":\"example-vm-23\",\"io.kubernetes.pod.namespace\":\"default\",\"io.kubernetes.pod.uid\":\"d027e718-e6cf-48c1-b26e-812cb267b6be\"} io.kubernetes.cri-o.LogPath:/var/log/pods/default_example-vm-23_d027e718-e6cf-48c1-b26e-812cb267b6be/c71cc5637475c40c75cc3531ee30ffb13bac2f02c5e80a3d25f5e3148e7b239a.log | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.Metadata:{\"name\":\"example-vm-23\",\"uid\":\"d027e718-e6cf-48c1-b26e-812cb267b6be\",\"namespace\":\"default\"} io.kubernetes.cri-o.MountPoint:/var/lib/containers/storage/overlay/0b87886d923f2684363c37c5979edf05e787702f066529aaa4583bb52c4e91dd/merged io.kubernetes.cri-o.Name:k8s_example-vm-23_default_d027e718-e6cf-48c1-b26e-812cb267b6be_0 io.kubernetes.cri-o.Namespace:default io.kubernetes.cri-o.NamespaceOptions:{\"pid\":1} | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.PortMappings:[] io.kubernetes.cri-o.PrivilegedRuntime:false io.kubernetes.cri-o.ResolvPath:/var/run/containers/storage/overlay-containers/c71cc5637475c40c75cc3531ee30ffb13bac2f02c5e80a3d25f5e3148e7b239a/userdata/resolv.conf io.kubernetes.cri-o.RuntimeHandler:kata-qemu io.kubernetes.cri-o.SandboxID:c71cc5637475c40c75cc3531ee30ffb13bac2f02c5e80a3d25f5e3148e7b239a io.kubernetes.cri-o.SeccompProfilePath: | 12:43 |
kata-irc-bot | io.kubernetes.cri-o.ShmPath:/var/run/containers/storage/overlay-containers/c71cc5637475c40c75cc3531ee30ffb13bac2f02c5e80a3d25f5e3148e7b239a/userdata/shm io.kubernetes.pod.name:example-vm-23 io.kubernetes.pod.namespace:default io.kubernetes.pod.uid:d027e718-e6cf-48c1-b26e-812cb267b6be | 12:43 |
kata-irc-bot | k8s.ovn.org/pod-networks:{\"default\":{\"ip_addresses\":[\"10.129.2.21/23\"],\"mac_address\":\"de:9c:60:81:02:16\",\"gateway_ips\":[\"10.129.2.1\"],\"ip_address\":\"10.129.2.21/23\",\"gateway_ip\":\"10.129.2.1\"}} k8s.v1.cni.cncf.io/network-status:[{\n \"name\": \"ovn-kubernetes\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.21\"\n ],\n \"mac\": \"de:9c:60:81:02:16\",\n \"default\": true,\n \"dns\": {}\n}] | 12:43 |
kata-irc-bot | k8s.v1.cni.cncf.io/networks-status:[{\n \"name\": \"ovn-kubernetes\",\n \"interface\": \"eth0\",\n \"ips\": [\n \"10.129.2.21\"\n ],\n \"mac\": \"de:9c:60:81:02:16\",\n \"default\": true,\n \"dns\": {}\n}] kubernetes.io/config.seen:2020-04-14T12:24:57.128717256Z kubernetes.io/config.source:api]"``` | 12:43 |
kata-irc-bot | <fidencio> | 12:43 |
kata-irc-bot | <fidencio> It seems to be the pod related info, not the general config | 12:43 |
*** vgoyal has joined #kata-dev | 12:45 | |
*** dklyle has joined #kata-dev | 12:56 | |
*** sgarzare has joined #kata-dev | 12:59 | |
kata-irc-bot | <fidencio> Okay, explained at: https://github.com/kata-containers/documentation/blob/master/how-to/containerd-kata.md | 13:04 |
kata-irc-bot | <fidencio> "This `ConfigPath` option is optional. If you do not specify it, shimv2 first tries to get the configuration file from the environment variable `KATA_CONF_FILE`. If neither are set, shimv2 will use the default Kata configuration file paths (`/etc/kata-containers/configuration.toml` and `/usr/share/defaults/kata-containers/configuration.toml`)." I still think it's a bug though. If not specificied, it should look at the default kata | 13:05 |
kata-irc-bot | config specified at the build time | 13:05 |
*** crobinso has joined #kata-dev | 13:08 | |
*** pcaruana has quit IRC | 13:28 | |
kata-irc-bot | <graham.whaley> @fidencio - can you clarify 'config specified at the build time' - specified where for what build? I see those two default config paths hard wired into the katautils code base - they don't look over-ridable to me at a glance | 13:37 |
kata-irc-bot | <fidencio> Seems they are overridable. @CONFIG_PATH@ should do the trick. | 13:41 |
kata-irc-bot | <fidencio> But I just realised that Fedora package may be screwed up | 13:42 |
kata-irc-bot | <fidencio> In Fedora package we try to override: "CONFDIR" and "DEFAULTSDIR", which should be good enough to have CONFIG_PATH, which is defined as "$CONFDIR/$CONFIGFILE" | 13:43 |
kata-irc-bot | <fidencio> But no, CONFDIR / DEFAULTSDIR, those are not overridable | 13:43 |
kata-irc-bot | <fidencio> So, I'm pretty sure it's an issue, but now I'm more than sure the issue is in the Fedora package | 13:44 |
kata-irc-bot | <graham.whaley> ok, thx for the update. I'm not convinced there is not a potential issue with kata between the `defaultRuntimeConfiguration` var in the cli generated file (generated from the Makefile) and the var in pkg/katautils/config-settings.go, which I think is not auto generated.... hence, maybe kata-runtime v1 you can config at makefile, but shimv2 you cannot - that might also be worth looking at if you hit a dead end. | 13:58 |
kata-irc-bot | <fidencio> That's what I'm going to test right now (well, when my cluster is up ...), because you should be able to configure both at makefile, shouldn't you? | 14:02 |
kata-irc-bot | <fidencio> having shimv2 not being configurable does look like a bug to me | 14:02 |
kata-irc-bot | <fidencio> Also, `defaultRuntimeConfiguration` is declared as a `var`, instead of a `const` in `pkg/katautils/config-settings.go`, which makes me think that it can be overriden | 14:05 |
kata-irc-bot | <graham.whaley> heh, I think I noted at the top of that file it is a var so the test suite can over-ride it... | 14:05 |
kata-irc-bot | <graham.whaley> oh, oops, wrong file... the other file says that | 14:06 |
kata-irc-bot | <graham.whaley> but, yes, I would think it should be over-rideable - and maybe we need to use the same Makefile sed method.. | 14:06 |
kata-irc-bot | <fidencio> Yeah, I'll do the following checks: • If passing `CONFIG_PATH=...` to as a make var in the Fedora solves the issues, then we're good; • If not, I'll keep tracing it down and most likely open an issue / provide a patch to kata itself • And then, get back to the first step ... | 14:08 |
*** pcaruana has joined #kata-dev | 14:16 | |
*** fidencio has joined #kata-dev | 14:19 | |
kata-irc-bot | <fidencio> BTW, meeting is in an hour, right? I'm never sure whether my calendar is right about this one | 14:23 |
kata-irc-bot | <graham.whaley> my calendar says in about 28 minutes.... @xu @eric.ernst - is that right? | 14:32 |
kata-irc-bot | <eric.ernst> Yeah | 14:34 |
kata-irc-bot | <archana.m.shinde> @xu @eric.ernst @salvador.fuentes I will have to skip today's arch call | 14:48 |
kata-irc-bot | <fidencio> @eric.ernst, so, the binaries you mentioned are supposed to be created by the distro? | 15:56 |
kata-irc-bot | <fidencio> If so, we don't create anything like that on Fedora | 15:56 |
kata-irc-bot | <fidencio> and we use CRI-O just calling "/usr/bin/containerd-kata-shim-v2" without any argument | 15:56 |
kata-irc-bot | <fidencio> which will lead to configPath being "", leading to use the default one ... | 15:57 |
kata-irc-bot | <fidencio> which is the default one in according to kata, not the default one specified by the distro | 15:57 |
*** devimc has quit IRC | 16:21 | |
*** devimc has joined #kata-dev | 16:21 | |
*** sgarzare has quit IRC | 16:31 | |
*** hashar has quit IRC | 16:49 | |
kata-irc-bot | <fidencio> @archana.m.shinde, I've never revendored something. /o\. I'll try to open a PR later Today. | 17:59 |
kata-irc-bot | <graham.whaley> @fidencio just fyi, I think this is the code in kata-deploy that generates the shim script re-directs: https://github.com/kata-containers/packaging/blob/master/kata-deploy/scripts/kata-deploy.sh#L199-L217 If you are not generating them for your distro, then I presume you are only supporting one kata hypervisor (qemu?) under containerd, and only have one kata runtime entry in your containerd config file... | 18:30 |
kata-irc-bot | <graham.whaley> I don't know how we do this for OBS packages btw - we'd have to ask @jose.carlos.venegas.m , and implicate @eric.ernst some more ... | 18:30 |
kata-irc-bot | <fidencio> @graham.whaley, that's interesting. So, I don't have containerd, at all, here. | 18:46 |
kata-irc-bot | <salvador.fuentes> @fidencio here a guide for revendoring, hope it helps: https://github.com/kata-containers/community/blob/master/VENDORING.md | 18:46 |
kata-irc-bot | <fidencio> I'm running shimv2 against CRI-O | 18:46 |
kata-irc-bot | <fidencio> So, let me create that file. Seems that the way to go is passing KATA_CONF_FILE | 18:47 |
kata-irc-bot | <fidencio> Still, it does look wrong that the default defined config path is not respected | 18:47 |
kata-irc-bot | <fidencio> @salvador.fuentes, cool, let me read that, thanks! | 18:48 |
kata-irc-bot | <graham.whaley> @fidencio - sure - you might also (if you have not already) check out how kata-deploy sets up the crio conf file - also in that same shell script. | 19:25 |
kata-irc-bot | <fidencio> the crio conf file is solid. I've made it work \o/ | 19:27 |
kata-irc-bot | <fidencio> it was just this bit that I left behind as I had an easy workaround, as creating /etc/kata-containers/ | 19:27 |
gwhaley | fidencio: excellent - and have a good evening! :-) | 19:30 |
* gwhaley signs off.. | 19:30 | |
*** gwhaley has quit IRC | 19:32 | |
kata-irc-bot | <fidencio> @eric.ernst https://github.com/kata-containers/runtime/issues/2610 | 19:55 |
kata-irc-bot | <fidencio> I've added the workaround of using the script setting un `KATA_CONF_FILE` on Fedora | 19:55 |
*** tobberydberg_ has quit IRC | 20:17 | |
fidencio | devimc: I will have to bother you | 20:18 |
fidencio | devimc: although fuentess shared the doc explaining how to update the vendor, seems that I'm doing something stupid on my side | 20:18 |
fidencio | devimc: I've changed the revision of agent on Gopkg.toml | 20:19 |
fidencio | devimc: it complains about "it has a dependency on github.com/sirupsen/logrus with constraint v1.4.2, which has no overlap with existing constraint" | 20:20 |
fidencio | devimc: okay, no problem, let's also bump logrus then | 20:20 |
devimc | fidencio: let me try on my side, what is the version of the agent? | 20:21 |
devimc | sha | 20:21 |
fidencio | devimc: 5bf8d4cc4614d0e0b3202f0a39d723195e341df8 | 20:21 |
fidencio | devimc: that's the commit, the latest one | 20:22 |
*** tobberydberg has joined #kata-dev | 20:22 | |
devimc | fidencio: weird, it didn't bring anything, just Gopkg.* were modified | 20:27 |
devimc | fidencio: seems like there are no changes in agent's pkg or protocols | 20:27 |
fidencio | devimc: what happens if you use `dep ensure -update`? | 20:27 |
fidencio | devimc: that's weird, because there's changes in the agent | 20:27 |
devimc | fidencio: indeed, there are changes, but not in pkg or protocols | 20:29 |
devimc | fidencio: runtime only cares about changes in those directories | 20:30 |
*** tobberydberg has quit IRC | 20:30 | |
*** crobinso has quit IRC | 20:30 | |
fidencio | devimc: wait ... | 20:31 |
fidencio | devimc: I'm dealing with shim, not with runtime | 20:31 |
devimc | ohh | 20:31 |
devimc | lol | 20:31 |
fidencio | devimc: sorry, that was my fault, I was pretty sure I had mentioned that (but I didn't) | 20:31 |
devimc | fidencio: I faced the same issue in the runtime, take a look https://github.com/kata-containers/runtime/pull/2351/commits | 20:36 |
*** tobberydberg has joined #kata-dev | 20:36 | |
*** tobberydberg has quit IRC | 20:37 | |
fidencio | devimc: so, update one by one? | 20:38 |
devimc | fidencio: yes | 20:38 |
devimc | fidencio: start with logrus | 20:38 |
*** canyounot has joined #kata-dev | 20:38 | |
*** tobberydberg has joined #kata-dev | 20:42 | |
*** tobberydberg has quit IRC | 20:43 | |
*** canyounot has quit IRC | 20:43 | |
*** tobberydberg has joined #kata-dev | 20:45 | |
*** tobberydberg has quit IRC | 20:45 | |
*** tobberydberg has joined #kata-dev | 20:46 | |
*** tobberydberg has quit IRC | 20:46 | |
*** tobberydberg has joined #kata-dev | 20:46 | |
*** tobberydberg has quit IRC | 20:47 | |
*** tobberydberg has joined #kata-dev | 20:47 | |
*** tobberydberg has quit IRC | 20:47 | |
*** tobberydberg has joined #kata-dev | 20:51 | |
devimc | fidencio: https://github.com/devimc/kata-shim/commits/topic/vendor/updateAgent | 20:53 |
devimc | last two commits ;) | 20:53 |
*** tobberydberg has quit IRC | 20:55 | |
fidencio | devimc: I've done that on my side, took me some time to format it in a similar way you previously did | 20:57 |
fidencio | devimc: in any case, at this point I guess it's better if you just open the PR | 20:57 |
fidencio | and I will review it | 20:57 |
fidencio | devimc: ah, no, I have all the changelog on my side | 20:58 |
devimc | fidencio: ohh no, it's your fix, feel free to take and rebase my patches | 20:58 |
fidencio | devimc: so, maybe better if I open the PR | 20:58 |
devimc | fidencio: yeah - go for it | 20:59 |
*** tobberydberg has joined #kata-dev | 21:01 | |
*** tobberydberg has quit IRC | 21:02 | |
kata-irc-bot | <archana.m.shinde> devimc: so you needed to update a package one by one to get rid of the error fidencio mentioned | 21:03 |
*** tobberydberg has joined #kata-dev | 21:03 | |
kata-irc-bot | <archana.m.shinde> ? | 21:03 |
fidencio | seems so, it's not as practical as I'd expect but well ... | 21:04 |
kata-irc-bot | <archana.m.shinde> @fidencio: to be honest, thats the first time I am seeing such an error | 21:05 |
devimc | fidencio: it's better, that way you can include a shortlog per package | 21:05 |
kata-irc-bot | <archana.m.shinde> has worked pretty well so far | 21:05 |
devimc | fidencio: nudge - btw don't forget the shortlog | 21:06 |
fidencio | devimc: yeah, I've added the shortlog (well, git log --pretty=oneline v1.0.4..v1.4.2) | 21:06 |
fidencio | not sure if you use a different format | 21:06 |
devimc | fidencio: git log --no-merges --abbrev-commit --pretty=oneline "${old_commit}..${new_commit}" | 21:07 |
devimc | ;) | 21:07 |
fidencio | cool! | 21:07 |
*** devimc has quit IRC | 21:07 | |
*** ailan has quit IRC | 21:08 | |
*** ailan has joined #kata-dev | 21:08 | |
kata-irc-bot | <fidencio> Interesting ... | 21:18 |
kata-irc-bot | <fidencio> I'm not using Julio commits, mainly because I want to have it done from my side | 21:19 |
kata-irc-bot | <fidencio> but I only change logrus version on `Gopkg.toml` , run a `dep ensure` and the version in the lock is wrong | 21:20 |
kata-irc-bot | <fidencio> :S | 21:20 |
*** sameo has quit IRC | 21:31 | |
*** sameo has joined #kata-dev | 21:32 | |
kata-irc-bot | <archana.m.shinde> thats odd | 21:35 |
kata-irc-bot | <archana.m.shinde> @fidencio | 21:35 |
kata-irc-bot | <archana.m.shinde> I saw the same | 21:35 |
kata-irc-bot | <archana.m.shinde> I had to change the version to revision | 21:35 |
kata-irc-bot | <archana.m.shinde> to fix that | 21:35 |
kata-irc-bot | <archana.m.shinde> not sure if logrus is not tagged correctly | 21:36 |
kata-irc-bot | <fidencio> that's it! | 21:40 |
kata-irc-bot | <fidencio> @archana.m.shinde++ @julio.montes++ | 21:41 |
kata-irc-bot | <fidencio> Okay, finally updated the PR. | 21:47 |
*** sameo has quit IRC | 21:47 | |
kata-irc-bot | <fidencio> Thanks for the patience :slightly_smiling_face: | 21:47 |
kata-irc-bot | <salvador.fuentes> @archana.m.shinde @fidencio @jose.carlos.venegas.m shim travis failure should be fixed by https://github.com/kata-containers/shim/pull/245, can you please ack? | 22:22 |
kata-irc-bot | <salvador.fuentes> I think that we can skip full testing as the change only affects travis file | 22:23 |
kata-irc-bot | <archana.m.shinde> done @salvador.fuentes | 22:44 |
kata-irc-bot | <fidencio> @salvador.fuentes, I don't have powers to do reviews on Shim :S | 22:44 |
kata-irc-bot | <salvador.fuentes> @fidencio I have added you to the shim team, but not sure if you'll be able to do reviews on existing PRs | 22:58 |
kata-irc-bot | <fidencio> @salvador.fuentes, worked like a charm! | 23:05 |
kata-irc-bot | <fidencio> Thanks for adding me there! | 23:05 |
*** crobinso has joined #kata-dev | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!