Friday, 2019-04-05

« Thursday, 2019-04-04 Index Saturday, 2019-04-06 »
*** eernst has joined #kata-dev00:27
*** eernst has quit IRC00:31
*** eernst has joined #kata-dev00:42
*** eernst has quit IRC00:51
*** eernst has joined #kata-dev03:05
kata-irc-bot<eric.ernst> @bergwolf around?03:35
kata-irc-bot<eric.ernst> @xu @zhangwei555?03:35
kata-irc-bot<eric.ernst> looking for an easy +1: https://github.com/kata-containers/agent/pull/51703:36
kata-irc-bot<eric.ernst> @teawater03:36
kata-irc-bot<eric.ernst> Thanks @teawater :slightly_smiling_face:03:51
*** eernst has quit IRC04:11
*** sameo has joined #kata-dev05:02
*** igordc has quit IRC05:19
*** kata-irc-bot has quit IRC05:45
*** kata-irc-bot has joined #kata-dev05:45
*** tmhoang has joined #kata-dev06:12
*** sgarzare has joined #kata-dev06:42
*** sameo has quit IRC06:48
*** auk has quit IRC07:06
*** sameo has joined #kata-dev07:42
*** gwhaley has joined #kata-dev08:04
*** davidgiluk has joined #kata-dev08:05
kata-irc-bot<graham.whaley> Good morning @stefanha @davidgiluk - I said I'd get you a link to the LVM based block snapshotter work that has bee PR'd up to containerd... it can be seen at https://github.com/containerd/containerd/pull/3136 - pushed by @gmmaharaj :slightly_smiling_face:08:14
davidgilukgraham.whaley: Thanks!08:16
stefanhagraham.whaley: cool, will take a look08:25
stefanhagraham.whaley: Sounds like the purpose is to work with CRI-O's LVM support?08:26
stefanhaIt's not clear to me how this compares/interacts with Docker storage graph drivers.08:27
kata-irc-bot<graham.whaley> @stefanha - yeah, I don't know what the plans are around making this work (or not) with a wider set of stacks, ecosystems etc. I'll let @gmmaharaj @eric.ernst and @manohar.r.castelino answer that.08:28
stefanhahttps://github.com/containerd/containerd/blob/master/snapshots/snapshotter.go08:31
stefanha^-- seems to me containerd's snapshotter is a complete replacement for Docker storage graph drivers08:31
stefanhagraham.whaley: At first glance it looks like Kata can still use block devices with this.08:34
stefanhagraham.whaley: It creates an LVM thin-provisioned volume, sets up a file system on it (default: XFS).08:35
stefanhagraham.whaley: When a child snapshot (e.g. image layer) is created, a new volume is created based on the parent and its filesystem contents.08:35
stefanhagraham.whaley: kata-runtime should see a rootfs that is on an LVM block device.  I think today's block devices would still work.08:36
kata-irc-bot<calm.wu> @graham.whaley Excuse me. kubectl logs coredns-7d98bc987b-l8j2b -n kube-system "dial tcp 10.96.0.1:443: connect: network is unreachable", but [root@192 k8s_space]# kubectl get svc NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   16m12:41
kata-irc-bot<calm.wu> telnet 10.96.0.1 443 is ok12:42
kata-irc-bot<graham.whaley> ooh, k8s network stuff @calm.wu :slightly_smiling_face: - not really my thing - hopefully when maybe @archana.m.shinde comes online she may have some thoughts. In the meantime, any details on what you ran (pod yaml), and how your k8s stack network is set up (cni, calico, flannel etc.) would probably be helpful. Maybe best to put those details on a github Issue. slack is good for quick short responses, but if we need to collect yaml12:44
kata-irc-botand config data to be able to reproduce, then a github Issue is better.12:44
kata-irc-bot<calm.wu> Ok thanks! @graham.whaley12:46
*** devimc has joined #kata-dev13:22
kata-irc-bot<graham.whaley> @calm.wu - I think you just closed your tomcat github issue '421' - did you mean to do that? :slightly_smiling_face:13:30
kata-irc-botAction: graham.whaley sees re-opened again now ;-)13:31
kata-irc-bot<graham.whaley> hi @thierry @salvador.fuentes - pushed a gerrit to zuul to try and fix our (my) missing golang issue - let's see what you think: https://review.openstack.org/#/c/650313/15:09
kata-irc-bot<graham.whaley> I took the 'find first' inspiration from the original old zuul instance way for kata. I hope the sub-dir file search/include works in my playbook subdir..15:10
kata-irc-bot<thierry> I'll have a look, though I'm not really an Ansible expert15:13
kata-irc-bot<graham.whaley> heh, me neither @thierry :slightly_smiling_face: - the hardest bit is I can't really try it out locally before submitting to zuul, which might not run it either.... so, we may have to 'bat in the dark' a little bit. Reading how it worked on the old setup, it's obvious what it is trying to do, and feels right. now it is down to if I got no typos and if zuul is happy with it...15:16
davidgilukany nemu people know what the state of the v4.0rc0 branch is - I get an acpi related seg when I try it15:21
*** dklyle has joined #kata-dev15:24
kata-irc-bot<sebastien.boeuf> davidgiluk: It's WIP but not there yet. Yang has a PR for this: https://github.com/intel/nemu/pull/21015:30
davidgiluksebastien: Ah thanks15:33
*** sgarzare has quit IRC15:44
kata-irc-bot<manohar.r.castelino> @calm.wu can you check is core-dns can reach the upstream dns server.. typically sitting behind a firewall you cannot16:06
kata-irc-bot<manohar.r.castelino> so I normally point it to a DNS container can reach16:07
kata-irc-bot<manohar.r.castelino> @calm.wu what I do is16:07
kata-irc-bot<manohar.r.castelino> Apply https://github.com/mcastelino/testapi/blob/master/k8s/customize-coredns.yaml16:07
kata-irc-bot<manohar.r.castelino> where FIXME_DNS_IP needs to be replaced by a DNS you can reach16:08
*** devimc has quit IRC16:24
*** gwhaley has quit IRC16:50
*** dklyle has quit IRC16:56
*** dklyle has joined #kata-dev16:56
*** igordc has joined #kata-dev17:28
gmmahastefanha: the plan is to have a common LVM library that both containerd and crio can use.17:46
gmmahafirst step is to get this and the LVM snapshotter that peter hunt has for cri-o landed in the respective project.17:47
gmmahathen https://github.com/haircommander/lvm-go/ be the common library handler for both ofthem17:47
gmmahaall images are thinly provisioned and if there is a parent, we create a snapshot off it to get the next layer.17:48
*** devimc has joined #kata-dev17:59
kata-irc-bot<sebastien.boeuf> @greg.bock hey so I have spent some time setting up Kata with virtio-fs, and I have been able to run into the same issue you got, but I reproduced with `docker run ... ubuntu`. When running `apt update` it hangs after a few seconds because the virtiofs daemon fails (I'm not sure I get the same panic as you). Now, the interesting point is that it actually works if I run the daemon with the cache option `always`. Something is buggy18:07
kata-irc-botregarding the virtqueues, but when most of the data are simply mmap18:07
kata-irc-bot<sebastien.boeuf> mmaped, everything works18:07
kata-irc-bot<sebastien.boeuf> davidgiluk: stefanha: ^^18:07
kata-irc-bot<sebastien.boeuf> we have a problem with `cache=none` here18:08
stefanhasebastien.boeuf: Thanks, it's a known issue we recently debugged18:26
stefanhacache=always will be the default for now18:26
kata-irc-bot<greg.bock> thanks for the update18:31
kata-irc-bot<greg.bock> I'll verify on my setup here in a few18:32
kata-irc-bot<greg.bock> Should I keep the patch you gave me yesterday in? Does it matter either way?18:36
kata-irc-bot<greg.bock>18:40
kata-irc-bot<greg.bock> got much further than before18:40
kata-irc-bot<greg.bock> odd18:41
kata-irc-bot<greg.bock> ```[root@78821e7f25dc /]# rm -f /var/lib/yum/yumdb/n/77651d1cdf0adc19dafdecb490426f20e15f8554-nss-pem-1.0.3-5.el7_6.1-x86_64/reason.tmp  rm: cannot remove '/var/lib/yum/yumdb/n/77651d1cdf0adc19dafdecb490426f20e15f8554-nss-pem-1.0.3-5.el7_6.1-x86_64/reason.tmp': Input/output error [root@78821e7f25dc /]# rm -rf /var/lib/yum/yumdb/ ```18:41
kata-irc-bot<greg.bock>18:55
kata-irc-bot<greg.bock> walked up the dir path, had to go all the way in18:58
kata-irc-bot<greg.bock> if i delete another file in the dir then try from outside the dir it also seems to work18:58
kata-irc-bot<sebastien.boeuf> stefanha: it looks like it happens when we're trying to pass too much data through the virtqueues18:59
kata-irc-bot<sebastien.boeuf> stefanha: and for some reason the buffer cannot be accessed because it's not part of the shared guest RAM19:00
kata-irc-bot<sebastien.boeuf> stefanha: do you have more pointers on this, I'd like to understand what's going on :slightly_smiling_face:19:00
davidgilukah that one19:01
davidgiluksebastien.boeuf: Yeh that's when the kernel tries to get the fuse daemon to read directly into the DAX mapping19:02
kata-irc-bot<sebastien.boeuf> oh really?19:02
kata-irc-bot<greg.bock> sometimes is just works after it seems19:02
kata-irc-bot<greg.bock>19:02
davidgilukyeh I know Stefan and Vivek have a fix for that - it's a fun corner case19:02
davidgilukI remember it was also in rpm19:03
kata-irc-bot<sebastien.boeuf> davidgiluk: any pointer to the fix? becausewe have one patch from Vivek already and I wonder if that's the one you're talking about19:03
davidgilukhmm let me see, I think it was a kernel fix19:03
kata-irc-bot<sebastien.boeuf> davidgiluk: this one? ``` diff --git a/fs/fuse/file.c b/fs/fuse/file.c index 4cb8c8a8011c..cde18dad46c9 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -206,8 +206,9 @@ static struct fuse_dax_mapping *alloc_dax_mapping(struct fuse_conn *fc)   out_kick:         /* If number of free ranges are below threshold, start reclaim */ -       free_threshold = (fc->nr_ranges * FUSE_DAX_RECLAIM_THRESHOLD)/100; -       if (free_threshold19:04
kata-irc-bot> 0 && fc->nr_free_ranges < free_threshold) { +       free_threshold = max((fc->nr_ranges * FUSE_DAX_RECLAIM_THRESHOLD)/100, +                               (unsigned long)1); +       if (fc->nr_free_ranges < free_threshold) {                 pr_debug("fuse: Kicking dax memory reclaim worker. nr_free_ranges=0x%ld nr_total_ranges=%ld\n", fc->nr_free_ranges, fc->nr_ranges);                 queue_delayed_work(system_long_wq, &fc->dax_free_work, 0);19:04
kata-irc-bot      } ```19:04
kata-irc-bot<sebastien.boeuf> gre19:04
*** neonsea has joined #kata-dev19:04
kata-irc-bot<greg.bock> ```[root@localhost linux]# git diff diff --git a/fs/fuse/file.c b/fs/fuse/file.c index 4cb8c8a8011c..cde18dad46c9 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -206,8 +206,9 @@ static struct fuse_dax_mapping *alloc_dax_mapping(struct fuse_conn *fc)    out_kick:         /* If number of free ranges are below threshold, start reclaim */ -       free_threshold = (fc->nr_ranges * FUSE_DAX_RECLAIM_THRESHOLD)/100; -       if19:04
kata-irc-bot(free_threshold > 0 && fc->nr_free_ranges < free_threshold) { +       free_threshold = max((fc->nr_ranges * FUSE_DAX_RECLAIM_THRESHOLD)/100, +                               (unsigned long)1); +       if (fc->nr_free_ranges < free_threshold) {                 pr_debug("fuse: Kicking dax memory reclaim worker. nr_free_ranges=0x%ld nr_total_ranges=%ld\n", fc->nr_free_ranges, fc->nr_ranges);                 queue_delayed_work(system_long_wq,19:04
kata-irc-bot&fc->dax_free_work, 0);         } ```19:04
kata-irc-bot<sebastien.boeuf> @greg.bock weird behavior19:04
davidgilukdoesn't sound like it19:04
kata-irc-bot<sebastien.boeuf> @greg.bock let's see if davidgiluk has a magic patch for us :slightly_smiling_face:19:05
kata-irc-bot<greg.bock> oh, i thought they posted the one above mine, i'm blind19:05
neonseahi! I'm using Kata for my uni dissertation/final project, studying infosec. First off - brilliant work, Kata works like a charm and fits into my project perfectly19:05
davidgiluksebastien: I've prodded vgoyal - although I'm going soon so perhaps he might mail it you if I'm out19:06
neonseadoes anyone know whether it's possible to change the value returned by uname without recompiling the kernel? Currently, it's "Linux [...].container", which.. isn't necessarily a problem by itself, but I'd rather not it be there. Any ideas?19:07
kata-irc-bot<sebastien.boeuf> davidgiluk: yep that's fine he could email to me or to the kata-dev ML19:07
davidgiluksebastien: he says he doesn't currently have one, but that cache=always doesn't hit it19:08
kata-irc-bot<greg.bock> im running cache=always19:08
kata-irc-bot<greg.bock>19:09
kata-irc-bot<greg.bock> I should probably also retest on 1.4, still not sure I ported everything to 1.5 right ;)19:09
kata-irc-bot<sebastien.boeuf> neonsea: I'd say use `sed` to get only what you want from uname output19:09
kata-irc-bot<sebastien.boeuf> @greg.bock hehe :slightly_smiling_face:19:09
davidgilukgreg: OK, I'll let him know19:10
kata-irc-bot<sebastien.boeuf> davidgiluk: also you could convince him to join slack or IRC :slightly_smiling_face:19:10
davidgiluksebastien: Now that's getting harder....19:10
neonsea@sebastien.boeuf would work, but it's not for me - it's for for fooling someone else19:11
kata-irc-bot<sebastien.boeuf> hehe :)19:11
neonseai can't go into too much detail unfortunately, but it's to do with honeypots19:11
kata-irc-bot<sebastien.boeuf> gotta run for now folks19:12
kata-irc-bot<greg.bock> thanks @sebastien.boeuf ~!19:12
neonseafarewell19:12
kata-irc-bot<greg.bock> @neosea https://github.com/kata-containers/packaging/tree/master/kernel19:16
neonseasweet, cheers19:17
kata-irc-bot<greg.bock> rebuilding the kernel isn't too bad iirc19:30
kata-irc-bot<greg.bock> been a while since i rebuilt with the packaging tools19:30
*** davidgiluk has quit IRC19:36
neonsea@greg.bock it's not an issue at all! and it's perhaps the cleanest way of doing it anyways19:41
*** igordc has quit IRC19:44
*** igordc has joined #kata-dev19:46
*** sameo has quit IRC20:09
*** tmhoang has quit IRC20:29
neonseaThe `arch` command didn't work on my (ironically) Arch system, so I replaced it all with `uname -m`21:03
neonseaPR as well https://github.com/kata-containers/packaging/pull/42421:03
*** neonsea has quit IRC21:22
*** brtknr has quit IRC22:01
*** bandini has quit IRC22:01
*** devimc has quit IRC22:01
*** kgz has quit IRC22:06
*** kgz has joined #kata-dev22:08
*** auk has joined #kata-dev23:58
« Thursday, 2019-04-04 Index Saturday, 2019-04-06 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!