| *** LinuxMe has joined #kata-dev | 00:04 | |
| *** LinuxMe has quit IRC | 00:10 | |
| *** annabelleB has joined #kata-dev | 00:35 | |
| *** annabelleB has quit IRC | 00:37 | |
| *** LinuxMe has joined #kata-dev | 01:05 | |
| *** zerocoolback has joined #kata-dev | 01:09 | |
| *** LinuxMe has quit IRC | 01:11 | |
| *** annabelleB has joined #kata-dev | 02:22 | |
| *** annabelleB has quit IRC | 02:31 | |
| *** changcheng has joined #kata-dev | 03:14 | |
| *** changcheng has quit IRC | 03:16 | |
| *** LinuxMe has joined #kata-dev | 03:23 | |
| *** LinuxMe has quit IRC | 03:28 | |
| kata-irc-bot | <eric.ernst> @xu how’d the meetup go? | 04:02 |
|---|---|---|
| kata-irc-bot | <eric.ernst> @james.o.hunt - wanted to see how the agent tracing work is going. I was helping with some debug and thinking how helpful it’d be. Or at least fun, anyway. :) | 04:15 |
| kata-irc-bot | <xu> not bad, will summary it later today | 05:19 |
| *** annabelleB has joined #kata-dev | 05:48 | |
| *** LinuxMe has joined #kata-dev | 06:05 | |
| *** annabelleB has quit IRC | 06:08 | |
| *** LinuxMe has quit IRC | 06:10 | |
| *** marcov has quit IRC | 06:39 | |
| *** marcov has joined #kata-dev | 06:39 | |
| *** lpetrut has joined #kata-dev | 07:09 | |
| *** LinuxMe has joined #kata-dev | 07:53 | |
| *** gwhaley has joined #kata-dev | 07:56 | |
| *** LinuxMe has quit IRC | 07:58 | |
| *** davidgiluk has joined #kata-dev | 08:03 | |
| *** jodh has joined #kata-dev | 08:15 | |
| * gwhaley throws a mebibyte @marcov ;-) | 08:53 | |
| marcov | @gwhaley mebibyte, aka the old trick used by hd manufactures to rip you off )) | 09:01 |
| davidgiluk | is there something to forcibly clean up after crashed containers? Given the proddings I'm doing with qemu I can leave things in quite a mess | 09:46 |
| gwhaley | hi davidgiluk. not specifically in kata, no. I tend to use a combination of pgrep/pkill to clear out any kata-* (proxy and shim) and qemu* . We probably accumulate cruft down in the /var work dirs though. Yeah, crashing out containers can leave a little mess around. | 09:49 |
| * gwhaley sees 'k a t a' turn up as 'j e s s f r a z' over on slack. meh, tired of chasing that - but will note to @openstack in case they want to go fix again... | 09:51 | |
| davidgiluk | haha | 09:51 |
| davidgiluk | perhaps it would be easier to rename the project :-) | 09:51 |
| gwhaley | or rename jess :-) | 09:51 |
| gwhaley | apparently it's an issue with the 'mappings' - I think if something fails/falls out then it ends up either taking the first or the next answer - which ends up mapping k-a-t-a to jess - ho hum. | 09:52 |
| *** LinuxMe has joined #kata-dev | 09:54 | |
| davidgiluk | gwhaley: I get things like k-runtime list hanging after I've nuked a qemu | 09:57 |
| davidgiluk | or at least taking a long time | 09:57 |
| davidgiluk | and then lieing to me telling me it's still running | 09:57 |
| gwhaley | ah, that's not a good thing. I've chased a few of those down before. In the past we could end up with something stuck holding a lock down in the /var/vc area. or... | 09:58 |
| gwhaley | maybe you still have the shim/proxy pair for that container running, so the runtime is asking them to ask the qemu, and they are sat and then time out... | 09:58 |
| gwhaley | yes, there is work to do on capturing bad death/shutdowns. | 09:58 |
| gwhaley | I have been killing proxy's this week, and that takes the shim with it, but leaves the qemu around I think | 09:59 |
| *** LinuxMe has quit IRC | 09:59 | |
| gwhaley | davidgiluk - have you tried a 'docker kill' on the conatiner (even though your qemu has gone away), maybe that will hit the proxy/shim pair, and nuke them (after potential timeout), and clear up the docker side as well | 10:00 |
| davidgiluk | oh I've already nuked the proxy | 10:01 |
| gwhaley | I wonder where the runtime list is hanging up then. If you are curious, you could enable the runtime debug in the config file and capture what it tells you in the journal logs - and then post us a github Issue ;-) | 10:02 |
| davidgiluk | oh, I hadn't realised that even 'list' would follow that debug; in that case I can chase it | 10:03 |
| gwhaley | I think it will - list will go walk the /var/lib/vc/sbs dirs gathering info. I'm presuming that will get some debug out of virtcontainers... If you get a solid hang up, then you can 'dlv' debug the hung up runtime and get the stacks of the goroutines. That is where I used to find we deadlocked on a lockfile in that area - but, sboeuf rewrote all that code (much nice/faster now), and I've not seen one of them for some time | 10:05 |
| davidgiluk | in the past I have recovered it by nuking the sbs dir | 10:06 |
| gwhaley | oh, you might want to check how many subdirs you have down in that dir btw - I think things might leak under some circumstances :-( for instance, my test box has 0 containers running, and I have 221 dirs there :-( | 10:06 |
| gwhaley | and I suspect 'list' walks them all, even if they are not 'active' | 10:07 |
| davidgiluk | I also have to forcibly clean up the overlays when that happens, otherwise I wonder where the free space went | 10:08 |
| gwhaley | yeah. docker eats space... it collects up images, volumes and stuff.... I had to 'docker system prune -a --volumes' yesterday to grab 5G back... and that is not a k a t a thing - that is just docker :-) | 10:09 |
| gwhaley | jodh - did you just cherry pick the same PR 3 times? I thought we only ever had to do 2 picks, as we only have one master and two stables - what did I miss? ;-) | 10:14 |
| jodh | gwhaley: we have 3 stable branches now, but maybe we need to discuss that ;) | 10:14 |
| jodh | gwhaley: also, it wasn't a simple cherry pick - conflict resolution was required ;) | 10:15 |
| gwhaley | and, you hit a cyclo complexity on one of them already ;-) | 10:15 |
| jodh | gwhaley: I'm a bit confused though - is there a problem with raising PRs for the stable branches? | 10:18 |
| gwhaley | jodh - nope. I just wondered why you had done 3 of them when we are only meant to have 2 maintained stable branches at any one time. I suspect we have not deprecated the oldest one when we were meant to. | 10:23 |
| kata-irc-bot | <james.o.hunt> @eric.ernst - we have 3 stable branches now, but we document we only maintain 2: https://github.com/kata-containers/documentation/blob/master/Stable-Branch-Strategy.md#branch-management | 10:25 |
| kata-irc-bot | <james.o.hunt> @eric.ernst - regarding agent tracing, nothing significant to add since last update - trying to track down why the trace transaction is not completing. | 10:27 |
| *** gwhaley has quit IRC | 10:59 | |
| *** devimc has joined #kata-dev | 11:57 | |
| *** gwhaley has joined #kata-dev | 12:15 | |
| *** dims has quit IRC | 12:30 | |
| *** lpetrut has quit IRC | 12:33 | |
| *** dims has joined #kata-dev | 12:34 | |
| *** zerocoolback has quit IRC | 13:22 | |
| *** sameo has joined #kata-dev | 13:38 | |
| *** annabelleB has joined #kata-dev | 14:34 | |
| *** annabelleB has quit IRC | 14:40 | |
| *** dklyle has joined #kata-dev | 15:06 | |
| *** annabelleB has joined #kata-dev | 15:10 | |
| *** fuentess_ has joined #kata-dev | 15:14 | |
| *** annabelleB has quit IRC | 15:27 | |
| *** devananda has joined #kata-dev | 15:30 | |
| *** annabelleB has joined #kata-dev | 15:33 | |
| *** sameo has quit IRC | 16:05 | |
| *** annabelleB has quit IRC | 16:12 | |
| *** dklyle has quit IRC | 16:16 | |
| *** dklyle has joined #kata-dev | 16:16 | |
| *** LinuxMe has joined #kata-dev | 16:33 | |
| *** dklyle has quit IRC | 16:37 | |
| *** annabelleB has joined #kata-dev | 16:39 | |
| *** annabelleB has quit IRC | 16:43 | |
| *** annabelleB has joined #kata-dev | 16:52 | |
| *** gwhaley has quit IRC | 16:58 | |
| *** jodh has quit IRC | 17:14 | |
| *** dklyle has joined #kata-dev | 17:30 | |
| *** dklyle has quit IRC | 17:42 | |
| *** dklyle has joined #kata-dev | 17:50 | |
| *** sjas has joined #kata-dev | 17:56 | |
| *** dklyle has quit IRC | 18:02 | |
| *** annabelleB has quit IRC | 18:07 | |
| *** dklyle has joined #kata-dev | 18:20 | |
| *** annabelleB has joined #kata-dev | 18:25 | |
| *** dklyle has quit IRC | 18:29 | |
| *** annabelleB has quit IRC | 18:32 | |
| *** annabelleB has joined #kata-dev | 18:34 | |
| *** dklyle has joined #kata-dev | 19:07 | |
| *** dklyle has quit IRC | 19:14 | |
| *** davidgiluk has quit IRC | 19:30 | |
| *** annabelleB has quit IRC | 19:32 | |
| *** devananda has quit IRC | 19:47 | |
| *** annabelleB has joined #kata-dev | 20:54 | |
| *** dklyle has joined #kata-dev | 21:36 | |
| *** devimc has quit IRC | 21:37 | |
| *** dklyle has quit IRC | 21:45 | |
| *** LinuxMe has quit IRC | 22:47 | |
| *** LinuxMe has joined #kata-dev | 23:23 | |
| *** LinuxMe has quit IRC | 23:27 | |
| *** annabelleB has quit IRC | 23:31 | |
| *** LinuxMe has joined #kata-dev | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!