| *** devimc has quit IRC | 00:36 | |
| *** annabelleB has joined #kata-dev | 00:37 | |
| *** annabelleB has quit IRC | 00:47 | |
| *** eernst has joined #kata-dev | 01:09 | |
| *** eernst has quit IRC | 01:12 | |
| *** eernst has joined #kata-dev | 01:12 | |
| *** dlw has joined #kata-dev | 01:28 | |
| *** LinuxMe has joined #kata-dev | 01:29 | |
| *** LinuxMe has quit IRC | 01:33 | |
| *** eernst has quit IRC | 01:42 | |
| kata-irc-bot | <mnaser> so it looks like clarkb has gotten the job as far as prety much 'operating' the same way as kata-proxy in jenkins (which is failing) | 01:50 |
|---|---|---|
| kata-irc-bot | <mnaser> does anyone want to volunteer to have a look at why it's failing (cc @salvador.fuentes @sebastien.boeuf) | 01:50 |
| *** eernst has joined #kata-dev | 01:53 | |
| *** eernst has quit IRC | 02:10 | |
| *** eernst has joined #kata-dev | 02:11 | |
| *** david-lyle has joined #kata-dev | 03:00 | |
| *** david-lyle has quit IRC | 03:09 | |
| kata-irc-bot | <eric.ernst> @verytired1 - it looks like someone does have katacontainers for dockerhub. Wondering how I can get access to setup some container images... | 03:35 |
| kata-irc-bot | <verytired1> Yeah...I haven't yet figured that out. Still working on it | 03:37 |
| kata-irc-bot | <verytired1> I thought we'd set it up, but none of the info I've tried seemed to work | 03:38 |
| kata-irc-bot | <eric.ernst> Yeah. Okay, thanks for the update. | 03:39 |
| kata-irc-bot | <eric.ernst> i see both kata and katacontainers organizations are taken. So is wtf, in my thorough testing. | 03:40 |
| *** eernst has quit IRC | 03:47 | |
| *** eernst has joined #kata-dev | 03:48 | |
| kata-irc-bot | <verytired1> Haha | 03:48 |
| *** zerocoolback has joined #kata-dev | 03:55 | |
| *** eernst has quit IRC | 04:03 | |
| *** eernst has joined #kata-dev | 04:04 | |
| kata-irc-bot | <sebastien.boeuf> @mnaser sure, @salvador.fuentes and I can take a look tomorrow. Where is the latest log you got ? | 04:04 |
| kata-irc-bot | <mnaser> @sebastien.boeuf right now the latest zuul log is the same as the one in jenkins so you can find the Kata-proxy 16.04 job there | 04:05 |
| kata-irc-bot | <sebastien.boeuf> ok | 04:06 |
| kata-irc-bot | <sebastien.boeuf> @mnaser looking at https://github.com/kata-containers/proxy/pull/74, and the log from 16.04 is basically a glitch in the CI. There is no reason for this test to fail, given the PR itself. If we re-run it, it should pass, and this should have the same effect on zuul | 04:10 |
| kata-irc-bot | <sebastien.boeuf> @mnaser any chance you can retrigger the one on zuul ? | 04:11 |
| kata-irc-bot | <mnaser> @sebastien.boeuf correct but it’s also failing in Jenkins too | 04:11 |
| kata-irc-bot | <sebastien.boeuf> Yeah I guess we have a race condition in the code/tests and this should be fixed, but it's not consistent since most of the CI passes on other PRs | 04:12 |
| kata-irc-bot | <mnaser> http://jenkins.katacontainers.io/job/kata-containers-proxy-ubuntu-16-04-PR/13/console | 04:12 |
| kata-irc-bot | <mnaser> So just recheck / rebuild ? | 04:13 |
| kata-irc-bot | <mnaser> clarkb: what’s the best way to retrigger.. can we type recheck in the PR? | 04:14 |
| clarkb | mnaser I'm nit sure recheck will work until we update the perms on the kata repo foe the app. pushing a new commit should work | 04:15 |
| kata-irc-bot | <sebastien.boeuf> clarkb: there is no way to simply trigger a re-build ? | 04:16 |
| *** sjas_ has joined #kata-dev | 04:16 | |
| clarkb | there is, but we have to update the permissions that kata allows the zuul app to have | 04:17 |
| kata-irc-bot | <sebastien.boeuf> I can do that | 04:17 |
| clarkb | (has to be able to see issue comments because PR comments are issues under the hood or something) | 04:17 |
| clarkb | the zuul app config is set up properly I think you just have to refresh them and accept them in the proxy repo | 04:17 |
| clarkb | mnaser if that is done I think you can just recheck comment | 04:18 |
| kata-irc-bot | <mnaser> Cool, or @sebastien.boeuf can do it himself to track progress too :) | 04:18 |
| *** sjas has quit IRC | 04:19 | |
| kata-irc-bot | <sebastien.boeuf> all permissions are already granted to the zuul app in github | 04:19 |
| kata-irc-bot | <sebastien.boeuf> this should work | 04:19 |
| kata-irc-bot | <sebastien.boeuf> clarkb: so I just need to comment "/recheck" | 04:21 |
| kata-irc-bot | <sebastien.boeuf> ? | 04:21 |
| clarkb | just 'recheck' no / | 04:21 |
| kata-irc-bot | <sebastien.boeuf> ok | 04:21 |
| kata-irc-bot | <sebastien.boeuf> can we grant some permissions to specific users to be able to rebuild ? | 04:21 |
| kata-irc-bot | <sebastien.boeuf> I mean "recheck" | 04:22 |
| clarkb | no, currently it is global | 04:22 |
| kata-irc-bot | <sebastien.boeuf> ok | 04:22 |
| clarkb | (anyone can do it) | 04:22 |
| kata-irc-bot | <sebastien.boeuf> clarkb: any way I can check if the recheck took effect ? | 04:23 |
| kata-irc-bot | <sebastien.boeuf> https://github.com/kata-containers/proxy/pull/74#issuecomment-398269817 | 04:23 |
| clarkb | it shoukd update the status on the PR itself. Also you can look under third-party-check at https://zuul.openstack.org | 04:24 |
| clarkb | I dont see it there yet. Was ability to see issue comments granted to the app on that repo? | 04:24 |
| clarkb | I can debug this tomorrow better. Late here now | 04:27 |
| clarkb | but likely a permissions issue | 04:27 |
| kata-irc-bot | <sebastien.boeuf> github proxy repo -> settings -> integration & services ? | 04:27 |
| kata-irc-bot | <sebastien.boeuf> okay, let's see that tomorrow ;) | 04:28 |
| clarkb | https://docs.openstack.org/infra/system-config/zuulv3.html#github-projects | 04:28 |
| clarkb | maybe try reinstall it using those steps | 04:29 |
| *** eernst has quit IRC | 05:41 | |
| *** eernst has joined #kata-dev | 05:41 | |
| kata-irc-bot | <eric.ernst> https://github.com/kata-containers/documentation/blob/master/Developer-Guide.md#set-up-a-debug-console | 06:15 |
| *** sameo has joined #kata-dev | 06:27 | |
| *** fiddletwix has quit IRC | 06:28 | |
| *** fiddletwix has joined #kata-dev | 06:28 | |
| *** sameo has quit IRC | 06:41 | |
| *** jodh has joined #kata-dev | 06:44 | |
| *** jodh has quit IRC | 06:44 | |
| *** jodh has joined #kata-dev | 06:44 | |
| *** sameo has joined #kata-dev | 07:23 | |
| *** gwhaley has joined #kata-dev | 08:00 | |
| *** davidgiluk has joined #kata-dev | 08:03 | |
| *** eernst has quit IRC | 08:26 | |
| *** zerocoolback has quit IRC | 08:33 | |
| *** zerocoolback has joined #kata-dev | 08:35 | |
| *** zerocoolback has quit IRC | 08:47 | |
| *** zerocoolback has joined #kata-dev | 08:48 | |
| *** zerocoolback has quit IRC | 08:52 | |
| *** zerocoolback has joined #kata-dev | 08:54 | |
| *** zerocoolback has quit IRC | 08:55 | |
| *** zerocoolback has joined #kata-dev | 09:13 | |
| *** zerocoolback has quit IRC | 09:18 | |
| *** zerocoolback has joined #kata-dev | 09:19 | |
| *** zerocoolback has quit IRC | 09:23 | |
| *** dlw1 has joined #kata-dev | 09:41 | |
| *** dlw has quit IRC | 09:41 | |
| *** dlw1 is now known as dlw | 09:41 | |
| *** zerocoolback has joined #kata-dev | 09:41 | |
| *** zerocoolback has quit IRC | 09:49 | |
| *** zerocoolback has joined #kata-dev | 09:49 | |
| *** zerocoolback has quit IRC | 09:54 | |
| *** eernst has joined #kata-dev | 10:03 | |
| *** zerocoolback has joined #kata-dev | 10:21 | |
| *** eernst has quit IRC | 10:34 | |
| *** dlw has quit IRC | 10:51 | |
| *** sameo has quit IRC | 10:52 | |
| *** gwhaley has quit IRC | 10:59 | |
| kata-irc-bot | <eric.ernst> @zhangwei555 @bergwolf @xu - was thinking we should do a 1.0.1 for some recent bug fixes. Thoughts? Objections? | 11:02 |
| kata-irc-bot | <bergwolf> +1, v1.0.1 sounds good to me | 11:03 |
| kata-irc-bot | <eric.ernst> We have a fix for a lame failure (run container with command foo will leave qemu process on host) which is probably best to fix in packages now, and I want to make sure releaSe process doesn’t get too rusty. | 11:05 |
| kata-irc-bot | <zhangwei555> 1.0.1 works for me | 11:12 |
| *** gwhaley has joined #kata-dev | 12:06 | |
| *** dlw has joined #kata-dev | 12:46 | |
| *** sameo has joined #kata-dev | 12:48 | |
| *** devimc has joined #kata-dev | 13:03 | |
| *** LinuxMe has joined #kata-dev | 13:07 | |
| *** lamego has joined #kata-dev | 13:31 | |
| *** annabelleB has joined #kata-dev | 13:55 | |
| *** dlw has quit IRC | 14:21 | |
| *** annabelleB has quit IRC | 14:25 | |
| kata-irc-bot | <xu> +1 agree | 14:41 |
| *** annabelleB has joined #kata-dev | 14:43 | |
| *** annabelleB has quit IRC | 14:53 | |
| *** annabelleB has joined #kata-dev | 14:53 | |
| *** jodh has quit IRC | 15:10 | |
| *** manjeets has quit IRC | 15:35 | |
| *** dklyle has joined #kata-dev | 15:41 | |
| *** fiddletwix has quit IRC | 15:41 | |
| *** dklyle has quit IRC | 15:46 | |
| *** fuentess has joined #kata-dev | 15:58 | |
| *** dklyle has joined #kata-dev | 16:10 | |
| *** zerocoolback has quit IRC | 16:13 | |
| kata-irc-bot | <salvador.fuentes> @mnaser hi, here is the script that can reproduce the kernel panic issue on the ubuntu 17.10 machines: https://gist.github.com/chavafg/d00fa4dbefb144e6b7eeceb5e1ad9c65 | 16:18 |
| kata-irc-bot | <sebastien.boeuf> clarkb: @mnaser @salvador.fuentes hey guys, would you be available for a meeting about the status of Zuul CI, and if you need anything from us ? | 16:40 |
| clarkb | I can be around, when were you thinking? | 16:40 |
| kata-irc-bot | <sebastien.boeuf> clarkb: @mnaser what about 1.30 to 2.00 PM today ? | 16:48 |
| clarkb | what timezone is that? | 16:48 |
| kata-irc-bot | <sebastien.boeuf> oh sorry ! PST | 16:48 |
| clarkb | ya I can do both of those times pacific time | 16:48 |
| kata-irc-bot | <sebastien.boeuf> cool | 16:49 |
| kata-irc-bot | <sebastien.boeuf> clarkb: @mnaser @salvador.fuentes: here is my zoom meeting: https://zoom.us/j/398799714 | 16:54 |
| kata-irc-bot | <sebastien.boeuf> 1.30PM PST | 16:54 |
| kata-irc-bot | <salvador.fuentes> thanks | 16:54 |
| kata-irc-bot | <sebastien.boeuf> np | 16:55 |
| kata-irc-bot | <mnaser> cool, i can be there too | 17:10 |
| kata-irc-bot | <sebastien.boeuf> cool @mnaser ! | 17:12 |
| *** annabelleB has quit IRC | 17:18 | |
| kata-irc-bot | <salvador.fuentes> @sebastien.boeuf clarkb, I think the recheck button is working | 17:19 |
| kata-irc-bot | <sebastien.boeuf> oh nice | 17:19 |
| kata-irc-bot | <salvador.fuentes> I've just triggered a job and see activiy at: https://zuul.openstack.org/ (searching for 'kata') | 17:19 |
| kata-irc-bot | <sebastien.boeuf> let's see how it goes | 17:21 |
| kata-irc-bot | <sebastien.boeuf> and this actually triggers builds on vexxhost VMs, right ? | 17:22 |
| kata-irc-bot | <salvador.fuentes> yes, I think so | 17:23 |
| *** annabelleB has joined #kata-dev | 17:24 | |
| kata-irc-bot | <sebastien.boeuf> ok | 17:29 |
| clarkb | yup on the same vexxhost flavor | 17:29 |
| clarkb | different images though as we build our own | 17:29 |
| kata-irc-bot | <sebastien.boeuf> clarkb: sounds good. Did Tristan make some progress on the Azure plugin? | 17:32 |
| clarkb | I think it is pushed and ready for testing if there is an account to use | 17:32 |
| kata-irc-bot | <sebastien.boeuf> oh nice! FWIW, we had some issues with the previous Azure Pass because it expired... But we have another one and we can discuss the details during the meeting | 17:34 |
| clarkb | ok | 17:39 |
| clarkb | https://etherpad.openstack.org/p/kata-on-zuul-notes I've been tryingto keep notes there fwiw | 17:40 |
| *** dklyle has quit IRC | 18:10 | |
| *** LinuxMe has quit IRC | 18:16 | |
| *** LinuxMe_ has joined #kata-dev | 18:18 | |
| *** dklyle has joined #kata-dev | 18:32 | |
| *** annabelleB has quit IRC | 18:32 | |
| *** annabelleB has joined #kata-dev | 18:35 | |
| *** davidgiluk has quit IRC | 19:04 | |
| *** gwhaley has quit IRC | 19:28 | |
| *** LinuxMe_ has quit IRC | 19:45 | |
| *** anteaya has joined #kata-dev | 19:46 | |
| *** anteaya has quit IRC | 19:49 | |
| *** annabelleB has quit IRC | 19:51 | |
| *** annabelleB has joined #kata-dev | 19:56 | |
| *** dklyle has quit IRC | 20:00 | |
| *** anteaya has joined #kata-dev | 20:02 | |
| *** LinuxMe has joined #kata-dev | 20:04 | |
| *** sameo has quit IRC | 20:20 | |
| *** anteaya has quit IRC | 20:50 | |
| *** anteaya has joined #kata-dev | 20:51 | |
| clarkb | http://logs.openstack.org/52/576052/2/check/tempest-full/9a399f2/controller/logs/screen-n-sch.txt.gz | 20:53 |
| clarkb | http://logstash.openstack.org/#/dashboard/file/logstash.json?query=(message:%20%5C%22FAILED%20with%20status:%20137%5C%22%20OR%20message:%20%5C%22FAILED%20with%20status:%20143%5C%22)%20AND%20tags:%20%5C%22console%5C%22%20AND%20voting:1&from=864000s | 20:55 |
| clarkb | http://logs.openstack.org/74/74/9e1f7eede869376225a5f4b35c8f8efdfabb7426/third-party-check/kata-runsh/7bef64d/ara-report/ | 20:57 |
| clarkb | some examples of how we do job log reporting and tracking | 20:57 |
| clarkb | salvador.fuentes: probably the first thing we want to look at is grabbing crio logs properly. Since that is what is failing and log collection for that isn't working properly | 21:00 |
| clarkb | salvador.fuentes: https://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/kata-runsh/post.yaml#n54 is where I attempt to collect those logs. This is a normal ansible playbook. I'm happy to push fixes for that if you have ideas or help you push them yourself | 21:02 |
| kata-irc-bot | <salvador.fuentes> ok, sure, let me take a look | 21:02 |
| kata-irc-bot | <salvador.fuentes> clarkb, @sebastien.boeuf I just remember, that crio does not log into the journal when it runs the tests, because the service is down. Each one of the tests starts a cri-o process and the output is sent to the stdout, which is the output we see when the crio tests fail | 21:05 |
| clarkb | aha | 21:05 |
| kata-irc-bot | <sebastien.boeuf> yeah, crio only logs if it fails, otherwise it won't provide any output | 21:06 |
| clarkb | cool I think that is working as expected then | 21:06 |
| kata-irc-bot | <salvador.fuentes> yes, working as expected ;) | 21:07 |
| clarkb | should we remove the crio log collection I posted above? will that ever have data (like during the k8s testing maybe?) | 21:07 |
| clarkb | before we get too far salvador.fuentes the jobs are broken down into three phases, pre-run, run, post-run. The pre-run playbooks are allowed to be retried if they fail so this is where I have put your setup.sh and package installs https://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/kata-runsh/pre.yaml, run is where the actual testing happens and then post runs regardless of | 21:08 |
| clarkb | run's result | 21:08 |
| clarkb | you will see all three playbooks in that directory | 21:08 |
| clarkb | The other piece that will need work is updating that pre.yaml playbook to handle distros that are not ubuntu | 21:09 |
| kata-irc-bot | <salvador.fuentes> clarkb: yes, it will have data when we run k8s, so lets have it there | 21:09 |
| clarkb | ok | 21:09 |
| kata-irc-bot | <salvador.fuentes> clarkb: need to leave for a couple of hours, I'll take a look at the yaml files when I'm back and continue if you are still online, if not we can continue tomorrow | 21:19 |
| clarkb | sounds good. I should be around later today. Feel free to ask questions. I'm taking middle of my day off tomorrow to spend time with kids but should be around in the early morning (I am in the pacific timezone) | 21:20 |
| clarkb | I'm actually nearish to the intel hillsboro campuses if anyone is in the area | 21:20 |
| kata-irc-bot | <sebastien.boeuf> clarkb: really ? oh that's cool, @archana.m.shinde @eric.ernst and myself are working in Jones Farm campus in Hillsboro | 21:23 |
| clarkb | cool. My brother just got a small rowboat and wants to take my kids out ot hagg lake tomorrow. So I get to do that for a few hours :) | 21:24 |
| clarkb | more than happy to meet in the area too if you think it will help | 21:24 |
| kata-irc-bot | <sebastien.boeuf> clarkb: sounds good ;) | 21:25 |
| *** anteaya has quit IRC | 21:29 | |
| *** devimc has quit IRC | 21:40 | |
| *** anteaya has joined #kata-dev | 22:21 | |
| *** lamego has left #kata-dev | 22:39 | |
| *** dklyle has joined #kata-dev | 22:43 | |
| *** david-lyle has joined #kata-dev | 22:45 | |
| *** eernst has joined #kata-dev | 22:59 | |
| *** david-lyle has quit IRC | 23:04 | |
| *** anteaya has quit IRC | 23:09 | |
| *** LinuxMe has quit IRC | 23:10 | |
| *** annabelleB has quit IRC | 23:32 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!