Monday, 2020-09-07

*** ysandeep|away is now known as ysandeep03:53
*** ratailor has joined #oooq04:34
*** skramaja has joined #oooq04:49
*** ykarel has joined #oooq04:56
*** marios has joined #oooq05:32
*** udesale has joined #oooq05:45
*** marios has quit IRC06:41
*** marios has joined #oooq06:45
*** bogdando has joined #oooq06:49
*** jtomasek has joined #oooq06:50
*** jpena|off is now known as jpena07:04
*** amoralej|off is now known as amoralej07:09
*** holser has joined #oooq07:20
*** saneax has joined #oooq07:31
*** owalsh has quit IRC07:48
*** holser has quit IRC07:50
*** holser has joined #oooq07:52
*** owalsh has joined #oooq07:52
arxcruz|ruckakahat|rover: good morning, how are we today? sorry, I'm not feeling well08:23
mariosarxcruz|ruck: sorry to hear it arxcruz|ruck get well soon08:25
*** derekh has joined #oooq08:31
*** apetrich has joined #oooq08:36
*** apetrich has quit IRC08:37
*** apetrich has joined #oooq08:38
*** jaosorior has joined #oooq08:44
akahat|roverarxcruz|ruck, o/08:46
akahat|roverarxcruz|ruck, we are good.. don't worry.. i08:46
akahat|roveri'll take care.. get well soon.08:46
arxcruz|ruckmarios: akahat|rover wow, don't worry, I'm not sick, just not feeling well like low blood pressure08:48
*** ykarel has quit IRC09:10
*** ykarel has joined #oooq09:11
*** ykarel is now known as ykarel|away09:20
*** ykarel|away has quit IRC09:25
ysandeepguys, is it possible to access old taiga board tasks?10:01
bhagyashrismarios, Hi, i am still not able to figure out why stage setup is not reaching to this point : https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/logic.py#L24610:14
bhagyashrismarios, i did stage setup on master as well and put break point there ^ but it's not reaching to that point10:15
mariosbhagyashris: ack... i am also stepping currently with the py tests... lets discuss in more detail on our call later ?10:16
bhagyashrismarios, ok sure10:17
*** tosky has joined #oooq10:33
*** bogdando has quit IRC10:35
*** dtantsur|afk is now known as dtantsur10:38
*** jtomasek has quit IRC10:54
arxcruz|ruckbhagyashris: you can use pudb to debug, it's pretty nice, and you can see graphically all the variables, I recommend10:57
bhagyashrisarxcruz|ruck, yeah i am using pdb10:57
*** ratailor has quit IRC10:57
bhagyashrisarxcruz|ruck, but issue is with code10:58
arxcruz|ruckbhagyashris: not pdb, pudb10:58
bhagyashrisarxcruz|ruck, ahh ok10:58
arxcruz|ruckbhagyashris: https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fpythonspot.com%2Fwp-content%2Fuploads%2F2016%2F08%2Fpython-debugging-pudb.png&f=1&nofb=110:59
bhagyashrisarxcruz|ruck, ack11:00
owalshhey, akahat|rover, arxcruz|ruck: is there an issue with train containers in CI?11:14
owalshhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8f1/749896/1/check/tripleo-ci-centos-8-undercloud-containers/8f10022/logs/undercloud/home/zuul/undercloud_install.log11:14
*** jtomasek has joined #oooq11:14
owalshNot found image: https://registry-1.docker.io/v2/tripleotraincentos8/centos-binary-neutron-server/manifests/c3103a3af637b1ff751b6914c4e24f7011:14
*** udesale_ has joined #oooq11:15
*** udesale has quit IRC11:18
arxcruz|ruckowalsh: let me check11:18
owalshmaybe just that image missing, standalone passed but it's using tripleotraincentos8/centos-binary-neutron-server-ovn11:19
arxcruz|ruckowalsh: I need to check with weshay|ruck we were having some issues with train on centos 811:20
owalsharxcruz|ruck: ack, thanks11:23
*** jtomasek has quit IRC11:25
ysandeepakahat|rover, arxcruz|ruck We faced below issue on 04th in upstream multinode job, We are seeing similiar error in downstream consistently(2/2 tries). In upstream job are green now, Are you aware of any bug(including fix released) around similiar error:-11:26
ysandeephttps://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-common-train/ccb13fb/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz11:26
ysandeep~~~11:26
ysandeep2020-09-04 07:24:26 | Host 192.168.24.16 not found in /home/zuul/.ssh/known_hosts11:26
ysandeep2020-09-04 07:24:26 | Overcloud configuration failed.11:26
ysandeep~~~11:26
arxcruz|ruckowalsh: this seems to happen only in train, and only one time, I'll investigate with weshay|ruck when he's available11:29
*** jpena is now known as jpena|lunch11:30
arxcruz|ruckysandeep: Connection broken: ConnectionResetError11:30
arxcruz|ruckakahat|rover: have you see this happening before?11:31
owalsharxcruz|ruck: https://353602d923747538603d-1a831c5ecd04a8c4708dc7397e88c9d7.ssl.cf5.rackcdn.com/750123/1/check/tripleo-ci-centos-8-containers-multinode/4a2ecbe/logs/undercloud/home/zuul/undercloud_install.log11:32
owalshhttp://tripleo-cockpit.usersys.redhat.com/d/VTiDErfWk/jobs-exploration?orgId=1&var-influxdb_filter=branch%7C%3D%7Cstable%2Ftrain&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-8-containers-multinode11:34
arxcruz|ruckowalsh: oh... it seems zuul don't return the real time jobs...11:35
arxcruz|ruckhttps://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-undercloud-containers11:35
arxcruz|ruckI see only two11:35
arxcruz|ruckah, containers multinode as well11:36
arxcruz|ruckI see11:36
owalshyes, just centos8 containers multinode and undercloud. Standalone is ok (not using that image)11:37
chandankumararxcruz|ruck: can you check whether it is happening on vexxhost cloud?11:45
arxcruz|ruckchandankumar: which one ?11:46
arxcruz|ruckso many failures11:46
chandankumararxcruz|ruck: few of my failures does not have logs11:47
arxcruz|ruckchandankumar: i need some context11:47
chandankumararxcruz|ruck: based on this message https://wiki.openstack.org/wiki/Infrastructure_Status -> deleted bogus 2001:db8:: addresses on mirror01.ca-ymq-1.vexxhost.opendev.org once more does the conection broken error is happing due to vexxhost cloud11:48
*** jtomasek has joined #oooq11:50
arxcruz|ruckchandankumar: okay, so, what exactly is happening, which jobs are you seeing failing ?11:50
chandankumarfor my case: logs are not there, https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7eb/750119/4/check/tripleo-ci-centos-8-scenario002-standalone/7ebdca4/logs/ -> undercloud logs are not there11:51
chandankumarand https://2cab3e164952e2b59e5f-9f3c64f3d814a9e4d0be6167f83e8b00.ssl.cf2.rackcdn.com/749645/22/check/tripleo-ci-centos-8-content-provider/7362598/ logs are also not there but the job failed11:52
*** jtomasek has quit IRC12:00
akahat|roverarxcruz|ruck, ysandeep hey..12:06
akahat|roverno i didn't seen those error before..12:06
akahat|roverysandeep, have you filed bug for it?12:06
ysandeepakahat|rover, no12:07
arxcruz|ruckakahat|rover: would you mind to fill the bug?12:09
mariospanda12:10
akahat|roverarxcruz|ruck, yes.. i'll... i'm looking in to it..12:11
arxcruz|ruckok12:11
arxcruz|ruckty12:11
*** ysandeep is now known as ysandeep|afk12:15
*** jpena|lunch is now known as jpena12:30
akahat|roverysandeep|afk, arxcruz|ruck i've looked in to the jobs.. most of the jobs what i found are failing with the tripleotraincentos8/centos-binary-neutron-server image not found.12:35
akahat|roverarxcruz|ruck, ysandeep|afk i'm not quite sure about the sskey issue.. so filing bug at the moment is not good idea.. if job fails with the same error again.. then we can..12:36
akahat|roverssh key issue *12:36
arxcruz|ruckakahat|rover: add a note in our ruck rover at least12:37
akahat|roverarxcruz|ruck, yup..12:37
akahat|roverarxcruz|ruck, https://bugs.launchpad.net/tripleo/+bug/1894664 image is not present in the docker registry..12:41
openstackLaunchpad bug 1894664 in tripleo "[train] containers and containers-multinode jobs failing with container not found" [Critical,Triaged]12:41
arxcruz|ruckakahat|rover: ??12:42
*** ysandeep|afk is now known as ysandeep12:42
akahat|roverarxcruz|ruck, tripleotraincentos8/centos-binary-neutron-server this image is not present in the docker hub.12:43
arxcruz|ruckakahat|rover: yup, that's why is failing12:43
ysandeepakahat|rover, ack, thanks!12:44
akahat|roverarxcruz|ruck, can you point me how to fix it?12:44
akahat|rovermarios, chandankumar ^^12:44
arxcruz|ruckakahat|rover: I'm not sure tbh12:44
mariosakahat|rover: in call will check in bit?12:53
akahat|rovermarios, ok13:01
*** jtomasek has joined #oooq13:01
mariostripleo-ci scrum13:01
chandankumarakahat|rover: how it got missed?13:02
chandankumarakahat|rover: you need to manually pull and push the container to docker.io from promoter13:07
akahat|roverchandankumar, ah.. that's weird. :|13:08
bhagyashrismarios, rebased and pushed the patch13:28
mariosbhagyashris: thank you!13:42
*** jtomasek has quit IRC13:50
arxcruz|ruckmarios: akahat|rover chandankumar it's related to train, i know weshay|ruck did some manua changes on train last friday, I'm not sure if manually push the container without his approval would be a good idea13:54
chandankumararxcruz|ruck: gates are busted na?13:55
arxcruz|ruckchandankumar: only stable/train13:56
arxcruz|ruckchandankumar: do you know how to manually push the container?13:57
chandankumararxcruz|ruck: I have not done that, marios might knows13:59
arxcruz|ruckmarios: ^14:00
mariosarxcruz|ruck: chandankumar: e.g. https://gist.github.com/weshayutin/692c80f2d4179dcf2108265bd9218bc914:01
mariosbuildah push 02a65dcb3a4f docker.io/tripleomaster/centos-binary-horizon:0481ffa881dbc1aa61d9810533d8522914:01
arxcruz|ruckman, i'm really not comfortable doing that14:01
chandankumararxcruz|ruck: only one container is missing na?14:05
*** jbadiapa has joined #oooq14:06
chandankumararxcruz|ruck: I have pulled the missing container locallu14:09
chandankumararxcruz|ruck: https://bugs.launchpad.net/tripleo/+bug/189466414:09
openstackLaunchpad bug 1894664 in tripleo "[train] containers and containers-multinode jobs failing with container not found" [Critical,Triaged]14:09
arxcruz|ruckchandankumar: you did?14:10
chandankumararxcruz|ruck: I have not pushed yet14:10
arxcruz|rucklet me add you in the email i sent to weshay|ruck14:10
chandankumarlet me push that one14:12
*** jpena is now known as jpena|away14:12
chandankumararxcruz|ruck: I have pushed the images14:17
chandankumar*image14:17
arxcruz|ruckchandankumar: dhanyavaad14:18
arxcruz|ruckowalsh: it seems chandankumar pushed the missing image, may you recheck your patch ?14:18
*** udesale_ has quit IRC14:27
* chandankumar out, see ya tomorrow14:31
*** saneax has quit IRC15:12
*** jpena|away is now known as jpena15:47
owalsharxcruz|ruck: ack, was afk for a bit. Should have result for https://review.opendev.org/750123 before my patch15:51
*** marios has quit IRC15:58
*** jpena is now known as jpena|away16:15
*** ysandeep is now known as ysandeep|away16:40
*** amoralej is now known as amoralej|off16:45
*** jpena|away is now known as jpena16:54
*** derekh has quit IRC17:03
*** dtantsur is now known as dtantsur|afk17:18
*** jbadiapa has quit IRC17:22
*** jpena is now known as jpena|off17:23
*** skramaja has quit IRC18:11
*** apetrich has quit IRC19:36
*** apetrich has joined #oooq21:53
*** apetrich has quit IRC21:53
*** tosky has quit IRC23:16

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!