Tuesday, 2020-07-07

*** rlandy has quit IRC00:00
*** rfolco has quit IRC00:48
*** ysandeep|away is now known as ysandeep02:26
*** dsneddon has quit IRC03:06
*** jmasud_ has joined #oooq03:21
*** jmasud has quit IRC03:24
*** jmasud_ has quit IRC03:27
*** jmasud has joined #oooq03:28
*** skramaja has joined #oooq04:03
*** ykarel|away is now known as ykarel04:04
*** ratailor has joined #oooq04:56
*** udesale has joined #oooq05:35
*** pojadhav|afk is now known as pojadhav05:49
*** bhagyashris is now known as bhagyashris|brb05:50
*** jschlueter has quit IRC05:52
*** marios has joined #oooq06:01
*** bhagyashris|brb is now known as bhagyashris06:16
bhagyashrisHi all, As per our discussion i have rename the integration pipelines and all the patches are up for review : https://review.rdoproject.org/r/#/c/28403/ https://review.rdoproject.org/r/#/c/28404/ https://review.rdoproject.org/r/#/c/28405/ request you to add in your review list. Thank you :)06:25
bhagyashrishttps://tree.taiga.io/project/tripleo-ci-board/task/1865?kanban-status=144727506:25
bhagyashrischandankumar, marios pojadhav ysandeep akahat weshay_ruck zbr ^^06:26
pojadhavbhagyashris, ack06:27
ysandeepbhagyashris, sure06:27
mariosbhagyashris: ack06:28
bhagyashristhank you all :)06:29
*** jfrancoa has joined #oooq06:33
chandankumarbhagyashris, since you are making changes on pipeline, you also need to submit a change here https://review.rdoproject.org/r/gitweb?p=rdo-infra/rdo-infra-playbooks.git;a=blob;f=contrib/prune_logserver.sh#l7206:33
bhagyashrischandankumar, yes i saw that06:34
chandankumarbhagyashris, we can add a readme with the rdo-jobs zuul.d dir with new naming conventions06:34
bhagyashriseven in https://review.rdoproject.org/codesearch/?q=openstack-periodic-master&i=nope&files=&repos= i saw06:34
bhagyashrislot many places the pipelines names are used06:35
chandankumarbhagyashris, we also need to fix cockpit06:35
bhagyashrisbut i was not sure about that06:35
bhagyashrisso do i need to change there as well06:35
chandankumarbhagyashris, for cockpit sync with weshay_ruck and for prune server, send the patch, dpawlik2 will help there06:36
bhagyashrischandankumar, ack06:36
bhagyashrischandankumar, thank you :)06:36
ysandeepbhagyashris, just want to confirm - previously there was some discussion about segregating c7 train and c8 train jobs in different pipelines, but we are back to zuul.d/integration-pipeline-stable2.yaml for both c7/c8 train.. Is that segregation plan dropped now?06:48
pojadhavchandankumar, marios : please help to merge this https://review.opendev.org/#/c/738568/06:50
ysandeeppojadhav, tripleo-ci-centos-8-standalone-on-multinode-ipa failed.. Should we wait till we get a green run there?06:51
ysandeeplooks like its currently failing because of lack of containers - https://abf795b96ba14df61b38-6bdee08b27c25482e4dba43ccebb604b.ssl.cf2.rackcdn.com/738568/2/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/8a40d15/logs/undercloud/home/zuul/standalone_deploy.log06:52
bhagyashrisysandeep, imo as per the new plan that should get drop06:53
ysandeepbhagyashris, okay , thanks!06:56
bhagyashrisysandeep, afaik why we decided to segregate both c7/c8 train because the openstack-periodic-24hr pipeline was getting flooded with (c7 train, c8 train and stein jobs) so now that's get resolved with new design06:58
bhagyashrisysandeep, otherwise will need new naming again and currently stable2 is train and that is the only release which runs on both c7 and c807:00
bhagyashrisimo, better to keep in same pipeline and template :)07:02
*** amoralej|off is now known as amoralej07:04
ysandeepbhagyashris, yes I agree with that combine is much clean .. but i think we missed this point in yesterday's discussion and https://review.rdoproject.org/r/#/c/28403/3/zuul.d/integration-pipeline-stable2.yaml seems too many job but i will let other to comment there..07:05
bhagyashrisysandeep, sure will wait for others opinion07:06
mariosack pojadhav07:07
ysandeepbhagyashris, left a comment on patch for other's reviewers too07:16
*** dtantsur|afk is now known as dtantsur07:18
pojadhavysandeep, that job is non voting07:18
*** pojadhav is now known as pojadhav|brb07:18
ysandeeppojadhav|brb, okay good then, thanks!07:19
bhagyashrisysandeep, ack thank you :)07:22
mariosbhagyashris: ack on https://review.rdoproject.org/r/#/c/28403/ & friends but IMO we don't need to rush that i think most of the team should at least +1 it... maybe start by bringing to community call today07:22
*** rascasoft has joined #oooq07:24
*** ccamacho has joined #oooq07:25
bhagyashrismarios, ack thanks for your review :) yeah :D i have added all the team members in reviewers list :)07:26
mariosbhagyashris: ack i think you'll find you will also need to harrass them at regular intervals ;)07:26
bhagyashrismarios, exactly will do that :)07:33
*** tosky has joined #oooq07:36
*** dsneddon has joined #oooq07:39
*** dsneddon has quit IRC07:44
*** saneax has joined #oooq07:47
*** derekh has joined #oooq08:30
*** ysandeep is now known as ysandeep|lunch08:34
zbrpanda: marios: please help me with https://review.rdoproject.org/r/#/c/28390/ -- should be quite easy09:14
*** jfrancoa has quit IRC09:17
*** jfrancoa has joined #oooq09:27
*** jfrancoa has quit IRC09:27
*** jfrancoa has joined #oooq09:31
*** pojadhav|brb is now known as pojadhav09:34
arxcruzmarios: https://review.opendev.org/#/c/738788/ while zuul is happy :D09:37
chandankumarsshnaidm|ruck, arxcruz marios https://review.opendev.org/#/c/737727/32 good to go09:39
arxcruzchandankumar: https://review.opendev.org/#/c/738788/ also gtg09:41
*** jfrancoa has quit IRC09:44
*** jfrancoa has joined #oooq09:45
*** holser has quit IRC09:48
*** ysandeep|lunch is now known as ysandeep09:48
mariosarxcruz: chandan beat me to it09:48
marioszbr: chandankumar: added to queue09:49
arxcruzsudo marios do it!09:51
*** tosky has quit IRC10:00
*** tosky has joined #oooq10:01
pandamarios: zbr https://review.rdoproject.org/r/27626 is becoming urgent ...10:08
zbrpanda: i cannot help you with changes that cannot be merged10:12
pandazbr: now it can, and you can help me anyway just telling me if there's anything else missing to do10:13
pandaor something you don't like10:13
zbrpyyaml commenting is really weird, it is or it is not a requirement?10:14
zbrif it is, it should be in, even if you need to put different conditions for py2710:14
zbrother than this, no. GTG from me.10:15
pandazbr: it's a requiremente, but I guess it's pulled automatically. If I make it explicit py27 breaks10:15
*** holser has joined #oooq10:15
zbrsomething is wrong, i am afraid we cannot just remove it. you can lower version only for py27. do you have link?10:16
pandazbr: a link of what ?10:17
pandazbr: I think it fails only locally10:17
zbrto the error caused when you keep it10:17
pandazbr: zuul tests pass10:17
zbrin that case, i do not care yet.10:17
pandaERROR:   py27: could not install deps [pytest-cov, -r/workspace/repos/rdo-infra-ci-config/test-requirements.txt, -r/workspace/repos/rdo-infra-ci-config/ci-scripts/dlrnapi_promoter/requirements.txt, -r/workspace/repos/rdo-infra-ci-config/ci-scripts/dlrnapi_promoter/test-requirements.txt, -r/workspace/repos/rdo-infra-ci-config/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf/requirements.txt]; v =10:17
pandaInvocationError('/workspace/repos/rdo-infra-ci-config/.tox/py27/bin/pip install -U pytest-cov -r/workspace/repos/rdo-infra-ci-config/test-requirements.txt -r/workspace/repos/rdo-infra-ci-config/ci-scripts/dlrnapi_promoter/requirements.txt -r/workspace/repos/rdo-infra-ci-config/ci-scripts/dlrnapi_promoter/test-requirements.txt10:17
panda-r/workspace/repos/rdo-infra-ci-config/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf/requirements.txt', 1)10:17
zbrahh, I know whyt10:18
zbrremove it for now10:18
zbryou cannot specify requirements in multiple locations and include them in the same install command10:18
zbrconflicts10:18
zbranother reason why promoter should be role, so he would be able to own the deps install part10:19
zbranyway, rebase and merge.10:19
mariospanda: ack10:23
*** bhagyashris is now known as bhagyashris|brb10:37
*** holser has quit IRC11:06
*** bhagyashris|brb is now known as bhagyashris11:12
weshay_ruckpanda, have you looked at http://promoter.rdoproject.org/centos8_train.log today?11:29
weshay_ruckhttps://review.opendev.org/#/c/739523/ merged11:29
weshay_ruckI see new errors11:29
weshay_ruckpanda, containers and overcloud images failed11:30
weshay_ruckysandeep, FYI.. still hitting some things w/ c8-train promotion11:30
ysandeepweshay_ruck, yes i saw11:31
pandaweshay_ruck: it's trying to pull the ppc images11:32
weshay_ruckpanda, so we just need to turn ppc off?11:33
weshay_rucksoniya29|rover, http://dashboard-ci.tripleo.org/d/QrkFP-zGz/upstream-and-rdo-promotions11:33
pandaweshay_ruck: done now, we can restart and see11:33
weshay_rucksshnaidm|ruck, soniya29|rover https://review.opendev.org/#/c/739637/11:40
sshnaidm|ruckarxcruz, chandankumar any thoughts about this error? CRITICAL tempest [-] Unhandled error: tempest.lib.exceptions.UnexpectedContentType: Unexpected content type provided  https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_add/737759/3/gate/tripleo-ci-centos-8-scenario001-standalone/add4f92/job-output.txt11:41
*** ysandeep is now known as ysandeep|afk11:41
weshay_rucksshnaidm|ruck, soniya29|rover https://review.opendev.org/#/c/739135/11:42
arxcruzsshnaidm|ruck: no, first time seeing it11:42
arxcruzsshnaidm|ruck: only happen this time, or is it happening all the time ?11:44
sshnaidm|ruckarxcruz, this specific error saw once in gate, but I'm checking 10 failures in gates now and all of them because of tempest failing..11:45
arxcruzsshnaidm|ruck: with this error right ?11:45
sshnaidm|ruckarxcruz, no, this specific is once so far, continue looking in11:45
arxcruzok11:45
*** udesale_ has joined #oooq11:49
*** jfrancoa has quit IRC11:50
*** jfrancoa has joined #oooq11:50
*** derekh has quit IRC11:50
*** udesale has quit IRC11:52
chandankumarweshay_ruck, https://review.opendev.org/737727 we are good to go and switch to new container build with native push11:52
chandankumarweshay_ruck, check my last comment11:52
*** amoralej is now known as amoralej|lunch12:00
weshay_ruckchandankumar, \0/ woot https://logserver.rdoproject.org/89/28389/3/check/periodic-tripleo-build-containers-ubi-8/cfa393b/logs/build.log and https://registry.rdoproject.org:8443/oapi/v1/namespaces/testingmaster/imagestreamtags/12:01
weshay_rucklook good12:01
weshay_ruck2020-07-07 07:40:26 | Pushing trunk.registry.rdoproject.org/testingmaster/centos-binary-tempest:a1a85c0578faa814647172f8ab62c311 image with: sudo buildah push --tls-verify=False trunk.registry.rdoproject.org/testingmaster/centos-binary-tempest:a1a85c0578faa814647172f8ab62c311 docker://trunk.registry.rdoproject.org/testingmaster/centos-binary-tempest:a1a85c0578faa814647172f8ab62c31112:01
weshay_ruckvery nice12:01
chandankumarweshay_ruck, merge it sir12:01
chandankumarweshay_ruck, rlandy on pto today?12:02
weshay_ruckchandankumar, aye captain12:02
weshay_ruckchandankumar, don't think so12:02
chandankumarweshay_ruck, ok, thanks!12:02
weshay_ruckchandankumar, bonus points for getting someone to fix12:03
weshay_ruck2020-07-07 07:40:47 | ============= SUMMARY ===============12:03
weshay_ruck2020-07-07 07:40:47 | Expected................... 11012:04
weshay_ruck2020-07-07 07:40:47 | Built...................... 10912:04
weshay_ruck2020-07-07 07:40:47 | =====================================12:04
*** rfolco has joined #oooq12:04
chandankumarweshay_ruck, kevin knows about this issue12:04
*** ysandeep|afk is now known as ysandeep12:06
ykarelchandankumar, ^^ issue i mentioned12:09
ykarelduplicate designate_base12:09
ykarelin tripleo-common12:09
ykarelhttps://github.com/openstack/tripleo-common/blob/master/container-images/tripleo_containers.yaml12:10
chandankumarhttps://github.com/openstack/tripleo-common/blob/master/container-images/tripleo_containers.yaml#L4912:11
chandankumarlet me fix thatr12:11
mariosysandeep: have you seen an issue in periodic centos8 train jobs like "./logfile_02May2018.log:44:"reosolve issues " actually this is customer focus but the points identifed are basically what we do every time we have a customer case/bz.12:11
ykarelack12:11
mariosysandeep: sorry wrong paste, that one ./logfile_03Apr2017.log:160:* calls: customer call (engineering field engagement) pneedle msutton sofer matbu morazi - lab minor update 2 issues, undercloud iface bonding issue, overcloud update failing. will file bz12:11
marios./logfile_02May2018.log:44:"reosolve issues " actually this is customer focus but the points identifed are basically what we do every time we have a customer case/bz.12:11
mariosoh come on :D12:11
mariosysandeep: sorry i ... this tripleo_common.image.image_uploader [  ] Error detected in auth headers: error insufficient_scope12:12
mariosysandeep: i am filing a bug and can't see it already in https://bugs.launchpad.net/tripleo12:12
ysandeepmarios, can you please point me to complete logs, may be i recognizes them12:13
mariosysandeep: i saw it in train update and ussuri standalone-upgrade12:13
mariosysandeep: sure, sec12:13
mariosysandeep: https://2a4f7f67bcfb8d0b3cd4-f9c86595ee722ed5765b04c30efd2d71.ssl.cf1.rackcdn.com/739459/5/check/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/c1a2641/job-output.txt12:14
ysandeeplooking12:14
mariosysandeep: ack, question is if you've filed a bug for that/if you've seen it already sounds like no12:14
mariosysandeep: thanks12:14
ysandeepmarios, these are the one i aware you about last week.. its failing because of no containers here https://hub.docker.com/u/tripleotraincentos812:15
mariosysandeep: i see... do we have a bug for it? or is it tracked somewhere else12:16
ysandeepmarios, no bz yet.. but weshay_ruck is trying to promote containers for c8 train12:16
mariosysandeep: are we doing something/... ok i see12:16
mariosysandeep: well there was a train promotion last night12:16
mariosysandeep: but it looks like it didn't fix it yet12:16
pandaall aboard the promotion train12:16
pandawhooo whoooo12:16
ysandeepmarios, we got promotion.. :)12:17
ysandeeppanda++ weshay_ruck++12:17
ysandeepthank you guys!12:17
* marios sees panda driving - pulls emergency brake & jumps12:17
pandazbr: can you help me understand what are the plans with py27 ?12:17
weshay_ruckysandeep, panda wait wait12:17
mariosysandeep: but the problem remains12:18
weshay_ruckc8-train did not promote12:18
mariosysandeep: those logs are from today12:18
*** holser has joined #oooq12:18
mariosweshay_ruck: oh you just created previous manually or something?12:18
weshay_ruckmarios, yes12:18
mariosweshay_ruck: i thought it was a proper promotion12:18
mariosweshay_ruck: ack ok i see sorry12:18
zbrpanda: sure, remove them as soon as possible (can drop it in some places faster than in others)12:18
mariosysandeep: so we didn't get a promotion ^ weshay_ruck manually fixed the issue i had yesterday12:18
weshay_ruckmarios, panda we need to turn off manifest creation12:18
ysandeepmarios, aahhh12:18
pandaweshay_ruck: I turned off manifest push ...12:18
zbrif we have a specific place where we need to keep py27 support, we can contain that, isolating from others.12:19
pandabut it's not enough ...12:19
mariosweshay_ruck: more problems with ppc manifests missing or somethign?12:19
zbrso changes made to one place do not break another12:19
*** rlandy has joined #oooq12:19
pandazbr: is it ok if I remove every trace of py27 from tox ?12:19
zbrlikely not12:19
pandazbr: at this point I would need to fix the dependency12:20
zbri do have a good example where it does not make sense12:20
weshay_ruckpanda, do you have a vi session open on the promoter?12:20
pandazbr: and I'm not sure it's worth it.12:20
rlandychandankumar: hey - https://code.engineering.redhat.com/gerrit/#/c/205466/ - you got the downstream node_failure fixed?12:20
pandaweshay_ruck: yes, exited now12:20
zbrif we have an ansible module written in python, one that also needs to run on centos-712:20
weshay_rucklooks like master is promoting again12:20
chandankumarrlandy, yes boss12:20
rlandychandankumar: yay - thank you12:21
chandankumarrlandy, few more patches12:21
zbrbut if we can control where it runs, we can go py36 pure12:21
chandankumarrlandy, https://code.engineering.redhat.com/gerrit/203744 and https://code.engineering.redhat.com/gerrit/20547412:21
chandankumarrlandy, container build is working fine now12:22
rlandychandankumar: ok - working downstream as well ... except for ovn ...12:22
* rlandy gets12:22
pandaweshay_ruck: there's a bug in the ppc part12:23
pandait's failing the playbook12:23
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/78/205378/4/check/tripleo-build-containers-ubi-8-internal-rhel-8-rhos-17/2ec1b1c/logs/build.log12:23
pandaweshay_ruck: I'll try to fix it12:23
weshay_ruckpanda, thanks12:23
weshay_ruckzbr, rfolco marios ci-config monitoring.. on the main cockpit page.. next time I launch an env.. I'll fix the legend for the graphs.  http://dashboard-ci.tripleo.org/d/wb8HBhrWk/cockpit?orgId=112:24
rlandyNo match for argument: python3-ovn-octavia-provider12:24
rlandyError: Unable to find a match: python3-ovn-octavia-provider12:24
weshay_ruckscroll down to periodic jobs12:24
rlandychandankumar: ^^12:24
ysandeeprlandy, hey good morning o/ , please add these to your review list https://code.engineering.redhat.com/gerrit/#/c/205475/ and https://code.engineering.redhat.com/gerrit/#/c/205353/12:25
rlandymust be missing that downstream python3-ovn-octavia-provider12:25
rlandyysandeep: hi - https://code.engineering.redhat.com/gerrit/#/c/205475 looks ok to me but pls ask tristan or nhicher to confirm12:27
ysandeeprlandy, okay, thanks!12:27
weshay_ruckreviews please https://review.rdoproject.org/r/#/c/28403/12:29
weshay_ruckhttps://review.rdoproject.org/r/#/c/28405/12:29
chandankumarrlandy, looking12:29
rlandyhttps://code.engineering.redhat.com/gerrit/#/c/205353/ should be ok - must be taking that anyways12:30
rlandychandankumar: are you all set with container push now?12:31
chandankumarrlandy, yes12:31
rlandychandankumar: then can we meet for a  few?12:31
rlandyI need the same playbook changes downstream12:31
chandankumarrlandy, yes sure12:32
weshay_ruckrlandy, https://review.rdoproject.org/r/#/c/28403/12:33
rlandylooking12:33
chandankumarrlandy, https://meet.google.com/?hs=197&pli=1&authuser=012:34
rlandychandankumar: will be there in a esc - just checking pipeline reviews12:34
rlandyweshay_ruck: one comment on https://review.rdoproject.org/r/#/c/28403/12:37
rlandyotherwise good to go12:37
rlandychandankumar: joining12:40
chandankumarrlandy, https://meet.google.com/mhq-bvmm-hwv?pli=1&authuser=012:41
*** ratailor has quit IRC12:56
bhagyashrisrlandy, weshay_ruck i keep integration-pipeline-stable4-5.yaml instead of zuul.d/integration-pipeline-stable4-plus.yaml because we keep the queens and rocky jobs there ... otherwise if you want me to change it i will do that quickly np :) thank you :)12:58
weshay_ruckI think it's fine the way you have it12:59
bhagyashrisweshay_ruck, ack thank you :)13:00
*** rascasoft has quit IRC13:08
*** amoralej|lunch is now known as amoralej13:08
mariosweshay_ruck: nice thanks :D (sorry just saw the notification re ci-config periodic on grafana13:15
weshay_ruck:)13:16
*** TrevorV has joined #oooq13:16
*** Goneri has joined #oooq13:18
*** derekh has joined #oooq13:19
chandankumarrlandy, check the chatter on #rhos-delivery13:25
rlandychandankumar: thanks - looking13:26
rfolcoping community meeting13:29
rfolcoarxcruz, rfolco, zbr, panda, sshnaidm, rlandy, marios, ysandeep, bhagyashris, soniya29, pojadhav, akahat, weshay, chandankumar13:29
rfolcohttps://meet.google.com/bqx-xwht-wky?authuser=113:29
rfolcohttps://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw?both13:29
weshay_ruckrfolco, I'm in a manager thang13:30
weshay_ruckcarry on w/o me13:30
rfolcoack weshay_ruck13:30
rlandychandankumar++13:30
rlandythanks for getting this sorted13:30
*** Goneri has quit IRC13:33
mjturekrfolco: any idea what causes this? Not sure what's supposed to generate this folder or what's looking for it https://logserver.rdoproject.org/ci.centos.org/tripleo-upstream-containers-build-master-ppc64le/2460/logs/logs/containers-build-errors.log13:39
rfolcomjturek, will look in a few after my current mtg13:40
mjturekthx, np13:40
chandankumarrlandy, marios weshay_ruck sshnaidm|ruck bhagyashris ysandeep https://review.rdoproject.org/r/#/c/28404/1/zuul.d/upstream.yaml@121 have a look when free13:56
rlandywill look after meeting13:56
ysandeepchandankumar, I agree with segregating if we are hitting resource crunch because of more number of jobs. i posted similiar question on this https://review.rdoproject.org/r/#/c/28403/3/zuul.d/integration-pipeline-stable2.yaml13:58
*** Goneri has joined #oooq14:02
mariosack chandankumar14:05
bhagyashrischandankumar, ack will do the changes soon :) thanks14:07
ysandeeprlandy, I was thinking if we should probably use rhel8.2 instead of 8.1 as base image here:- https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/sf-config.git;a=blob;f=nodepool/tripleo.yaml#l9414:08
rlandyysandeep: that's fine14:08
rlandywe run rhos-release 8.214:08
rlandyso we come out with 8.2 mostly14:09
*** bhagyashris is now known as bhagyashris|dinn14:09
rlandybut switching the node is fine14:09
ysandeeprlandy, you mean we perform a yum update after "rhos-release 8.2"14:09
rlandyif you look at yum.repos.d we will get 8.2 repos14:11
rlandysome test update14:11
rlandydepends'14:11
rlandyupdating the node is good14:11
ysandeeprlandy, okay, I will prepare a patch for your review - to switch to 8.2 - I found a TODO to remove 8.1 image - https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/sf-config.git;a=blob;f=nodepool/nodepool.yaml#l7314:13
ysandeeprlandy, thanks!14:14
ysandeepmarios, rlandy chandankumar need reviews on "Multinode base job" https://code.engineering.redhat.com/gerrit/#/c/201375/ whenever time permits14:17
mariosack ysandeep14:17
chandankumarysandeep, put this also https://code.engineering.redhat.com/gerrit/#/c/205474/3/zuul.d/standalone-jobs.yaml@24 in base job definitions14:19
*** ccamacho has quit IRC14:19
ysandeepchandankumar, ack updating14:20
chandankumarysandeep, thanks!14:20
soniya29|roverweshay_ruck, sshnaidm|ruck, I see too many stein jobs with skipped result14:24
weshay_rucksoniya29|rover, did the containers build job fail?14:24
soniya29|roverweshay_ruck, nope, it is also skipped14:25
sshnaidm|rucksoniya29|rover, on a patch?14:25
rfolcomjturek, hmmm this happens here https://github.com/openstack/tripleo-ci/blob/16756e77b7fd21cec596d08fea3ca06755aea223/roles/build-containers/templates/build.sh.j2#L8614:26
soniya29|roversshnaidm|ruck, no logs, So no idea14:26
sshnaidm|rucksoniya29|rover, where do you see these jobs?14:26
rfolcomjturek, I don't know why /tmp/containers-* don't exist, but maybe its worth adding a || true like in https://github.com/openstack/tripleo-ci/blob/db711241b9265d83b1a20afe04ba6b294eb3512d/roles/build-containers/tasks/post.yaml#L2014:26
soniya29|roversshnaidm|ruck, https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-24hr#14:27
rfolcomjturek, let me check our buildah logs to compare14:27
sshnaidm|rucksoniya29|rover, that was yesterday14:27
sshnaidm|rucksoniya29|rover, because of failures on vexxhost, you can see node_failure there14:27
mjturekahhh cool thanks rfolco14:27
sshnaidm|rucksoniya29|rover, periodic-tripleo-centos-7-stein-promote-consistent-to-tripleo-ci-testing is the job that starts all other jobs, if it fails - all the rest will be skipped14:28
rfolcomjturek, look this https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-master-containers-build-push/d69e3f5/logs/containers-build-errors.log14:28
rfolcomjturek, lol14:28
mjturekhahaha cool14:29
mjturekso fix needs to be in tripleo-ci?14:29
ysandeepchandankumar, i have updated https://code.engineering.redhat.com/gerrit/#/c/201375/14:29
rfolcomjturek, hmmmm look this https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-master-containers-build-push/d69e3f5/logs/buildah-builds/14:31
mjturekrfolco interesting.. we don't have that docker folder14:32
rfolcomjturek, container-builds is not under /tmp/, its under /tmp/buildah-builds14:32
mjturekhm we copy over the entire buildah-builds folder from what I remember but need to double check that14:34
rfolcomjturek, I am not 100% sure, but when building with buildah, temp build path is different from when building with docker...14:34
soniya29|roversshnaidm|ruck, ack14:34
*** holser has quit IRC14:35
*** holser has joined #oooq14:36
sshnaidm|ruckpanda, can we delete on tripleo-infra tenant: promoter-new     | SHUTOFF | private=192.168.100.34, 38.145.33.17414:37
rfolcomjturek, maybe /tmp/*/container-*/docker ? or /tmp/buildah-builds/container-*/ if buildah14:38
rfolcomjturek, but that error is not making the job to fail... is it ?14:38
mjturekI don't think it is, but it's the first time in awhile I've seen anything in the container-build-errors file14:39
ysandeepweshay_ruck, rlandy something is wrong here - https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=linters14:40
ysandeeplinters failing with node failure14:40
weshay_ruck#rhos-ops14:40
*** ccamacho has joined #oooq14:44
zbrpanda: weshay_ruck marios: zuul ui change: https://review.opendev.org/#/c/739559/ -- you know how to use dasboard jobs site-preview feature, right?14:44
weshay_ruckzbr, perhaps a link in the commit message w/ a screenshot would help?14:46
zbrweshay_ruck: https://sbarnea.com/ss/Screen-Shot-2020-07-07-15-46-54.50.png14:47
zbrss are easy to make but they do not give good review ability.14:47
zbrmainly each reviewer needs to know how to browse zuul UI using site-preview14:47
zbrbeing able to test zuul ui changes without merging is so awesome! only downside is that these test links are not shareable.14:48
zbrbasically this removes the top level frame container (not really removes but makes it not visible as a table)14:49
zbrreduces complexity a little, at some point I seen 4 nested tables...14:49
*** saneax has quit IRC14:51
*** saneax has joined #oooq14:55
*** bhagyashris|dinn is now known as bhagyashris14:56
*** saneax has quit IRC14:56
*** saneax has joined #oooq14:57
*** saneax has quit IRC15:01
*** saneax has joined #oooq15:04
*** saneax has quit IRC15:05
bhagyashrismarios, weshay_ruck rlandy rfolco chandankumar ysandeep sshnaidm|ruck request you to please look into ysandeep comment on https://review.rdoproject.org/r/#/c/28404/ and would like your opinions on it. thank you :)15:09
rlandybhagyashris: left my reply there to ysandeep comment15:11
weshay_ruckpanda, think we'll get the train promotion this time?15:13
pandaweshay_ruck: I looked at bit more, the entire ppc part should be skipped when manifest_push if false, so swe shjould not get the ppc error again15:14
weshay_ruckanybody need anything?15:14
weshay_ruckpanda, rock on15:14
pandaI need some water.15:15
* weshay_ruck books flight.. water boy otw15:15
rlandysshnaidm|ruck: want to meet to discuss the work for reducing jobs on rdocloud?15:15
weshay_ruckhttps://images.app.goo.gl/eyNc4vtQ6jNCBWHZ715:15
weshay_ruckand removing some jobs against tripleo-ci again15:16
sshnaidm|ruckrlandy, https://meet.google.com/ejg-fnkg-nbb15:16
rlandyjoining15:16
*** pojadhav is now known as pojadhav|dinner15:22
weshay_rucksshnaidm|ruck, rlandy https://docs.google.com/document/d/1DgPbxqkW6nz9cXmtEz6WR61sHkQbRgbVDBtRzW2v72A/edit15:23
bhagyashrisrlandy, ack thanks15:23
*** bogdando has joined #oooq15:31
*** ysandeep is now known as ysandeep|away15:32
*** dtantsur is now known as dtantsur|brb15:49
*** ccamacho has quit IRC15:49
*** ykarel is now known as ykarel|away15:57
*** amoralej is now known as amoralej|off15:57
*** marios has quit IRC16:02
chandankumarweshay_ruck, rlandy please merge this https://review.rdoproject.org/r/#/c/28369/ and https://review.rdoproject.org/r/#/c/28370/ once upstream review lands16:05
chandankumarrelated to new container build16:06
chandankumarand it will close new container build user story16:06
rlandyack16:06
weshay_rucknice16:07
chandankumarweshay_ruck, rlandy https://review.opendev.org/#/c/739415/ and this also16:07
weshay_ruckchandankumar, I have questions about https://review.opendev.org/#/c/739415/16:07
weshay_rucklet me look at this again16:08
chandankumarweshay_ruck, sure16:08
weshay_ruckchandankumar, are you 100% the podman login does NOT leave the creds you need for buildah?16:08
weshay_ruckchandankumar, also this looks like we just login in w/ both16:08
chandankumarweshay_ruck, yes,16:08
weshay_ruckmaybe we should chat for a sec16:09
chandankumarweshay_ruck, check the commit msg16:09
weshay_ruckah I see16:09
chandankumarhttps://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-rdo-base/container-login.yaml#L5516:09
chandankumarwill be removed once upstream patch lands16:09
weshay_ruckchandankumar, ysandeep|away ok.. centos-8 train has been pushed to docker.io16:14
weshay_ruckfull promotion there16:14
weshay_ruckhttps://trunk.rdoproject.org/centos8-train/16:14
weshay_ruckpanda++16:14
pandafinally ../.16:15
*** skramaja has quit IRC16:20
*** ysandeep|away is now known as ysandeep16:21
ysandeepweshay_ruck, panda thank you16:21
rlandyweshay_ruck: sshnaidm|ruck: shared google doc with you both on psiphx16:25
rlandywe can share that with infra once you review16:25
*** udesale_ has quit IRC16:29
*** chandankumar is now known as raukadah16:38
pandazbr: so is this still a problem with the mirror ? https://logserver.rdoproject.org/29/28429/3/check/molecule-container-push-delegated-centos-7/a6dc1e1/job-output.txt16:39
*** bogdando has quit IRC16:40
*** dtantsur|brb is now known as dtantsur16:41
*** Trevor_V has joined #oooq16:52
*** TrevorV has quit IRC16:57
*** derekh has quit IRC17:27
*** pojadhav|dinner is now known as pojadhav17:34
*** dtantsur is now known as dtantsur|afk17:38
rfolcozbr, are you planning to fix provision/default scenario ? let me know if need help... as the other scenarios include it17:55
weshay_rucksshnaidm|ruck, heh.. you don't have to do this anymore :) but it's fine if you do18:43
weshay_ruckjust ping me :)18:43
rlandyraukadah: ok - we're building container downstream with new changes ... https://sf.hosted.upshift.rdu2.redhat.com/logs/78/205378/5/check/tripleo-build-containers-ubi-8-internal-rhel-8-rhos-17/c8148dc/logs/build.log19:18
rlandyredhat-binary-nova-libvirt19:28
rlandylol - we're back to the rhel redhat game19:28
*** jfrancoa has quit IRC19:31
rlandyweshay_ruck: sshnaidm|ruck: ok if I merge https://review.rdoproject.org/r/#/c/28370/ - it will change container push in master pipeline19:40
sshnaidm|ruckrlandy, ack19:42
rlandysshnaidm|ruck: k - feel free to revert if it kill something19:44
*** jfrancoa has joined #oooq19:48
*** Trevor_V has quit IRC20:22
*** TrevorV has joined #oooq20:25
*** TrevorV has quit IRC20:42
weshay_ruckrlandy, can we merge bhagyashri's changes to the pipeline names?21:01
weshay_ruckshall we double check that?21:02
weshay_ruckrlandy, https://review.rdoproject.org/r/#/c/28403/ https://review.rdoproject.org/r/#/c/28405/ https://review.rdoproject.org/r/#/c/28404/21:03
rlandyweshay_ruck: I did21:08
rlandyhttps://review.rdoproject.org/r/#/c/28403/21:08
rlandyI didn't merge21:08
rlandyweshay_ruck: I merged the three patches chandan asked for21:09
weshay_ruckrlandy, aye.. so wdyt.. re: the renames?21:09
rlandyweshay_ruck: yeah21:10
rlandyI think everyone agreed21:10
weshay_ruckk.. please review / vote and if u like what you see I'll merge21:10
rlandyweshay_ruck: merging https://review.rdoproject.org/r/#/c/28403/21:11
rlandyweshay_ruck: https://review.rdoproject.org/r/#/c/28405/ can't rebase21:12
weshay_ruckrlandy, https://review.rdoproject.org/r/#/c/28404/21:12
rlandyweshay_ruck: voted on https://review.rdoproject.org/r/#/c/28404/21:12
rlandywaiting to merge that on https://review.rdoproject.org/r/#/c/28403/21:13
weshay_ruckthanks21:14
rlandyugh I hate config21:15
rlandyweshay_ruck: all set now with merges?21:24
weshay_ruckthink so21:28
* weshay_ruck looks21:28
weshay_ruckconflict on https://review.rdoproject.org/r/#/c/28405/21:30
weshay_ruckno21:30
weshay_ruckmeh.. tomorrow21:31
weshay_ruckconfig review is still getting checked by the rdo-registry test21:32
rlandyno big deal on https://review.rdoproject.org/r/#/c/28405/ - just clean up21:39
*** jfrancoa has quit IRC21:53
*** rlandy is now known as rlandy|bbl22:08
*** dsneddon has joined #oooq22:25
*** jbadiapa has quit IRC22:28
*** johnsom has quit IRC22:51
*** zbr has quit IRC22:53
*** tosky has quit IRC23:01
*** rlandy|bbl is now known as rlandy23:21
*** Goneri has quit IRC23:25

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!