Thursday, 2020-08-27

rlandyowalsh: logging off now - will check in on status tomorrow01:15
*** rlandy has quit IRC01:15
*** ykarel|away has joined #oooq02:00
*** ykarel|away has quit IRC02:06
*** apetrich has quit IRC02:26
*** apetrich has joined #oooq02:27
*** ysandeep|away is now known as ysandeep02:39
*** jfrancoa has quit IRC02:41
*** jfrancoa has joined #oooq02:57
*** skramaja has joined #oooq03:57
*** ykarel|away has joined #oooq04:25
*** ykarel_ has joined #oooq04:27
*** ykarel|away has quit IRC04:30
*** ratailor has joined #oooq04:31
*** saneax has joined #oooq04:46
*** ykarel_ is now known as ykarel04:49
*** bhagyashris|away is now known as bhagyashris05:02
*** udesale has joined #oooq05:28
*** skramaja has quit IRC05:44
*** ratailor has quit IRC05:47
*** ratailor has joined #oooq05:48
*** ratailor has quit IRC06:24
*** skramaja has joined #oooq06:27
ysandeepchandankumar, I need an advice - I think we got a promotion issue in 16.2 (need to check with rlandy / weshay when they come online)- This tag was promoted to current tripleo - http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/current-tripleo/delorean.repo.md506:28
ysandeepBut we never even created container with this tag in integration line - https://registry-console-default.cloud.registry.upshift.redhat.com/registry#/images/tripleorhos-16-2/rhel-binary-aodh-api06:28
ysandeepBecause of this all component jobs are failing:-06:28
ysandeephttps://sf.hosted.upshift.rdu2.redhat.com/logs/69/209869/1/check/periodic-tripleo-ci-rhel-8-scenario010-standalone-octavia-rhos-16.2/87815e4/logs/undercloud/home/zuul/standalone_deploy.log06:28
ysandeep~~~06:28
ysandeepNot found image: https://docker-registry.upshift.redhat.com/v2/tripleorhos-16-2/rhel-binary-nova-novncproxy/manifests/05bde8dc67d20acd2208788ffd361f55"]06:28
ysandeep~~~06:29
ysandeepI am thinking, Can I use something like this https://code.engineering.redhat.com/gerrit/#/c/208803/6/zuul.d/tripleo-build-containers.yaml to create container with current-tripleo hash so that component line gets unblocked.06:29
*** jtomasek has joined #oooq06:30
ysandeepI can depends on ^^ and retrigger container build testpatch06:31
chandankumarysandeep: looking06:38
chandankumarysandeep: for above job container with this tag http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/tripleo-ci-testing/delorean.repo.md5 98fc67cafb2c78c488790718bd2bcec0 was needed06:42
chandankumarre-reading back sorry06:43
chandankumarysandeep: you can do that06:45
ysandeepchandankumar, thanks06:45
chandankumarysandeep: it will push the containers with current-tripleo dlrn md 5 hash to the same registry06:46
ysandeepchandankumar, ack, thank you soo much for confirmation06:46
*** ysandeep is now known as ysandeep|afk06:47
*** ratailor has joined #oooq07:03
*** bogdando has joined #oooq07:11
*** amoralej|off is now known as amoralej07:17
*** dtantsur|afk is now known as dtantsur07:40
*** tosky has joined #oooq07:47
*** derekh has joined #oooq07:52
*** pojadhav is now known as pojadhav|lunch07:52
*** ysandeep|afk is now known as ysandeep07:55
ysandeepchandankumar, fyi.. it worked, thanks!07:58
chandankumarysandeep: cool!07:58
ysandeepchandankumar, also when you get time please add https://code.engineering.redhat.com/gerrit/#/c/209759/ in your review list, test runs are green (result urls in last comment)08:00
akahat|roverchandankumar, arxcruz|ruck https://review.opendev.org/#/c/748160/08:27
akahat|roverhttps://review.opendev.org/#/c/720705/08:27
*** pojadhav|lunch is now known as pojadhav08:42
chandankumararxcruz|ruck: Hello08:46
arxcruz|ruckchandankumar: hey08:46
chandankumararxcruz|ruck: seen this issue https://logserver.rdoproject.org/88/28988/1/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens/880786a/job-output.txt08:46
chandankumararxcruz|ruck:  keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate.08:46
arxcruz|ruckchandankumar: didn't get there yet08:47
chandankumarnot sure08:47
chandankumarhttps://logserver.rdoproject.org/88/28988/1/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens/880786a/logs/undercloud/home/zuul/.config/openstack/clouds.yaml.txt.gz08:47
chandankumar        auth_url: https://10.0.0.5:13000//v308:47
chandankumaris causing that08:48
arxcruz|ruckchandankumar: you know how to fix it? without me thinking about it?08:50
chandankumaryes it is the issue08:50
arxcruz|ruck:D08:50
chandankumarhttps://logserver.rdoproject.org/88/28988/1/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky/74d67b3/logs/undercloud/home/zuul/.config/openstack/clouds.yaml.txt.gz08:51
chandankumarsee rocky oe08:51
chandankumarone08:51
*** ykarel has quit IRC08:51
*** ykarel has joined #oooq08:53
*** jfrancoa has quit IRC09:20
*** holser has quit IRC09:45
*** holser has joined #oooq09:59
*** derekh has quit IRC10:00
*** sshnaidm|afk is now known as sshnaidm10:30
*** ysandeep is now known as ysandeep|afk11:13
*** jbadiapa has joined #oooq11:27
owalshakahat|rover, arxcruz|ruck: hey, seen an issue with a bad namespace in containers-prepare-parameters.yaml?11:30
owalshhttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_742/746999/3/check/tripleo-ci-centos-8-standalone/7421f97/logs/undercloud/home/zuul/containers-prepare-parameters.yaml11:30
*** skramaja has quit IRC11:31
*** pojadhav is now known as pojadhav|away11:31
arxcruz|ruckowalsh: nope, haven't seen this, akahat|rover have you?11:31
akahat|roverowalsh, arxcruz|ruck no.11:31
*** Goneri has joined #oooq11:34
*** ysandeep|afk is now known as ysandeep11:34
akahat|roverowalsh, let's rerun job.. and check we can face the same issue again..11:34
owalshakahat|rover: already did11:35
akahat|roverowalsh, okay11:35
*** ratailor has quit IRC11:36
weshayysandeep, I can promote https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/buildset/655501dc97ef42aa998d17ebf7a11c9c build for you11:50
weshayif that will help11:50
weshay98fc67cafb2c78c488790718bd2bcec011:50
ysandeepweshay, i was able to create container with current-tripleo hash, using https://code.engineering.redhat.com/gerrit/#/c/208803/7/zuul.d/tripleo-build-containers.yaml , things looks fine now11:51
weshayaight11:51
ysandeep98fc67cafb2c78c488790718bd2bcec0 is a good hash too, can be promoted11:51
ysandeepweshay, but i didn't get how that happened, incase you have an idea.11:53
*** jfrancoa has joined #oooq11:55
weshayya.. we'll have to chat about it later..11:57
weshayysandeep, one note.. unrelated11:57
*** rlandy has joined #oooq11:57
weshayI've convinced rel-del to change the way upstream code is imported to internal11:57
weshaythey are no longer taking untested code internally11:57
weshaythey import at promoted components11:58
ysandeepweshay, that's really a good news11:58
weshayrlandy, ysandeep https://trunk.rdoproject.org/centos8-train/promoted-components/delorean.repo  ---> http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/consistent/delorean.repo11:58
*** skramaja has joined #oooq11:58
weshaymost of the repos are lining up11:58
weshayysandeep, rlandy compare the dlrn_hash11:59
rlandychecking11:59
rlandyweshay: is that train only or also master-> 17?11:59
weshayrlandy, master -> 1711:59
weshayrlandy, yes.. he just switched 17 as well.. it make take a few days for the changes to be reflected12:00
rlandyysandeep: I'm reading your email12:00
rlandytrying o see what happened with containers push12:00
rlandyrather than patching - we need to see why conatiners were not pushed12:00
ysandeepweshay, only glance and ui hash are different12:02
ysandeeprlandy, agree , I just patched to go ahead with my component job addition work12:03
rlandyweshay: why did you -2 https://review.opendev.org/#/c/748360/?12:04
rlandyysandeep: I didn't run the promotion script today12:10
*** pojadhav|away is now known as pojadhav12:10
rlandyweshay: did you?12:10
rlandyand yet current-tripleo did move12:10
rlandyit's marked 08/2712:10
rlandyI did that yesterday12:10
weshayrlandy, I did because there were no check/gate jobs for tripleo12:11
weshayrlandy, chandan just added one https://review.opendev.org/#/c/748418/12:11
rlandyweshay: lol - ok you are answering the puppet-oslo question12:12
rlandyweshay: I am asking if you ran the internal promotion script for 16.212:12
ysandeeprlandy, i think finally i have an idea why ovb job failed, overcloud nodes have latest podman12:14
ysandeepError12:14
ysandeep~~~12:14
ysandeeptime="2020-08-26T01:45:50Z" level=error msg="Error forwarding signal 23 to container 0b328de81b8378cf1be9e0d86fc98fd5ee093e13c09e9a78627813ecaed83afc: can only kill running containers. 0b328de81b8378cf1be9e0d86fc98fd5ee093e13c09e9a78627813ecaed83afc is in state stopped: container state improper"12:14
ysandeeppanic: close of closed channel12:14
ysandeep~~~12:14
ysandeephttps://sf.hosted.upshift.rdu2.redhat.com/logs/40/189440/89/check/periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2/35e079c/logs/overcloud-controller-0/var/log/extra/package-list-installed.txt.gz12:14
ysandeep~~~12:14
ysandeeppodman.x86_64                                 1.6.4-13.module+el8.3.0+7385+02cfa547           @rhosp-rhel-8.3-appstream12:14
ysandeep~~~12:14
rlandyysandeep: sorry - one sec12:14
rlandyI am trying to sync up dlrn12:14
weshayrlandy, no12:14
weshayI did not run promote12:14
rlandyand the dockerfiles and conatiners12:14
rlandyhow did thsi rev then??12:14
rlandyif none of us promoted it?12:14
ysandeepsure, i didn't touch anything.. not even login12:15
rlandy2020-08-26 12:38:39.383332 | primary |   "aggregate_hash": "5245879066c657f515264b04381c0a35",12:16
rlandylast hash12:16
*** udesale_ has joined #oooq12:17
rlandyysandeep: weshay: ok - 16.2 is kicking now again12:18
rlandylater this afternoon, I'll rerun promote script12:18
rlandythat should clear it12:19
rlandyor reproduce the same mismatch of hash12:19
rlandy17 is broken now12:19
*** udesale has quit IRC12:19
rlandyysandeep: wrt podman on the overcloud12:19
rlandyyou have a fix or you want to discuss?12:20
*** amoralej is now known as amoralej|lunch12:20
ysandeeprlandy, i don't have fix.. can we somehow workaround image build to push old repo instead?12:22
rlandyysandeep: yes...12:22
rlandylet me add that12:22
rlandyI need to add rhel beta to the repos in image build12:23
rlandybut we only want one package from there12:23
ysandeeprlandy, thank you, sry I didn't get much time to explore working around image build part12:24
rlandyysandeep: no problem .. we need to figure out how to install only one package from a repo in iamge build12:24
rlandyidk atm12:24
rlandyI know how add the repo12:24
rlandychandankumar: hey ^^ do you know how to specify to install one package from a repo in image build?12:25
* rlandy gets 12:25
chandankumarrlandy: where to install that package?12:25
ysandeepchandankumar, overcloud-full image12:26
rlandychandankumar: on ovb, we're building images12:26
rlandyand ysandeep found it's getting the newer podman12:26
rlandydib_yum_repo_conf12:26
rlandy^^ I can add the repo to get old podman12:27
rlandybut we only want podman from that repo ...12:27
rlandylike this ...12:27
rlandychandankumar: something like ...12:27
rlandy    includepkgs:12:28
rlandy      - podman12:28
rlandythat we use when we do repo-setup12:28
chandankumarlet me see12:28
*** ykarel has quit IRC12:28
*** ykarel has joined #oooq12:32
rlandyrhel-binary-neutron-server/manifests/8c12c90c79847e473446edafc780146612:34
rlandyoh dear12:34
rlandyhere we go again on 1712:34
chandankumarrlandy: can you try with include packages12:38
chandankumarand specify the version also12:38
rlandychandankumar: in dib_yum_repo_conf?12:39
rlandyOH12:40
rlandyI get it12:40
rlandychandankumar: sorry12:40
chandankumarrlandy: add the old repo12:40
chandankumarand add append includpkgs part in the repo12:40
chandankumarand then add it dib_yum_Repo_conf12:40
rlandychecking what repo looks like12:40
rlandychandankumar: yeah - sorry - I'm slow this morning :)12:40
rlandyysandeep: ^^ lol ... I need a vacation .. fixing that and rerunning OVB12:41
ysandeeprlandy, hahaha12:42
* rlandy feels stupid12:42
rlandysec - review coming up12:42
ysandeeprlandy, i think we need some further tuning for component job in this patch https://code.engineering.redhat.com/gerrit/#/c/20966612:43
ysandeepwith depends on this patch bm component job is failing with below error:-12:43
ysandeephttps://sf.hosted.upshift.rdu2.redhat.com/logs/85/209585/2/check/periodic-tripleo-ci-rhel-8-bm_envA-3ctlr_1comp-featureset001-baremetal-rhos-16.2/a23b534/logs/undercloud/home/zuul/build-err.log12:43
ysandeep~~~12:43
ysandeepOSError: No config file exists at: /home/zuul/src/code.engineering.redhat.com/openstack-tripleo-common/image-yaml/overcloud-images-python3.yaml12:43
ysandeep~~~12:43
ysandeepintegration line job going good.. fs001 bm intergration also in deploy step - https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/c2479cfb57d54f08a97c06cfb1ea43de?logfile=console.log12:44
rlandyysandeep: yeah - I hadn't update the 16.2 release file as much as the integration line one - working on that patch now12:47
ysandeeprlandy++12:48
*** ysandeep is now known as ysandeep|brb12:50
rlandyopenstack-tripleo-common.git/tree/image-yaml/overcloud-images-python3.yaml?h=rhos-16.2-trunk-patches12:55
rlandyexits12:55
rlandyexists12:55
rlandyysandeep|brb: confused about https://code.engineering.redhat.com/gerrit/#/c/209585/2..3//COMMIT_MSG12:59
rlandybaremetal became multinode?13:00
rlandyopenstack-tripleo-common/image-yaml/overcloud-images-python3.yaml definirely exists13:00
rlandynede to check required projects13:00
weshayrlandy, ysandeep|brb chandankumar zbr scrum13:02
*** ysandeep|brb is now known as ysandeep13:02
ysandeeprlandy, reading13:02
rlandyysandeep: will explain after scrum13:02
weshaysshnaidm, scrum13:02
chandankumarweshay: I will be skipping the meeting13:02
ysandeeprlandy, got it.. my bad13:06
*** ykarel has quit IRC13:07
*** ykarel has joined #oooq13:08
rlandyrekicked OVB13:09
*** amoralej|lunch is now known as amoralej13:11
*** skramaja has quit IRC13:23
rlandyysandeep: ok - I know what the failures on 17 are about - fixing that13:27
rlandypuppet-oslo13:27
ysandeeprlandy, okay, i have fixed my bm job too ..13:30
chandankumarrlandy: https://review.opendev.org/#/c/747259/13:47
chandankumarhttps://review.opendev.org/#/c/747259/6/roles/build-containers/defaults/main.yaml13:47
arxcruz|ruckchandankumar: https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-stein/8d58da1/logs/undercloud/home/zuul/tempest.log.txt.gz13:52
arxcruz|ruckis it related to that missing container ?13:52
ysandeepweshay, okay if i remove https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/standalone/defaults/main.yml#L23 ceph from here?14:00
ysandeeprlandy, ^^14:00
*** TrevorV has joined #oooq14:00
*** rlandy is now known as rlandy|mtg14:00
chandankumarsshnaidm: you are joining the DF call?14:01
sshnaidmchandankumar, yep14:01
chandankumarsshnaidm: ok then I think I can skip it14:01
chandankumarlet me know if any help needed14:01
sshnaidmchandankumar, sure14:02
*** pojadhav is now known as pojadhav|afk14:04
ysandeepweshay, ykarel rlandy|mtg fyi.. https://review.opendev.org/#/c/748449/14:32
ysandeep^^ remove ceph repos from standalone_container_prep_update_repo parameter.14:32
*** ysandeep is now known as ysandeep|away14:34
ykarelack, the change at time of nautilus migration was temporary, but never got reverted14:34
*** ykarel is now known as ykarel|away14:34
ysandeep|awayykarel|away,  thank you14:34
*** jbadiapa has quit IRC14:35
arxcruz|ruckweshay: I'm confused https://review.opendev.org/#/c/748360/14:43
arxcruz|ruckyou -114:43
arxcruz|ruckbut it's on gate now14:43
*** jbadiapa has joined #oooq14:46
*** bogdando has quit IRC14:52
*** apetrich has quit IRC14:54
*** ykarel|away has quit IRC15:06
*** rlandy|mtg is now known as rlandy15:09
weshaychandankumar, hey.. have a sec?15:16
weshaychandankumar, if you do.. join https://meet.google.com/eqy-jekd-pox?authuser=1 for just a min15:16
*** amoralej is now known as amoralej|off15:17
chandankumarweshay: joining15:17
zbrweshay: bit of fun, 4 years old review for adding support for jira in e-r, now we can make it happen ;)15:24
zbrhttps://review.opendev.org/#/c/385217/15:25
zbrand the code is not bad, includes tests15:25
weshayarxcruz|ruck, you need anything?15:35
arxcruz|ruckweshay: no15:38
rlandychandankumar: hey ... do you know where this is picking up podman/podman-docker from: ie: where is it specified? https://sf.hosted.upshift.rdu2.redhat.com/logs/40/189440/89/check/periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2/ec6421e/logs/undercloud/home/zuul/build.log.txt.gz15:47
chandankumarrlandy: https://opendev.org/openstack/tripleo-puppet-elements/src/branch/master/elements/overcloud-base/pkg-map#L615:48
rlandychandankumar: thanks15:48
*** cgoncalves has joined #oooq16:10
*** ysandeep|away is now known as ysandeep16:14
*** derekh has joined #oooq16:21
*** udesale_ has quit IRC16:23
*** dtantsur is now known as dtantsur|afk16:23
*** sshnaidm is now known as sshnaidm|afk16:24
*** tosky has quit IRC16:45
*** derekh has quit IRC16:47
*** pojadhav|afk is now known as pojadhav|away17:14
rlandyweshay: --agg-hash f31fc1ea72ab1f45eb1b56d06e3a1640 --job-id periodic-tripleo-ci-centos-8-standalone-centos8cr-container-build-master --info-url https://logserver.rdoproject.org/25/25325/40/check/periodic-tripleo-ci-centos-8-standalone-centos8cr-container-build-master/a1241c0 --timestamp 1598550399 --success True17:50
rlandy^^ dependency job with container build - passing17:50
weshayah nice17:53
* weshay looks17:53
rlandytestproject - will run in the pipeline on saturday18:10
*** holser has quit IRC18:33
*** holser has joined #oooq18:34
*** tosky has joined #oooq18:35
rlandyweshay: hey - want to meet to review promotions/container rename?19:00
weshayrlandy, yes.. working w/ damian for a minute in #tripleo.. need a few19:01
rlandyweshay: no rush19:01
weshayrlandy, can I borrow you for a forced promotion19:15
rlandyweshay: ack19:15
weshayand then we'll talk about names19:15
rlandytmux/ gmeet?19:16
weshayrlandy, https://meet.google.com/dxk-rbzn-hvv?authuser=119:17
weshayrlandy, aye19:17
weshayrlandy, https://trunk.rdoproject.org/api-centos8-master-uc/api/civotes_agg_detail.html?ref_hash=058c2f80d238fc694e979d1a5525dcac19:22
weshayrlandy,    commit_hash: 7a72d2a3e09ac23383c403af7136bd8b93b82c5619:22
weshayrlandy,    distro_hash: 35fa25a15365683e192f0fb2cbd9969f9a3b5af519:23
weshayhttps://trunk.rdoproject.org/centos8-master/component/validation/7a/72/7a72d2a3e09ac23383c403af7136bd8b93b82c56_35fa25a1/commit.yaml19:23
*** jfrancoa has quit IRC19:39
rlandyweshay: you drop?19:44
weshayrlandy, sorry.. computer got too hot19:46
weshayrlandy, back in19:46
rlandyrejoining19:46
rlandyweshay: https://review.rdoproject.org/r/#/c/29078/1/ci-scripts/dlrnapi_promoter/repo_client.py19:49
*** TrevorV has quit IRC19:54
weshayrlandy, ugh19:59
weshaycoming back20:00
weshayrlandy, ugh.. it's me20:41
*** jtomasek has quit IRC20:41
rlandyweshay: https://meet.google.com/dxk-rbzn-hvv?authuser=120:41
weshayrlandy, sec.. chrome issues20:42
weshaygetting 404's20:42
rlandyoh gosh20:42
weshayso weird20:42
weshayrlandy, coming20:43
*** matbu has quit IRC20:54
*** jbadiapa has quit IRC20:55
*** matbu has joined #oooq20:56
rlandyweshay: still failed21:28
rlandy"stderr": "no such manifest: trunk.registry.rdoproject.org/tripleomaster/centos-binary-sahara-engine:058c2f80d238fc694e979d1a5525dcac", "stderr_lines": ["no such manifest: trunk.registry.rdoproject.org/tripleomaster/centos-binary-sahara-engine:058c2f80d238fc694e979d1a5525dcac"],21:29
*** saneax has quit IRC21:32
*** Goneri has quit IRC21:39
rlandyrestarted22:11
*** holser has quit IRC22:33
*** holser has joined #oooq22:56
*** saneax has joined #oooq23:02
*** sanjayu_ has joined #oooq23:10
*** saneax has quit IRC23:13
weshayrlandy, promoted \0/23:18
*** sanjayu_ has quit IRC23:54
*** tosky has quit IRC23:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!