Friday, 2021-03-05

*** tosky has quit IRC00:11
*** chem has joined #oooq00:33
*** jmasud has quit IRC00:38
*** jmasud has joined #oooq00:58
*** rlandy|bbl is now known as rlandy02:31
rlandyweshay|ruck: thanks02:31
*** irclogbot_3 has quit IRC02:52
*** irclogbot_3 has joined #oooq02:54
rlandychandankumar: 16.2 is looking good so far  - may be able to promote there03:00
rlandychandankumar: pls review https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/778561 ... needed to get BM passing03:02
*** rlandy has quit IRC03:02
*** jmasud has quit IRC03:33
*** saneax has joined #oooq03:57
*** jmasud has joined #oooq04:28
*** ratailor has joined #oooq04:37
*** ykarel has joined #oooq04:38
*** ratailor_ has joined #oooq04:59
*** ratailor__ has joined #oooq05:02
*** ratailor has quit IRC05:03
*** ratailor_ has quit IRC05:06
*** ykarel has quit IRC05:50
*** ykarel has joined #oooq05:53
*** udesale has joined #oooq05:55
*** marios has joined #oooq06:07
*** ykarel_ has joined #oooq06:08
*** ykarel has quit IRC06:10
*** ykarel_ is now known as ykarel06:10
*** slaweq has joined #oooq06:59
*** jpodivin has joined #oooq07:52
*** jmasud has quit IRC08:14
*** jmasud has joined #oooq08:28
bhagyashri|roverarxcruz, how the containers are pushed to quay.io08:42
bhagyashri|roverarxcruz, are they checking this hash https://trunk.rdoproject.org/centos7-stein/current-tripleo/delorean.repo  before pushing the containers?08:44
*** jpena|off is now known as jpena08:54
*** derekh has joined #oooq09:14
marioschandankumar: please votes when you have time https://review.opendev.org/q/ca603f39b6d612fd2950cd7d7eb84ae3e2cecff909:16
mariosand anyone else around please ^^^ cherrypick v/u/t09:16
*** tosky has joined #oooq09:23
mariosthank you chandankumar09:42
akahatykarel, o/09:44
akahatykarel, could you please hold node. review: https://review.rdoproject.org/r/#/c/32069/ , Job: tox-py36-ci-config09:45
akahatssh public key github.com/amolkahat.keys add any key09:45
akahatthanks :)09:45
akahatchandankumar, ^^09:47
mariossudo get me a coffee09:49
chandankumarcoffee command not found09:51
marios:( /me gets up goes to kitchen09:51
chandankumarhttps://xkcd.com/378/09:53
marioshaha chandankumar :)09:54
chandankumarakahat: ^^ for you09:55
chandankumarjpena: hello, can you hold the node for tox-py36-ci-config  https://review.rdoproject.org/r/#/c/32069/ and github.com/amolkahat.keys09:56
chandankumarjpena: thanks!09:56
jpenachandankumar: done, ssh zuul@38.102.83.2010:00
chandankumarakahat: ^^10:01
chandankumarjpena: thanks :-)10:01
akahatchandankumar, jpena thanks :)10:01
ykarelakahat, just back from lunch, but seems ur task is done :)10:03
*** jmasud has quit IRC10:05
akahatykarel, yeah. :)10:05
*** holser has quit IRC10:21
*** saneax has quit IRC10:36
*** udesale has quit IRC10:59
*** udesale has joined #oooq10:59
bhagyashri|roverarxcruz, around?11:13
arxcruzbhagyashri|rover: yes11:13
arxcruzsup11:13
arxcruzbhagyashri|rover: omg, sorry, did not saw your message earlier11:13
arxcruzbhagyashri|rover: wanna sync this ?11:13
bhagyashri|roveryeah that would be great11:14
bhagyashri|roverarxcruz, you free?11:14
arxcruzbhagyashri|rover: yup11:14
bhagyashri|roverok11:14
bhagyashri|roverhttps://meet.google.com/toq-qijq-oft11:16
bhagyashri|roverarxcruz, ^11:16
arxcruzbhagyashri|rover: just a sec, it seems i'm having issues to connect11:16
bhagyashri|roverok11:16
*** ykarel_ has joined #oooq11:19
*** ykarel has quit IRC11:22
*** ykarel_ is now known as ykarel11:23
*** holser has joined #oooq11:35
*** holser has quit IRC11:37
*** holser has joined #oooq11:38
*** holser has quit IRC11:57
*** holser has joined #oooq12:01
chandankumarsteeping out for a bit12:03
*** ykarel_ has joined #oooq12:07
*** ykarel has quit IRC12:09
*** derekh has quit IRC12:13
*** rlandy has joined #oooq12:33
*** jpena is now known as jpena|lunch12:34
*** ratailor__ has quit IRC12:36
rlandychandankumar: hey12:38
rlandywhat happened with the promoter?12:39
chandankumarrlandy: I not have kicked it yet12:41
chandankumarrlandy: do you want to start?12:42
chandankumarrlandy: does for rhos-16.2 no hashes are yet available?12:46
*** frenzy_friday has quit IRC12:46
chandankumarrlandy: http://10.0.11.190/promoter_logs/rhel8_rhos-16.2.log12:46
chandankumarpipeline_integration-pcci-16.2_dlrn-rhel-8.4-virthost-3cont_2comp_3ceph-ipv4-geneve-ceph_rcj', 'periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2', 'pipeline_integration-pcci-16.2_dlrn-rhel-8.4-virthost-3cont_2comp_3ceph-ipv6-geneve-ceph_rcj'}12:47
chandankumarthese are missing jobs12:48
rlandychandankumar: let me check those two ...12:48
chandankumarrlandy: internal promoter is on, so any new hash will be available it will automatically pick and promote it12:48
rlandychandankumar: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/buildset/94afe34b7b3a46e1b939a10cf8257195 is a good buildset12:50
rlandythe patch you w+'ed will fix bm here12:50
rlandyfs001 is fine is scenario passed12:50
rlandylet me check QE tests12:50
bhagyashri|roverweshay|ruck, rlandy all periodic master jobs are passing12:51
rlandygreat12:51
bhagyashri|roverweshay|ruck, on c8 promoter currently i just enabled the master promotion12:51
rlandyhttps://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/QE/view/OSP16.2/job/phase1-16.2_compact-director-rhel-8.4-virthost-1cont_1comp_1ceph-ipv4-geneve-ceph/12:52
rlandychandankumar: idk - they are not 8.4 yet12:52
bhagyashri|rover**all periodic c8 master job**12:52
chandankumarrlandy: do we want to remove those jobs and promote it?12:52
chandankumarnot sure it will break the stufff12:52
rlandyhold on - let's ask12:53
chandankumarok12:53
rlandychandankumar: there is a hash in test now but the last one could go if we changed the criteria12:54
rlandylet's give attila a moment to respond then we can promote12:55
chandankumarrlandy: sure12:55
rlandychandankumar: the same hash is running on the current line13:08
rlandyweshay|ruck: ^^ are you ok with our promoting 16.2 removing the QE tests from criteria?13:08
weshay|ruckrlandy, which hash?13:09
weshay|ruckrlandy, chandankumar 16.2 infrared jobs are passing deployment, but failing here: http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/pipeline_integration-pcci-16.2_dlrn-rhel-8.4-virthost-3cont_2comp_3ceph-ipv4-geneve-ceph/12/undercloud-0/home/stack/tempest-dir/tempest-results-sanity.1.xml.gz13:10
rlandy56b5a98c2ed5fb374f7323a6fc66cd7013:10
weshay|ruckchandankumar, making the case about sharing tempest is getting stronger13:11
weshay|ruckrlandy, k.. /me looks sec13:11
rlandyweshay|ruck: ^^ it's a good buildset13:11
chandankumarweshay|ruck: awesome :-)13:11
weshay|ruckrepo_baseid: b651480887c76a18e5f538361f0ff82413:11
rlandyBM fix merged today13:11
rlandy?13:11
rlandyrepo_baseid: b651480887c76a18e5f538361f0ff824??13:11
rlandy2021-03-05 00:13:31.720729 | TASK [get_hash : read md5]13:11
rlandy2021-03-04 19:13:32.462452 | primary | 56b5a98c2ed5fb374f7323a6fc66cd7013:11
weshay|ruckI'm just letting attila know13:12
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/buildset/94afe34b7b3a46e1b939a10cf825719513:14
rlandyweshay|ruck: yep - I pinged him on openstack-pcci13:14
weshay|ruckrlandy, promote13:14
rlandyto tell him we wanted to try promote13:14
rlandychandankumar: cool - let's go for it13:14
chandankumarlet jump on promoter13:14
rlandychandankumar: is there a tmux?13:15
rlandyI am on as centos now13:15
weshay|ruckrlandy, su - promoter13:15
chandankumarweshay|ruck: rlandy https://meet.google.com/mfr-bkwr-mjn?authuser=0&hl=en13:16
rlandyok13:16
*** frenzy_friday has joined #oooq13:16
zbrweshay|ruck: there is a fresh thread going on service-discuss@ related to (lack of) reviews, you may want to mention something.13:19
weshay|ruckaye.. see it13:19
zbrmaybe some good things will come out of it13:20
zbri am sure there is no generic solution and approach may need to be different based on the project/group.13:20
rlandychandankumar: http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/tripleo-ci-testing/13:24
weshay|ruckhttp://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp16-2/tripleo-ci-testing/versions.csv13:25
*** jpena|lunch is now known as jpena13:28
arxcruzjpena: is there an endpoint on dlrn that bring only the actual hash ?13:31
arxcruzi tried https://trunk.rdoproject.org/api-centos8-master-uc/api/promotions?promote_name=current-tripleo for example, but it brings me per component13:32
jpenaarxcruz: the actual hash for what exactly?13:32
jpenaI mean13:32
arxcruzjpena: for example, i want to check which hash i should copy from trunk.rdo and/or docker.io to quay.io13:32
jpenafor current-tripleo, we have one hash per component, yes, but if you want the aggregate_hash we will have to look in a different place13:32
arxcruzjpena: okay, what about this aggregate_hash? do we have an api to bring it ?13:33
jpenaarxcruz:so https://trunk.rdoproject.org/api-centos8-master-uc/api/promotions?promote_name=current-tripleo&limit=1 will give you the latest promotion13:34
jpenait contains aggregate_hash13:34
jpenahttps://github.com/softwarefactory-project/DLRN/blob/master/doc/source/api.rst has the full API spec13:35
arxcruzjpena: thanks, i wasn't sure about the terminology13:36
arxcruzthat's exactly what i wanted :)13:36
*** amoralej is now known as amoralej|lunch13:47
rlandyweshay|ruck: frenzy_friday: sent stream sync meeting invite14:00
frenzy_fridayrlandy, thanks14:03
frenzy_fridayThere are some jobs which failed from ussuri and train integration lines. Checking why14:03
rlandyfrenzy_friday: do you need help there?14:08
frenzy_fridayrlandy, the same jobs failed in the integration line yesterday , lemme compare the failure and I'll ping you14:10
rlandyok14:19
chandankumarrlandy: weshay|ruck fixed that error, now hitting with registry issue14:23
chandankumarhttp://10.0.11.190/promoter_logs/rhel8_osp16-2.log14:23
*** amoralej|lunch is now known as amoralej14:24
*** ykarel_ is now known as ykarel14:32
*** jfrancoa has joined #oooq14:37
*** jfrancoa has quit IRC14:42
chandankumarrlandy: hello, in downstream we have container names starting with redhat-binary ?14:45
chandankumarrlandy: is it openstack- na?14:47
chandankumarrlandy: http://10.0.11.190/promoter_logs/container-push/20210305-144056.log14:48
chandankumarrlandy: container push started14:48
weshay|ruckchandankumar, it should be really fast14:49
chandankumarweshay|ruck: hopin in on call?14:49
weshay|ruckwhich?14:49
weshay|ruckchandankumar, ^14:50
chandankumarweshay|ruck: sorry, want to jump on the same gmeet14:51
chandankumarweshay|ruck: https://meet.google.com/mfr-bkwr-mjn?authuser=0&hl=en14:51
frenzy_fridayrlandy, the errors dont seem like stream issues, I can see them in few of the past runs, but they dont occur always14:58
frenzy_fridayhttps://logserver.rdoproject.org/59/32159/4/check/periodic-tripleo-ci-centos-8-scenario010-ovn-provider-standalone-ussuri/cb5e7b2/logs/undercloud/var/log/extra/errors.txt.txt.gz , https://logserver.rdoproject.org/60/32160/6/check/periodic-tripleo-ci-centos-8-scenario010-ovn-provider-standalone-train/005550c/logs/undercloud/var/log/extra/errors.txt.txt.gz14:58
weshay|ruckbhagyashri|rover, akahat is stein still blocked?15:07
bhagyashri|roverweshay|ruck, want to chat with you15:08
weshay|ruckk.. not avail atm.. can you irc?15:08
bhagyashri|roverweshay|ruck, stein promotion is going on ... I had chat with arxcruz and we stopped the copy-quay and now i am trying to promote the stein with new promoter code15:09
bhagyashri|roveri have new hash15:09
bhagyashri|roverso i am checking ... is the new promoter code pushing container to quay.io or not15:11
bhagyashri|roverweshay|ruck, http://10.0.111.180/promoter_logs/container-push/20210305-141846.log15:12
bhagyashri|roverweshay|ruck, but i observed one thing that the all the periodic master jobs are passing still there is issue/failure to push the containers15:13
bhagyashri|roverhere is the master container push log http://10.0.148.74/promoter_logs/container-push/20210305-150057.log ---- Failing15:18
rlandyweshay|ruck: https://review.opendev.org/c/openstack/tripleo-ci/+/77171715:20
*** ksambor has quit IRC15:29
*** ykarel has quit IRC15:37
*** udesale has quit IRC15:43
weshay|ruckbhagyashri|rover, do you know why it's failing?15:51
rlandyweshay|ruck: pls review before email goes out: http://pastebin.test.redhat.com/94501415:58
chandankumarrlandy: hello16:01
chandankumarrlandy: for image push we need to make so many changes16:01
chandankumarrlandy: the dir structure is totally different from upstream on image server16:02
weshay|ruckrlandy, you may want to speak to the pre-migration testing to give folks some insight that we're not dropping this w/o validation16:03
rlandychandankumar: weshay|ruck: should we just promote the old way today and give you guys some time to work on this?16:03
weshay|ruckfine by me16:04
rlandychandankumar: ok - take your time16:04
chandankumarrlandy: I can promote the container via new way if ok?16:05
rlandychandankumar: hmmm ... might mix up the old way16:06
rlandylet's juts stick to one at a time16:06
rlandyif we promote today, you get a whole week to get this right16:06
rlandychandankumar: weshay|ruck: ^^ ok? I am logging on to the old instance to promote the old way16:07
chandankumarrlandy: we can test the promoter logic for downstream this way16:09
chandankumarrlandy: you can retag the images manually logging into the image server16:09
chandankumarif ok?16:09
chandankumarrlandy: anyway is fine16:11
rlandychandankumar: weshay|ruck: I am ok with it - if ou guys ca fux any falout16:11
rlandyI am out next week - should be ok16:11
chandankumarrlandy: go via old way16:11
rlandylet me check the image scripts sec16:11
rlandychandankumar: hold on16:12
chandankumarrlandy: I need to fix the imae script16:12
bhagyashri|roverweshay|ruck, for master there is no space available16:12
bhagyashri|roverweshay|ruck, 2021-03-05 15:35:49,693 p=3805059 u=promoter n=ansible | failed: [localhost] (item=zaqar-wsgi) => {"ansible_loop_var": "item", "changed": false, "item": "zaqar-wsgi", "msg": "Error pulling trunk.registry.rdoproject.org/tripleomaster/openstack-zaqar-wsgi - code: None message: failed to register layer: Error processing tar file(exit status 1): open16:12
weshay|ruckbhagyashri|rover, did prune run?16:12
bhagyashri|roverweshay|ruck, not checked16:13
weshay|ruckbhagyashri|rover, run it by hand.. enable it in the script.. dlrn-promoter.sh..  restart16:13
bhagyashri|roverweshay|ruck, ok16:14
weshay|ruckbhagyashri|rover, make sure space is freed up before restarting16:14
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/dockerfiles/promote_16_2.sh16:14
rlandychandankumar: ^^16:14
rlandyshould be fine ...16:14
rlandyyou can use the new way to promote containers16:14
chandankumarrlandy: ok doing that16:14
rlandywe can promote images and dockerfiles afterwards16:14
rlandywith scripts16:14
rlandylet me know when conatiners are done16:15
chandankumarrlandy: dlrn secret issue16:18
chandankumarrlandy: promoter Exception while promoting hashes to API endpoint (401) UNAUTHORIZED: Unauthorized Access16:18
bhagyashri|roverweshay|ruck, i see in pdlrn-promoter.sh this is commented16:19
bhagyashri|roveri remember you commented that16:19
weshay|ruckbhagyashri|rover, right.. uncomment, rerun16:19
weshay|ruckrerun the command by hand16:19
rlandyk16:19
weshay|ruckrestart the promoter16:19
bhagyashri|roverweshay|ruck, sure16:19
bhagyashri|roverweshay|ruck, done16:25
weshay|ruck:)16:25
bhagyashri|roverweshay|ruck, for stein16:27
bhagyashri|roverhttp://10.0.111.180/promoter_logs/container-push/20210305-141846.log16:27
bhagyashri|rover2021-03-05 14:57:00,445 p=2937244 u=promoter n=ansible | failed: [localhost] (item=neutron-server-opendaylight) => {"ansible_loop_var": "item", "changed": true, "cmd": "echo -n \"Checking trunk.registry.rdoproject.org/tripleostein/centos-binary-neutron-server-opendaylight:ff6d3f7d5a8f34c470250caf68999a67454bd149_8d9f1861 ... \";\ndocker manifest inspect    --insecure \"trunk.registry.rdoproject.org/tripleostein/centos-binary-neutr16:28
bhagyashri|roveron-server-opendaylight:ff6d3f7d5a8f34c470250caf68999a67454bd149_8d9f1861\"\nif [[ \"$?\" == \"0\" ]]; then\n    echo neutron-server-opendaylight >> /tmp/parsed_containers-centos-7-stein.txt;\n    echo \"OK\";\nelse\n    echo \"FAIL\";\n    echo \"ERROR ========== centos-binary-neutron-server-opendaylight does not exists on registry, Please check the latest tripleo-ci-testing hash! ==========\";\n    exit 1\nfi\n", "delta":16:28
bhagyashri|rover"0:01:00.786110", "end": "2021-03-05 14:57:00.419095", "item": "neutron-server-opendaylight", "msg": "non-zero return code", "rc": 1, "start": "2021-03-05 14:55:59.632985", "stderr": "name unknown", "stderr_lines": ["name unknown"], "stdout": "Checking trunk.registry.rdoproject.org/tripleostein/centos-binary-neutron-server-opendaylight:ff6d3f7d5a8f34c470250caf68999a67454bd149_8d9f1861 ... FAIL\nERROR ========== centos-binary-neutr16:28
bhagyashri|roveron-server-opendaylight does not exists on registry, Please check the latest tripleo-ci-testing hash! ==========", "stdout_lines": ["Checking trunk.registry.rdoproject.org/tripleostein/centos-binary-neutron-server-opendaylight:ff6d3f7d5a8f34c470250caf68999a67454bd149_8d9f1861 ... FAIL", "ERROR ========== centos-binary-neutron-server-opendaylight does not exists on registry, Please check the latest tripleo-ci-testing hash!16:28
bhagyashri|rover=========="]}16:28
weshay|ruckopendaylight should be skipped16:28
bhagyashri|rovercentos-binary-neutron-server-opendaylight container is not present in trunk registry16:28
chandankumarrlandy: go via old way16:28
chandankumarrlandy: there is an auth issue with dlrn api16:29
chandankumarneed to debug16:29
chandankumarmay be food for friday16:29
chandankumarmay be food for monday16:29
chandankumarrlandy: weshay|ruck we also need to solve release name dot mess16:29
weshay|ruckchandankumar, aye16:31
chandankumarweshay|ruck: for downstream we need to keep a seperate promoter as we need to configue rh cert otherwise docker login does not work16:32
weshay|ruckchandankumar, yes16:37
weshay|ruckchandankumar, that was the plan16:37
*** jpodivin has quit IRC16:37
bhagyashri|roverweshay|ruck, yes opendaylight is skipped. but it's neutron-server-opendaylight that is not present on trunk registry16:38
weshay|ruckbhagyashri|rover, anything opendaylight should be skipped16:38
* weshay|ruck gets16:38
*** jmasud has joined #oooq16:38
bhagyashri|roverweshay|ruck, ok16:39
weshay|ruckbhagyashri|rover, https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/vars/main.yaml#L3316:39
bhagyashri|roverweshay|ruck, not sure why it's not getting skipped16:39
weshay|ruck - neutron-server-opendaylight16:39
weshay|ruckbhagyashri|rover, because it's not configured to16:40
weshay|ruckbhagyashri|rover, get me the full list of containers.. if anything other than neutron-server-opendaylight is failing16:40
weshay|ruckakahat, ^16:40
bhagyashri|roverweshay|ruck, ok got it , i see this https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/vars/main.yaml#L33 before, but i was in consideration that opendaylight is separate container16:41
bhagyashri|roverweshay|ruck, if you see here http://10.0.111.180/promoter_logs/container-push/20210305-141846.log that is the only container which is failing16:42
weshay|ruckzbr, is ansible-test really molecule underneath or $new16:44
zbrnope16:44
weshay|ruckman.. we have a lot of job security16:45
zbrboth are different tools, with different targets. ansible-test for modules, molecule for playbooks/roles.16:45
zbrwith no plans for one to displace the other, yet.16:46
zbrindeed16:46
weshay|ruckzbr, https://images.app.goo.gl/chPrQE7X772uPw9A616:46
zbrweshay|ruck: i am personally not happy with so many options, either. it is more complex than it appears.16:47
weshay|ruckvunderful16:47
zbrthe only chance for them to merge will be if ansible-test gets extracted out core (as you can imagine there is no much drive on the other side for making this happen)16:48
weshay|ruckso test is part of core16:49
weshay|ruckvunderful16:49
zbryes, which means that even if someone adds features that molecule has, it will be a different ansible-test for each version of ansible16:49
zbrso you will never get a missing feature needed to improve tested with an older but supported version of ansible (due to known support policy)16:50
zbrI already talked with pdm about that, but no conclusions yet.16:50
zbrweshay|ruck: on monday i will mostly unavailble, there is the Ansible Contributors Summit plus hospital appt, them already overlapping.16:52
weshay|ruckk16:58
mariosweshay|ruck: looks like we might actually get that merged https://review.opendev.org/c/openstack/releases/+/774244 "any time now" ;)17:00
mariosweshay|ruck: at least we have 2 votes now if herve revotes but i can ping him monday might be late for him (I updated it for comments from the other reviewer a little while ago wiping herve vote)17:01
rlandychandankumar: no - I will promote manaually when you are done17:02
rlandyhttps://e1eca528cc456b245ec6-0c809ad541effded46e0a56bdf657f92.ssl.cf2.rackcdn.com/778783/2/check/tripleo-build-containers-ubi-8/846e664/logs/index.html17:03
rlandyweshay|ruck: frenzy_friday: ok - volume mounts work17:03
*** jmasud has quit IRC17:03
*** marios is now known as marios|out17:09
bhagyashri|roverweshay|ruck, master promoted17:13
bhagyashri|roverhttps://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo.md517:14
bhagyashri|roverrlandy, ^17:16
weshay|ruckbhagyashri|rover, nice17:17
bhagyashri|roverweshay|ruck, rlandy if want me to stop the promotion for master c8 then i will make the changes on promoter server17:17
bhagyashri|roverif you want *17:17
chandankumarjpena: Hello17:19
jpenahi chandankumar17:19
chandankumarjpena: is there a way to check dlrnapi client is working fine?17:19
bhagyashri|roverweshay|ruck, fyi for stein promotion i did some hack in the code (removed the neutron-server-opendaylight form container_list)17:19
bhagyashri|roverweshay|ruck, i will push the patch to correct the name here https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/vars/main.yaml#L3317:20
jpenachandankumar: what do you mean by "working fine"? You can always run a query against a known read-only endpoint and see if it returns a result17:20
chandankumarjpena: http://10.0.11.190/promoter_logs/rhel8_osp16-2.log - promoter Exception while promoting hashes to API endpoint (401) UNAUTHORIZED: Unauthorized Access17:20
jpenachandankumar: that is pointing to wrong credentials17:21
jpenachandankumar: wait, are you promoting on a public server or inside the VPN?17:22
chandankumarjpena: yes17:22
chandankumarjpena: I am working on downstream promoter17:22
*** marios|out has quit IRC17:22
chandankumarjpena: inside the vpn17:23
chandankumartalking to internal dlrn server17:23
*** mjturek has quit IRC17:23
*** jmasud has joined #oooq17:24
bhagyashri|roverweshay|ruck, plz review when you have free time https://docs.google.com/document/d/1qJ04D0s7ybH4adTv-aXhZV9cHWFL6-LAtgZbp20GH9U/edit17:25
weshay|ruckbhagyashri|rover, nice work on the master promotion!!!17:34
weshay|ruck0 gate fails17:34
weshay|ruckhttp://dashboard-ci.tripleo.org/d/Z4vLSmOGk/cockpit?orgId=117:34
weshay|ruckLOOKING GOOD PEOPLE!17:35
weshay|ruckbhagyashri|rover, rlandy and I will turn off master promotions this afternoon17:35
weshay|ruckbhagyashri|rover, do you know by chance if manifest_push worked on master?17:35
weshay|ruckakahat, ^17:35
weshay|ruckor was that turned off17:35
* weshay|ruck looks17:35
bhagyashri|roverweshay|ruck, yes that was turned off17:36
weshay|ruckah.. turned off17:36
*** amoralej is now known as amoralej|off17:39
rlandychandankumar: let me know if/when you want me to kick the old promotion or if you are still working17:51
chandankumarrlandy: go ahead with old promoter17:51
chandankumarrlandy: something wrong with creds17:52
rlandychandankumar: k - thanks17:52
rlandyweshay|ruck: ^^ fyi - running old promoter for 16.217:52
rlandyoy - what happened here?17:53
rlandydid someone edit the old scripts?17:53
chandankumarrlandy: I am just copied the secrets from there to new promoter17:54
rlandychandankumar: np - probably just saw your edits17:55
*** mjturek has joined #oooq17:55
*** jpena is now known as jpena|off17:58
rlandyok - 16.2 is promoting18:01
*** irclogbot_3 has quit IRC18:03
*** irclogbot_0 has joined #oooq18:04
chandankumarrlandy: cool18:14
chandankumarrlandy: I will try with rhos-1718:14
chandankumarrlandy: when you will be back from vacations ?18:14
chandankumarrlandy: happy vacations in advance :-)18:14
*** irclogbot_0 has quit IRC18:24
*** irclogbot_1 has joined #oooq18:27
*** holser has quit IRC18:42
rlandychandankumar: one week18:42
rlandythank you18:42
rlandyI definitely need some time off :)18:42
*** holser has joined #oooq18:42
chandankumarrlandy: donot forgot to make pics :-)18:45
chandankumar*take18:45
frenzy_fridayrlandy, weshay|ruck all the 4 job reruns from train and ussuri failed again18:45
frenzy_fridayhttps://review.rdoproject.org/r/#/c/32063/18:46
rlandylooking18:49
rlandy2021-03-05 17:19:36.091732 | primary | async task did not complete within the requested time - 5700s18:55
rlandyfrenzy_friday: ^^ had a timeout18:55
rlandy2021-03-05 17:23:29.767637 | primary | async task did not complete within the requested time - 5700s18:56
rlandyall timing out18:56
frenzy_fridayyes, it is the same timeout since last 3 times18:56
frenzy_fridayERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to tcp:192.168.24.1:6641 to retrieve schema: Connection refused18:56
frenzy_fridayAnd Could not load 'glance.store.s3.Store': No module named 'boto3': ModuleNotFoundError: No module named 'boto3'18:58
frenzy_fridayunable to parse the registries configuration (/etc/containers/registries.conf): mixing sysregistry v1/v2 is not supported"18:58
rlandy ERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to tcp:192.168.24.1:6641 to retrieve schema: Connection refused19:04
rlandy^^ problem19:04
rlandyovn19:04
rlandyfrenzy_friday: there were a bunch of changes on octavia and train on 03/0119:10
rlandyI don;t think this is stream related19:10
rlandyyou can discuss with bhagyashri|rover and CIx19:10
frenzy_fridayrlandy, ok19:15
*** jmasud has quit IRC19:46
rlandyweshay|ruck: frenzy_friday: want to meet again?19:59
rlandymerge release file change to integration?19:59
rlandytwo patches have merged19:59
rlandyintegration master line is still running20:00
*** jmasud has joined #oooq20:01
*** slaweq has quit IRC20:17
weshay|ruckrlandy, ya.. I'm avail20:19
rlandyweshay|ruck: ok - let's do this20:20
weshay|ruckk20:21
rlandymeet.google.com/fwn-mjix-xbh20:21
*** jmasud has quit IRC20:49
*** jmasud has joined #oooq20:57
*** jmasud has quit IRC21:14
*** jmasud has joined #oooq21:57
rlandyweshay|ruck: logging out ... https://review.opendev.org/c/openstack/tripleo-quickstart/+/774911 not in yet - will check in saturday night22:19
*** rlandy has quit IRC22:19
*** jmasud has quit IRC23:02
*** jmasud has joined #oooq23:05
*** jmasud has quit IRC23:30
*** jmasud has joined #oooq23:37

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!