| *** rlandy|ruck|biab is now known as rlandy|ruck | 00:07 | |
| *** bhagyashris__ is now known as bhagyashris | 04:41 | |
| *** marios is now known as marios|ruck | 05:20 | |
| *** pojadhav|afk is now known as pojadhav | 05:29 | |
| soniya29|rover | marios|ruck, I am facing certain issues while connecting with vpn | 05:51 |
|---|---|---|
| marios|ruck | soniya29|rover: k there was a global dns issue yesterday so maybe related | 05:52 |
| marios|ruck | soniya29|rover: which server you trying to conne | 05:52 |
| marios|ruck | ... | 05:52 |
| marios|ruck | :D | 05:52 |
| soniya29|rover | marios|ruck, I tried connecting to Pune, Amsterdam, Beijing and even Red Hat Global VPN | 05:54 |
| soniya29|rover | marios|ruck, but not luck | 05:54 |
| marios|ruck | soniya|rover: same? anyway don't worry about vpn for now unless you need somethign d/stream or internal? | 06:08 |
| marios|ruck | anyone else in india facing the same issue? pojadhav bhagyashris chandankumar o/ soniya|rover has problems with vpn today | 06:08 |
| soniya|rover | marios|ruck, yeah, same. I have raised a ticket for it | 06:08 |
| * marios|ruck coffee brb | 06:08 | |
| marios|ruck | soniya|rover: fyi new gate blocker https://bugs.launchpad.net/tripleo/+bug/1937333 but resolution in progress | 06:08 |
| pojadhav | marios|ruck, mine is connected from yesterday :D, hope so till now not disconnected.. | 06:10 |
| soniya|rover | marios|ruck, okay | 06:10 |
| ykarel | soniya|rover, Pune vpn seems have issues | 06:12 |
| ykarel | try different one, /me connected with singpore currently | 06:12 |
| soniya|rover | ykarel, okay, give me a moment, I will just try it | 06:14 |
| soniya|rover | ykarel, no luck with singapore as well | 06:18 |
| soniya|rover | ykarel, I tried 3 times | 06:18 |
| ykarel | soniya|rover, then possibly some issue with your local network | 06:18 |
| ykarel | try changing to some other | 06:19 |
| soniya29|rover | ykarel, tried with mobile hotspot but same issues | 06:29 |
| bhagyashris | marios|ruck, yeah pune VPN have problem i can connect to Amsterdam | 06:29 |
| marios|ruck | bhagyashris: thanks i think soniya29|rover already tried ams | 06:30 |
| soniya29|rover | marios|ruck, bhagyashris, yes, recently tried again..but no success | 06:31 |
| marios|ruck | zbr: ready to merge? https://review.opendev.org/c/openstack/tripleo-repos/+/801621/10#message-9de745c8952981b3b16b7606ef24044a903221c1 | 06:48 |
| marios|ruck | arxcruz: https://review.opendev.org/c/openstack/validations-libs/+/801623/4#message-f8aa96f56bb12defdd866189ad0a495b72ba31b4 :) | 06:55 |
| *** amoralej|off is now known as amoralej | 06:56 | |
| *** marios|ruck is now known as marios | 07:01 | |
| *** marios is now known as marios|ruck | 07:05 | |
| *** marios_ is now known as marios | 08:06 | |
| zbr | marios|ruck: morning! yep, just added +W. | 08:07 |
| *** marios is now known as marios|ruck | 08:08 | |
| zbr | btw, are you aware of some prioblems with image building, i seem multiple failures on a recheck on https://review.opendev.org/c/openstack/tripleo-ansible/+/787767 | 08:08 |
| zbr | asking just to know if i should just "recheck" again or wait | 08:09 |
| zbr | i spotted few places where old-centos8 was still used instead of stream and that is not good, as the clock is ticking | 08:09 |
| marios|ruck | zbr: doesn't look familiar 2021-07-22 16:18:20.888 85939 ERROR tripleoclient.v2.tripleo_container_image.Build Stderr: 'level=debug msg="Pull Policy for pull [PullIfNewer]"\nerror creating build container: Error initializing source docker://registry.access.redhat.com/ubi8:latest: error pinging docker registry registry.access.redhat.com: Get "http://registry.access.redhat.com/v2/": dial tcp: lookup | 08:10 |
| marios|ruck | registry.access.redhat.com on 127.0.0.1:53: server misbehaving\nFailed to write to log, write /home/zuul/container-builds/61bbfae4-b1da-45d5-8409-ee5aa8bb2cd0/base/base-build.log: file already closed\nFailed to write to log, write /home/zuul/container-builds/61bbfae4-b1da-45d5-8409-ee5aa8bb2cd0/base/base-build.log: file already closed\n' | 08:10 |
| marios|ruck | https://1b8c54b5d412a3af0fef-40bd60678638a1db566d5d37b438f20d.ssl.cf1.rackcdn.com/787767/2/check/tripleo-ci-centos-8-content-provider/8e1b645/logs/undercloud/home/zuul/container_image_build.log | 08:10 |
| zbr | marios|ruck: do you suspect a problem with that line https://review.opendev.org/c/openstack/tripleo-ansible/+/787767/2/tripleo_ansible/roles/tripleo_container_image_build/defaults/main.yml ? | 08:11 |
| zbr | like some jobs not being able to pull from quay? | 08:11 |
| zbr | i coult try to use short name instead. | 08:11 |
| zbr | but considering that previous runs failed in other jobs, i suspect that the problem may be of different nature | 08:12 |
| zbr | the run from 18h ago passe all jobs but one which failed in POST, so my guess is that this change should report green unless we encounter some flakiness | 08:14 |
| marios|ruck | zbr: not sure if that's it but in general i haven't seen it elsewhere yet. thought the gate is a bit of a mess from yesterday outage and new blocker today so not a lot running yet | 08:15 |
| *** chem is now known as Guest1877 | 08:38 | |
| *** ykarel is now known as ykarel|lunch | 09:07 | |
| marios|ruck | chandankumar: do you remember if that was seen just once? I only see one log in the description https://bugs.launchpad.net/tripleo/+bug/1934994 | 09:33 |
| *** jpodivin_ is now known as jpodivin | 09:43 | |
| marios|ruck | soniya29|rover: o/ hey did you get a chance to check the trello cards yesterday? I didn't see any updates. i got up to the same place as yesterday, but going to go do something else now. would be great if you could have a look | 10:12 |
| marios|ruck | soniya29|rover: thank you | 10:12 |
| soniya29|rover | marios|ruck, i am over it | 10:12 |
| marios|ruck | soniya29|rover: if you find some time please look at 3rd card in degraded i just commented on the 2nd one (https://trello.com/c/V6SrUOIe/2018-cixlp1934994tripleociproa-octavia-healthmonitorscenariotest-are-failing-with-load-balanceris-immutable-and-cannot-be-updated#comment-60fa92d374b26a6708c5b2fb ) | 10:12 |
| marios|ruck | soniya29|rover: so basically 3 cards | 10:13 |
| marios|ruck | and one more in external | 10:13 |
| marios|ruck | soniya29|rover: just try to catchup with what the status is check the bug for any info that isn't on the card etc | 10:13 |
| soniya29|rover | marios|ruck, ack | 10:13 |
| *** ykarel|lunch is now known as ykarel | 10:33 | |
| marios|ruck | bhagyashris: akahat: aware of something wrong with promoter? logs look strange @ train http://38.102.83.109/promoter_logs/centos8_train.log no candidates? how can that be | 11:35 |
| marios|ruck | chandankumar: do you know if somethign was done recently to train promoter? ^^ | 11:37 |
| marios|ruck | :/ | 11:38 |
| bhagyashris | marios|ruck, looking | 11:42 |
| marios|ruck | bhagyashris: thanks | 11:43 |
| rlandy | marios|ruck: soniya29|rover: hey - reading through tripleo-ci chat | 11:53 |
| marios|ruck | rlandy: yeah just added couple points there easier to summarize like that | 11:58 |
| marios|ruck | brb coffee refresh | 11:58 |
| rlandy | marios|ruck: yep - got it - thanks for doc'ing that | 11:58 |
| rlandy | has a lot of failure last night | 11:58 |
| rlandy | DNS outage | 11:58 |
| marios|ruck | rlandy: yeah i saw ... this morning looked clear of that but it was hitting the new gate blocker | 12:05 |
| rlandy | always a reason for joy | 12:05 |
| marios|ruck | rlandy: fix is easy but the problem is getting it through the puppet-neutron gate https://review.opendev.org/c/openstack/puppet-neutron/+/801933/1#message-5a0eef40cd55034a153012c35c10fbfd49ab7912 and current run failing again https://zuul.openstack.org/status#801933 | 12:07 |
| chandankumar | marios|ruck: that https://bugs.launchpad.net/tripleo/+bug/1934994 was seen once, we proactively added it to skip list so we had not seen further failure, Sean investigated it but there was not much in the logs, and revert with testproject also passed and we have not seen more instance of the same. Bug is already marked as a invalid. | 12:08 |
| marios|ruck | chandankumar: k thanks for confirming. but lets not file bugs for things we see once. and especially lets not start skipping tempest tests for such things | 12:09 |
| marios|ruck | chandankumar: otherwise we will be chasing our tails even more than we already do ;) | 12:09 |
| chandankumar | marios|ruck: sure, :-) | 12:09 |
| marios|ruck | bhagyashris: akahat: Looks like this started immediately after the last promotion... | 12:19 |
| marios|ruck | bhagyashris: akahat: promotion at http://38.102.83.109/promoter_logs/centos8_train_2021-07-20T11:51.log then the next logfile has the 'no candidates' http://38.102.83.109/promoter_logs/centos8_train_2021-07-20T12:03.log and continue that way until today | 12:19 |
| bhagyashris | marios|ruck, looks weird to me | 12:20 |
| bhagyashris | because it is only happening for train | 12:21 |
| marios|ruck | bhagyashris: yeah we had promotions for other branches after that | 12:21 |
| marios|ruck | bhagyashris: but who knows what the status is there i mean why is it different from master did someone do something on that promoter recently? | 12:22 |
| marios|ruck | akahat: ? ^ | 12:22 |
| akahat | marios|ruck, looking in it. | 12:22 |
| bhagyashris | i need to check because i havent touch to promoter | 12:22 |
| bhagyashris | still lokking into code | 12:22 |
| akahat | marios|ruck, yes. that is expected. | 12:23 |
| chandankumar | akahat: bhagyashris last week we merged this https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33815 | 12:23 |
| marios|ruck | bhagyashris: akahat: thanks folks it isn't a fire now but we totally need that resolved before monday/tuesday latest if we are going to promote before the program call | 12:23 |
| marios|ruck | 15:23 < akahat> marios|ruck, yes. that is expected. | 12:23 |
| akahat | we have turned off debug logs to reduce logs | 12:23 |
| marios|ruck | akahat: the no candidates? | 12:23 |
| akahat | marios|ruck, yes. no candidates to promote. | 12:24 |
| akahat | are there any candidates ? | 12:24 |
| marios|ruck | akahat: how is it expected | 12:24 |
| bhagyashris | akahat, why only train | 12:24 |
| marios|ruck | akahat: i'm sure there would be some candidates since the last promotion i mean we have a new tripleo-ci-testing every day | 12:24 |
| akahat | i mean if there are any candidates then promoter is not detecting them | 12:24 |
| akahat | for c8 train?? | 12:25 |
| marios|ruck | akahat: checking | 12:25 |
| marios|ruck | tripleo-ci-testing/2021-07-23 06:13 | 12:25 |
| marios|ruck | https://trunk.rdoproject.org/centos8-train/ | 12:25 |
| marios|ruck | akahat: ^ | 12:25 |
| marios|ruck | current-tripleo/2021-07-20 07:12 | 12:26 |
| akahat | yeah.. there is new candidate but it is not picking it up. | 12:27 |
| akahat | need to check. | 12:28 |
| marios|ruck | akahat: yeah that is the problem and it means that it will promote nothing until we fix it | 12:28 |
| marios|ruck | akahat: please thanks | 12:28 |
| rlandy | marios|ruck: akahat: it looks to be the same hash: https://trunk.rdoproject.org/centos8-train/current-tripleo/delorean.repo.md5 2a68d3b9e14d86b57db95487cadce0e2 and https://trunk.rdoproject.org/centos8-train/tripleo-ci-testing/delorean.repo.md5 2a68d3b9e14d86b57db95487cadce0e2 | 12:28 |
| marios|ruck | rlandy: wtf... how can that be/? | 12:28 |
| rlandy | https://trunk.rdoproject.org/centos8-train/current-tripleo-rdo/delorean.repo.md5 | 12:29 |
| rlandy | also the same | 12:29 |
| marios|ruck | rlandy: does it just mean there is no newer content? but then why is it dated 23rd July? | 12:29 |
| marios|ruck | 15:25 < marios|ruck> tripleo-ci-testing/2021-07-23 06:13 | 12:29 |
| marios|ruck | 15:26 < marios|ruck> current-tripleo/2021-07-20 07:12 | 12:29 |
| akahat | rlandy, yes. both are same hash.. how it is possible... is dlrn drunk? | 12:29 |
| akahat | i mean those are different dates. | 12:29 |
| rlandy | marios|ruck: akahat: we'd have to check if any components rev'ed | 12:29 |
| rlandy | jpena|off is PTO | 12:30 |
| rlandy | but it would be right if nothing rev'ed | 12:30 |
| akahat | oh.. this is promotion to current-tripleo-rdo: http://38.102.83.109/promoter_logs/centos8_train_2021-07-20T11:51.log | 12:30 |
| marios|ruck | k i think we need a bug for that wdyt rlandy? | 12:30 |
| rlandy | idk yet | 12:30 |
| marios|ruck | rlandy: cos we may need somethign to point to for the program call if not resolved by then ... | 12:30 |
| rlandy | let's confirm that nothing rev'ed | 12:30 |
| marios|ruck | rlandy: right OK... soniya29|rover can you please follow ^^^ and if needed (ask before you do to confirm) can you please file a bug for that? | 12:31 |
| soniya29|rover | marios|ruck, sure | 12:31 |
| rlandy | (testenv) [rlandy@localhost ci-config]$ python3 roles/rrcockpit/files/telegraf_py3/ruck_rover.py --release train | 12:31 |
| marios|ruck | soniya29|rover: more pointers for the issue in gchat (see PROMOTIONS - chasing master, train promoter broken? | 12:31 |
| rlandy | marios|ruck: ^^ ran that | 12:31 |
| marios|ruck | rlandy: yeah but that also queries dlrn right so same result | 12:33 |
| marios|ruck | Hash under test: https://trunk.rdoproject.org/api-centos8-train/api/civotes_agg_detail.html?ref_hash=2a68 | 12:33 |
| marios|ruck | d3b9e14d86b57db95487cadce0e2 | 12:33 |
| rlandy | we'd have to do a component by component check | 12:34 |
| rlandy | only way | 12:34 |
| rlandy | the date could rev as we create tripleo-ci-testing on every integration jobs run | 12:34 |
| marios|ruck | rlandy: so the promoter might be telling the truth then? and the date is changed if one of the ocmponent promoted. but if it did, then it should be a new candidate hash tripleo-ci-testing no? | 12:34 |
| rlandy | marios|ruck: yep - but we'd need to confirm that | 12:35 |
| marios|ruck | rlandy: ah so the directory is recreated you mean | 12:35 |
| rlandy | akahat: ^^ I know I am late for our scenario010 meeting | 12:35 |
| rlandy | will ping when done here | 12:35 |
| akahat | marios|ruck, yes possibly | 12:35 |
| akahat | rlandy, ah.. sorry.. joining | 12:35 |
| rlandy | akahat: ^^ not yet | 12:36 |
| rlandy | going to confirm train for marios|ruck first | 12:36 |
| marios|ruck | rlandy: k go ahead i will look at components to check | 12:36 |
| rlandy | will ping you | 12:36 |
| akahat | rlandy, okay. np | 12:36 |
| rlandy | marios|ruck; the ruck_rover tool should help | 12:36 |
| rlandy | checking | 12:36 |
| rlandy | ah- better way ... | 12:37 |
| marios|ruck | rlandy: i don't see anythign to signa the stale content though in the ruck_rover tool ? | 12:37 |
| marios|ruck | rlandy: or am i missing it | 12:37 |
| marios|ruck | rlandy: i am trying --component all --release train now | 12:37 |
| akahat | tripleo-ci-testing, current-tripleo and current-tripleo-rdo are pointing at same hash.. :/ | 12:37 |
| marios|ruck | akahat: k so something went wrong with the last promotion sounds like? | 12:38 |
| marios|ruck | akahat: i noticed same in logs see http://38.102.83.109/promoter_logs/centos8_train_2021-07-20T12:03.log | 12:38 |
| marios|ruck | 2021-07-20 12:03:12,387 2230179 INFO promoter Target label 'current-tripleo': current hash is aggregate: 2a68d3b9e14d86b57db95487cadce0e2, commit: dc72a3b84e498f5110bb739c30e6afecb93a1296, distro: 871628e5c31aafc96db1390050dc1a6d7c62f2a9, extended: None, component: validation, timestamp: 1626761721 | 12:38 |
| rlandy | http://dashboard-ci.tripleo.org/d/mOvYIiOMk/component-pipeline-train?orgId=1 says thing changed | 12:38 |
| marios|ruck | 2021-07-20 12:03:16,727 2230179 INFO promoter Target label 'current-tripleo-rdo': current hash is aggregate: 2a68d3b9e14d86b57db95487cadce0e2, commit: 36705150b1082763a9597da4e89be03fbae788a2, distro: 6edf151d9389630feec1eef65f60e15b8a793947, extended: None, component: tripleo, timestamp: 1626765121 | 12:38 |
| marios|ruck | akahat: i.e. current-tripleo and current-tripleo-rdo are both aggregate: 2a68d3b9e14d86b57db95487cadce0e2, | 12:39 |
| marios|ruck | no actually that might be ok akahat | 12:39 |
| marios|ruck | i mean we promote current-tripleo to current-tripleo-rdo if the jobs pass so... | 12:39 |
| rlandy | honestly the only way to be sure is to do this manually | 12:39 |
| marios|ruck | maybe it just mean no new content if current is also saying the same | 12:40 |
| marios|ruck | rlandy: so did you find some way with ruck_rover tool or should i go back to clicking ;) | 12:40 |
| rlandy | marios|ruck: let's split the stack of components | 12:40 |
| rlandy | I'll start from baremetal | 12:40 |
| rlandy | do you want to start from validations | 12:40 |
| rlandy | we will meet in the middle | 12:40 |
| marios|ruck | rlandy: err hold on | 12:40 |
| marios|ruck | rlandy: the list i am looking at has glance at the top and tripelo at the bottom | 12:40 |
| akahat | yes.. that is also possibility.. if nothing gets in the c8 train branch then hash would be same. | 12:40 |
| marios|ruck | rlandy: http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1 | 12:40 |
| marios|ruck | rlandy: which list you looking at ;) | 12:41 |
| rlandy | https://trunk.rdoproject.org/centos8-train/component/ | 12:41 |
| marios|ruck | probably fro mdlrn directory | 12:41 |
| marios|ruck | rlandy: k | 12:41 |
| rlandy | alphabetical | 12:41 |
| marios|ruck | rlandy: ok looking at validation | 12:41 |
| marios|ruck | rlandy: already found one i think... validations https://trunk.rdoproject.org/centos8-train/component/validation/ | 12:42 |
| marios|ruck | rlandy: look at tripleo-ci-testing still on 13th but consistent from 22nd that is wrong no? | 12:42 |
| rlandy | baremetal matches | 12:42 |
| rlandy | looking at validations | 12:42 |
| rlandy | marios|ruck: https://paste.opendev.org/show/807678/ | 12:44 |
| marios|ruck | rlandy: ah so dates mean nothing :/ | 12:44 |
| rlandy | yep | 12:45 |
| marios|ruck | rlandy: but that makes debug much harder wasn't always like that no? | 12:45 |
| rlandy | need to check commit/distro hash | 12:45 |
| rlandy | always | 12:45 |
| marios|ruck | rlandy: are we doing that with the promoter maybe? | 12:45 |
| rlandy | the ruck rover tool was meant to make it easier | 12:45 |
| rlandy | BUT | 12:45 |
| rlandy | when the chips are down, I go back to manual | 12:45 |
| rlandy | so we are sure we have the right info | 12:45 |
| marios|ruck | rlandy: k i guess we can only rely on the date for current-tripleo ... anyway k thanks continuing then | 12:45 |
| rlandy | no -the promoter is a commit/distro game | 12:45 |
| zbr | frenzy_friday: apparently today we are doomed to face problems where local runs give different results than CI ones :( | 12:46 |
| rlandy | yep - because that is a pormotion-only hash | 12:46 |
| rlandy | the rest rev per run | 12:46 |
| frenzy_friday | zbr, true that | 12:46 |
| rlandy | we always promote promoted-components to tripleo-ci-testing per run | 12:46 |
| * rlandy carries on | 12:46 | |
| marios|ruck | rlandy: thx | 12:46 |
| rlandy | check the commit.yaml | 12:46 |
| marios|ruck | rlandy: yah | 12:47 |
| rlandy | baremetal, cinder - good so far | 12:48 |
| rlandy | marios|ruck: the cockpit is tracking the wrong component list | 12:52 |
| rlandy | http://dashboard-ci.tripleo.org/d/mOvYIiOMk/component-pipeline-train?orgId=1 is tracking master components | 12:52 |
| marios|ruck | rlandy: am at glance should i stop? | 12:52 |
| marios|ruck | rlandy: ack ok | 12:52 |
| rlandy | I am on common | 12:52 |
| marios|ruck | rlandy: k stop then | 12:53 |
| marios|ruck | rlandy: am doing compute | 12:53 |
| marios|ruck | rlandy: gimme sec will post but don't see any diff on these so far | 12:53 |
| rlandy | stopping | 12:53 |
| akahat | https://paste.centos.org/view/4a78c68a <-- this tells something different.. validation component which is under same hash is promoted to the tripleo-ci-testing but commit hash is same. | 12:54 |
| rlandy | https://paste.opendev.org/show/807679/ | 12:54 |
| rlandy | ^^ mine - nothing is diff | 12:54 |
| akahat | this is the result of some of the failures jobs. | 12:54 |
| akahat | which got passed. and got promoted to the tripleo-ci-testing | 12:55 |
| akahat | hope so i got it right ^^ | 12:55 |
| marios|ruck | rlandy: https://paste.opendev.org/raw/807680/ same can't see any diff | 12:55 |
| marios|ruck | rlandy: akahat: so then there is no newer content looks like | 12:55 |
| rlandy | marios|ruck: akahat: yep - looks right | 12:55 |
| marios|ruck | rlandy: akahat: sorry for false alarm. but how is that possible nothing into train in 5 days is weird no? | 12:55 |
| akahat | yup. | 12:55 |
| rlandy | marios|ruck: no worries, the cockpit is wrong | 12:56 |
| rlandy | ^^ you are right there is a bug somewhere | 12:56 |
| * rlandy hates tools - goes back to first principles | 12:56 | |
| rlandy | electrical engineering training :) | 12:56 |
| marios|ruck | bhagyashris: ^^^ fyi false alarm there is actually no new content soniya29|rover no need for bug | 12:57 |
| marios|ruck | let all go to the pub mojitos are on me | 12:57 |
| soniya29|rover | marios|ruck, okay | 12:57 |
| rlandy | akahat: give me five mins - will ping re: scenario010 | 12:57 |
| akahat | rlandy, ok | 12:57 |
| bhagyashris | marios|ruck, yup , it seems wired to me from start | 12:58 |
| rlandy | marios|ruck: soniya29|rover: I pinged the security group that multinode-ipa is voting again .. feel free to lean on their caesar is there are failures | 13:07 |
| rlandy | or dwilde in ade's absence | 13:08 |
| rlandy | akahat: k - ready - joining scenraio010 meeting | 13:08 |
| *** amoralej is now known as amoralej|lunch | 13:09 | |
| soniya29|rover | rlandy, sure | 13:09 |
| rlandy | https://ci.centos.org/view/rdo/view/promotion-pipeline/ | 13:09 |
| rlandy | no bad there - rerunning ussuri and victoria | 13:09 |
| rlandy | not | 13:09 |
| akahat | i'm in. | 13:09 |
| marios|ruck | rlandy: ack | 13:16 |
| rlandy | arxcruz: hey - are you core on https://opendev.org/osf/python-tempestconf? | 13:17 |
| arxcruz | rlandy: yes | 13:17 |
| rlandy | can you join us on https://meet.google.com/gdx-chgo-amf?authuser=0 | 13:18 |
| rlandy | https://opendev.org/osf/python-tempestconf/src/branch/master/.zuul.yaml#L45 | 13:18 |
| arxcruz | tosky: can we get rid of scenario010 job on tempestconf ? | 13:22 |
| arxcruz | tosky: we are deprecating scen010 on master, just on train | 13:23 |
| tosky | arxcruz: well, if it's disappearing... what was it meant to test (without digging into the various tables)? | 13:27 |
| tosky | arxcruz: which is the closest replacement? | 13:27 |
| tosky | kopecmartin: ^^ | 13:27 |
| arxcruz | https://review.opendev.org/c/osf/python-tempestconf/+/802024 | 13:29 |
| arxcruz | tosky: kopecmartin ^ | 13:29 |
| arxcruz | tosky: i think we were testing some octavia stuff | 13:30 |
| tosky | arxcruz: so is there another scenario with octavia? | 13:32 |
| arxcruz | tosky: only 10 runs octavia | 13:32 |
| tosky | arxcruz: so there is no testing for octavia at all? | 13:34 |
| tosky | that seems strange | 13:34 |
| arxcruz | tosky: not on tripleo side | 13:34 |
| arxcruz | rlandy: akahat ^ can you confirm it ? | 13:34 |
| arxcruz | tosky: we can run it as stable/train though | 13:34 |
| tosky | arxcruz: which doesn't mean it can't break | 13:36 |
| arxcruz | tosky: https://meet.google.com/gdx-chgo-amf?authuser=0 | 13:37 |
| arxcruz | tosky: oh, nevermind | 13:37 |
| rlandy | arxcruz: tosky: we can keep the scenario010 job - we will just need t add the train and ussuri versions | 13:37 |
| tosky | rlandy: I get it, I still don't get why there is no octavia testing anymore | 13:39 |
| tosky | is tripleo dropping octavia? | 13:39 |
| eagles | 2c no | 13:39 |
| marios|ruck | rlandy: (when you have time, no action needed fyi): I said I would look into centos7 periodics today and just found some time to do that. Situation summarized there https://review.opendev.org/c/openstack/tripleo-ci/+/800646/2#message-6c147dba41d78a64ee7f4ba78faa7365d50af74d - we need that patch to unblock the C7 container push job | 13:39 |
| eagles | (I hope not, it's hugely important to a lot of people) | 13:40 |
| rlandy | tosky:nobody is dropping octavia | 13:40 |
| eagles | :) | 13:40 |
| rlandy | to the contrary, we are aiming to make it voting | 13:40 |
| rlandy | in that process we are splitting out a train branch and a ussuri branch | 13:41 |
| tosky | rlandy: so does it mean that scenario010 will be replaced by another job which also includes octavia? | 13:41 |
| tosky | my point is: that job is used on python-tempestconf to test octavia; if that job is going to be removed, and octavia is still supported, I guess there will be another job which replaces the removed one | 13:41 |
| rlandy | the reason we pinged you guys is that tempestconf https://opendev.org/osf/python-tempestconf/src/branch/master/.zuul.yaml#L45 will need to be edited to run the train and ussuri branched versions as well | 13:41 |
| rlandy | tosky: no job is being removed | 13:42 |
| rlandy | we will add to it | 13:42 |
| tosky | rlandy: I'm still confused | 13:42 |
| rlandy | I will out up a patch for your review | 13:42 |
| tosky | rlandy: can you please clarify this discussion first, please? This discussion started as " we are deprecating scen010 on master, just on train " | 13:42 |
| tosky | that's the first step before removal | 13:42 |
| rlandy | eagles: https://opendev.org/openstack/puppet-octavia/src/branch/master/.zuul.yaml#L12 is in the same boat ... once https://review.opendev.org/c/openstack/tripleo-ci/+/800540 merges | 13:43 |
| tosky | I'd like to have a clear explanation of what's going on before deciding | 13:43 |
| rlandy | we will need to add tha train and ussuri tests | 13:43 |
| rlandy | tosky: to be clear, nothing is being removed ... scenario101 standlaone was non-voting | 13:44 |
| rlandy | it was failing on train and ussuri | 13:44 |
| rlandy | in the process of making the job voting | 13:44 |
| rlandy | we are creating new definitions for train and ussuri | 13:44 |
| tosky | that's a completely different scenario that was discussed at the beginning | 13:44 |
| rlandy | so those branches can be made voting when ready | 13:44 |
| rlandy | beginning? | 13:45 |
| rlandy | <arxcruz> tosky: we are deprecating scen010 on master, just on train ... that was the confusing line | 13:45 |
| tosky | rlandy: https://meetings.opendev.org/irclogs/%23oooq/%23oooq.2021-07-23.log.html#t2021-07-23T13:22:43 | 13:45 |
| rlandy | ^^ I think we confused arxcruz | 13:46 |
| rlandy | it is taying on master | 13:46 |
| arxcruz | rlandy: tosky sorry i express myself wrong :) | 13:46 |
| tosky | ok, so you would like to add more jobs for older branches | 13:47 |
| rlandy | tosky: ^^ clear now? if not we can meet and I'll explain more | 13:47 |
| tosky | please note we haven't done so far for any tripleo jobs | 13:47 |
| tosky | as the gates for tempestconf already runs a good amount of jobs, I wonder if we are not going block the queues | 13:48 |
| rlandy | tosky: "done so" ^^ what does that refer to? | 13:48 |
| rlandy | it won;t run extra jobs | 13:48 |
| rlandy | it will just run a jobs by a slightly diff name on train and ussuri | 13:48 |
| tosky | that's running extra jobs | 13:48 |
| tosky | tempestconf is not branched | 13:48 |
| tosky | right now we have python-tempestconf-tempest-devstack-admin, python-tempestconf-tempest-devstack-admin-{wallaby,victoria,ussuri,train} | 13:49 |
| tosky | are you suggesting to have the same for tripleo-ci-centos-8-scenario002-standalone? | 13:49 |
| rlandy | no | 13:50 |
| rlandy | that one is voting everywhere | 13:50 |
| rlandy | you're fine then | 13:50 |
| tosky | happy to know I'm fine, I'm still not sure what's just happened | 13:51 |
| tosky | back to internal stuff | 13:51 |
| arxcruz | rlandy: tosky i already abandon the patch, sorry, i missunderstood | 13:52 |
| rlandy | arxcruz: np | 13:54 |
| *** amoralej|lunch is now known as amoralej | 14:01 | |
| *** pojadhav- is now known as pojadhav | 14:16 | |
| *** ykarel is now known as ykarel|away | 14:56 | |
| rlandy | marios|ruck: looking at https://review.rdoproject.org/r/c/testproject/+/34654 | 14:56 |
| rlandy | do we need to rerun 20 and 39 to get master to promote? | 14:56 |
| marios|ruck | rlandy: yeah i already tried a recheck | 14:56 |
| marios|ruck | rlandy: that hash is from yesterday but good enough i think | 14:56 |
| rlandy | k - see that comment | 14:57 |
| marios|ruck | rlandy: which comment | 14:57 |
| rlandy | recheck | 14:57 |
| rlandy | they failed again | 14:57 |
| rlandy | : Get image expected checksum | 14:57 |
| marios|ruck | rlandy: https://review.rdoproject.org/zuul/status#34654 | 14:57 |
| marios|ruck | rlandy: they didt report yet | 14:58 |
| marios|ruck | rlandy: but yeah failure | 14:58 |
| rlandy | install erros | 14:59 |
| rlandy | errors | 14:59 |
| rlandy | marios|ruck: the ipa job is going to report now | 14:59 |
| rlandy | maybe we try one more recheck? | 15:00 |
| marios|ruck | rlandy: sure go ahead am looking at other stuff right now (doing some ptl things that need addressing) | 15:00 |
| marios|ruck | rlandy: i think i'm done looking at jobs for today :) my brain hurts and its only day #2 :/ | 15:00 |
| rlandy | k | 15:00 |
| rlandy | marios|ruck: yeah - I get that | 15:01 |
| * rlandy nicks | 15:01 | |
| *** rlandy is now known as rlandy|ruck | 15:01 | |
| rlandy|ruck | marios|ruck: other than master promotion, anything else? | 15:01 |
| rlandy|ruck | https://zuul.openstack.org/status#801933 | 15:02 |
| marios|ruck | rlandy|ruck: yeah that one is a gate blocker so def need that | 15:02 |
| rlandy|ruck | watching | 15:02 |
| marios|ruck | rlandy|ruck: gate and promotion blocker in fact | 15:02 |
| rlandy|ruck | amazing | 15:02 |
| marios|ruck | rlandy|ruck: is why i have it on depends-on of that master testproject | 15:03 |
| rlandy|ruck | yep - I see that | 15:03 |
| rlandy|ruck | maybe be causing build failure | 15:03 |
| rlandy|ruck | I'll recheck when it merges if fails again | 15:03 |
| marios|ruck | rlandy|ruck: thanks | 15:04 |
| rlandy|ruck | rekicked | 15:16 |
| marios|ruck | rlandy|ruck: did you see that one btw https://review.opendev.org/c/openstack/tripleo-ci/+/800646/2#message-6c147dba41d78a64ee7f4ba78faa7365d50af74d | 15:22 |
| *** holser is now known as holser_ | 15:24 | |
| rlandy|ruck | lucky we checked :) | 15:26 |
| rlandy|ruck | we = you | 15:26 |
| marios|ruck | rlandy|ruck: well no guarantees we'll get a promotion (user experience may vary, images for illustrative purposes only) but its the first step will revisit next week | 15:28 |
| marios|ruck | ah crap totally forgot about the happy friday call :/ | 15:34 |
| marios|ruck | no happy friday for me | 15:34 |
| marios|ruck | :( | 15:34 |
| rlandy|ruck | we missed you | 15:35 |
| marios|ruck | i'm sure arxcruz missed me most | 15:35 |
| rlandy|ruck | sigh ... unrequited love | 15:36 |
| marios|ruck | :D | 15:37 |
| *** holser_ is now known as holser | 15:51 | |
| marios|ruck | rlandy|ruck: starting shutdown sequence | 15:56 |
| rlandy|ruck | marios|ruck: k - have a good weekend | 15:57 |
| rlandy|ruck | https://review.rdoproject.org/r/c/testproject/+/34654 failed again | 15:57 |
| * rlandy|ruck waits on merge | 15:57 | |
| marios|ruck | come on scen2 ... waiting https://zuul.openstack.org/status#801933 so i can change topic in tripleo and reply to http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023805.html | 15:59 |
| marios|ruck | thanks rlandy|ruck re 34654 would be good to get master | 16:00 |
| marios|ruck | sweet merged | 16:03 |
| *** marios|ruck is now known as marios | 16:03 | |
| rlandy|ruck | nice | 16:06 |
| *** marios is now known as marios|out | 16:08 | |
| soniya29|rover | marios|out, rlandy|ruck, leaving for the day | 16:13 |
| *** amoralej is now known as amoralej|off | 16:29 | |
| rlandy|ruck | hmmm ... we have an issue with docker push | 17:15 |
| rlandy|ruck | akahat: I think we have a problem with master promotions | 17:20 |
| rlandy|ruck | no containers in docker for current-tripleo | 17:21 |
| rlandy|ruck | https://hub.docker.com/r/tripleomaster/openstack-keystone/tags?page=1&ordering=last_updated | 17:21 |
| rlandy|ruck | https://images.rdoproject.org/centos8/master/rdo_trunk/07715e073191d94abc2f35e6b90563f9/ | 17:23 |
| rlandy|ruck | we have images but no containers | 17:23 |
| rlandy|ruck | akahat: chandankumar: you guys around? | 17:34 |
| rlandy|ruck | arxcruz: ^^ around? | 17:50 |
| rlandy|ruck | anyone? | 17:50 |
| rlandy|ruck | need someone to check container repush | 17:50 |
| arxcruz | rlandy|ruck: something wrong ? | 18:04 |
| rlandy|ruck | arxcruz: yep ... | 18:04 |
| rlandy|ruck | https://hub.docker.com/r/tripleomaster/openstack-base/tags?page=1&ordering=last_updated | 18:04 |
| rlandy|ruck | docker and quay missing containers for current-tripleo | 18:04 |
| rlandy|ruck | https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo.md5 | 18:04 |
| rlandy|ruck | ^^ no containers with that tag | 18:04 |
| rlandy|ruck | killing validation libs | 18:05 |
| rlandy|ruck | trying to check rdo registry to see if the containers are there | 18:05 |
| rlandy|ruck | https://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomaster/imagestreamtags/ failing for e | 18:05 |
| rlandy|ruck | me | 18:05 |
| arxcruz | rlandy|ruck: i'm seeing 07715e073191d94abc2f35e6b90563f9 for tripleo-base on quay | 18:06 |
| rlandy|ruck | the keystone one was missing from docker | 18:06 |
| rlandy|ruck | do you see that one? | 18:06 |
| arxcruz | let me check keystone | 18:06 |
| arxcruz | rlandy|ruck: yup | 18:07 |
| arxcruz | keystone is there | 18:08 |
| rlandy|ruck | on docker? | 18:08 |
| arxcruz | https://quay.io/repository/tripleomaster/openstack-keystone?tab=tags | 18:08 |
| rlandy|ruck | or just on quay | 18:08 |
| arxcruz | on quay | 18:08 |
| arxcruz | rlandy|ruck: http://38.102.83.131/quay/tag/master-report.html | 18:08 |
| arxcruz | all containers were tagged successfully | 18:08 |
| rlandy|ruck | I don;t see it on docker | 18:08 |
| rlandy|ruck | https://hub.docker.com/r/tripleomaster/openstack-keystone/tags?page=1&ordering=last_updated | 18:08 |
| rlandy|ruck | arxcruz: do you see that? | 18:09 |
| rlandy|ruck | https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-standalone-validation-libs | 18:09 |
| rlandy|ruck | maybe just a hitch | 18:10 |
| arxcruz | rlandy|ruck: i'm not seeing, but we can copy this particular tag to docker no ? | 18:10 |
| arxcruz | from quay to docker ? | 18:10 |
| rlandy|ruck | arxcruz: yeah - want to do that from the promoter? | 18:10 |
| rlandy|ruck | or from toolbox? | 18:10 |
| arxcruz | never did it | 18:10 |
| arxcruz | but we can try | 18:10 |
| rlandy|ruck | k - should we tmux? | 18:11 |
| rlandy|ruck | arxcruz: the alternative is to wait and promote the next hash | 18:14 |
| rlandy|ruck | arxcruz: we should copy run promoter again | 18:15 |
| arxcruz | rlandy|ruck: just pushed the hash | 18:16 |
| arxcruz | https://hub.docker.com/layers/tripleomaster/openstack-keystone/07715e073191d94abc2f35e6b90563f9/images/sha256-747c0417c34606cd8bb7bf0bf3121089df4acfce58a745a7395c68365f4a224c?context=explore | 18:16 |
| arxcruz | rlandy|ruck: is there some other container missing | 18:16 |
| rlandy|ruck | arxcruz: thank you!!! | 18:16 |
| arxcruz | rlandy|ruck: is not tagged as current-tripleo though | 18:16 |
| arxcruz | should i do that on docker? | 18:16 |
| arxcruz | or just need to have the container there ? | 18:17 |
| rlandy|ruck | I don;t think it matters but they should match | 18:18 |
| *** holser is now known as holser_ | 21:07 | |
| *** holser_ is now known as holser | 21:54 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!