| *** yadnesh|away is now known as yadnesh | 04:30 | |
| *** marios is now known as marios|ruck | 06:09 | |
| marios|ruck | \o | 06:09 |
|---|---|---|
| ysandeep | o/ hello everyone | 06:47 |
| marios|ruck | rlandy|out: o/ merged https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/435580/11#message-f85492f95dc46db073024915d0cc540fe2009688 just added a thought there can be followup if we want | 07:30 |
| dpawlik | frenzy_friday, ysandeep__: hey, how did you create new projects in quay.rdoproject.org ? | 07:39 |
| dpawlik | I guess manually, right? | 07:39 |
| ysandeep__ | dpawlik, frenzy_friday knows best but afaik we followed https://hackmd.io/BCZz9NgvSGqFF4Rney0Hkg#Workflow | 07:45 |
| *** ysandeep__ is now known as ysandeep|afk | 07:49 | |
| dpawlik | ysandeep|afk: so the procedure is easier than as you have done | 07:53 |
| *** yadnesh is now known as yadnesh|afk | 08:01 | |
| dpawlik | ysandeep|afk: I guess tripleozed and tripleomaster can be removed, right? | 08:01 |
| dpawlik | ysandeep|afk: so this one is bootstrapping the project: https://softwarefactory-project.io/r/c/software-factory/sf-infra/+/26901 in Quay | 08:04 |
| frenzy_friday | ysandeep|afk, dviroel|afk yep thats the hackmd. tripleozed and tripleomaster can be removesd | 08:04 |
| dpawlik | then when it will generate the robot and it will get the token, it is just important to encrypt the token and put it once again into the sf-infra, so the pruner will start cleaning up the registry | 08:05 |
| dpawlik | so far, we have 4GB free space | 08:05 |
| dpawlik | due it was not cleaned | 08:05 |
| dpawlik | frenzy_friday: thanks for confirmation | 08:05 |
| dpawlik | I will remove those two projects, ok? | 08:05 |
| marios|ruck | ysandeep|afk: chandankumar: please review when you have time https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46337 | 08:19 |
| *** khomesh|away is now known as khomesh | 08:39 | |
| *** jpena|off is now known as jpena | 08:42 | |
| ysandeep|afk | 1 | 09:09 |
| *** ysandeep|afk is now known as ysandeep | 09:10 | |
| frenzy_friday | Hey marios|ruck , rlandy|out : dpawlik is regenerating the quay tokens as there was a log leaking the tokens. The push jobs will fail for some time till we merge the config repo patch (tbd after changing the tokens) which updates the zuul secrets | 09:13 |
| marios|ruck | frenzy_friday: dpawlik: i filed a bug, as we had 502 bad gateway for quay.rdoproject yesterday, is this related to what you're working on? https://bugs.launchpad.net/tripleo/+bug/1998762 | 09:14 |
| frenzy_friday | marios|ruck, no I guess. We have started regenerating tokens a few mins back | 09:15 |
| marios|ruck | frenzy_friday: k | 09:15 |
| marios|ruck | frenzy_friday: you can update us on this leak later in team call then | 09:15 |
| frenzy_friday | yep sure | 09:15 |
| *** yadnesh|afk is now known as yadnesh | 09:24 | |
| dpawlik | marios|ruck: quay was out of disk space due | 09:25 |
| dpawlik | zed repository was not added into the pruner | 09:26 |
| dpawlik | that's why was 502 | 09:26 |
| marios|ruck | dpawlik: ah thankyou, adding info on bug | 09:28 |
| dpawlik | https://review.rdoproject.org/r/c/config/+/46342 | 09:28 |
| marios|ruck | dpawlik: master is OK? | 09:32 |
| ysandeep | dpawlik, this pruner config is for quay registry? https://softwarefactory-project.io/r/c/software-factory/sf-infra/+/26900/1/playbooks/host_vars/quay.rdoproject.org.yaml#245 | 09:35 |
| dpawlik | yup | 09:36 |
| ysandeep | dpawlik, okay.. so i think we only need zed one here.. commented: https://softwarefactory-project.io/r/c/software-factory/sf-infra/+/26900/1#message-87014729929d857a334bd2291afc5df4f1126fd0 | 09:36 |
| ysandeep | u/v are already gone, and x/y release we never created branches/pipeline for them in tripleo | 09:37 |
| ysandeep | c7 train is gone as well.. so i think we can drop that as well. | 09:41 |
| ysandeep | tagging marios|ruck, chandankumar for review on https://softwarefactory-project.io/r/c/software-factory/sf-infra/+/26900 as well. | 09:41 |
| marios|ruck | ysandeep: ok will look | 09:42 |
| dpawlik | ysandeep: please leave there a comment, thanks | 09:48 |
| * dpawlik littlebit busy | 09:48 | |
| ysandeep | dpawlik: nw, left a comment there already | 09:49 |
| dpawlik | marios|ruck, ysandeep: I run pruner for zed. Space will be soon available | 10:13 |
| ysandeep | dpawlik++ thanks! | 10:15 |
| marios|ruck | thanks dpawlik | 10:19 |
| dpawlik | frenzy_friday: check it in free time https://review.rdoproject.org/r/c/config/+/46342 | 10:21 |
| dpawlik | marios|ruck, ysandeep: quick question: tripleotraincentos7 and tripleovictoriacentos8 are empty, is it correct? | 10:43 |
| dpawlik | should it been deleted? | 10:43 |
| dpawlik | should I delete them or leave ? | 10:44 |
| ysandeep | dpawlik, we don't have c7 train, victoria/ussuri line anymore | 10:44 |
| marios|ruck | dpawlik: yes delete | 10:44 |
| marios|ruck | dpawlik: nuke from orbit | 10:44 |
| dpawlik | xD | 10:44 |
| ysandeep | dpawlik, as maris said, we should just delete them | 10:44 |
| marios|ruck | no i dont just want to delete them | 10:45 |
| marios|ruck | ;) | 10:45 |
| dpawlik | <press the button and it will disappear> | 10:45 |
| marios|ruck | right i mean if orbit is difficult/expensive these days with the fuel costs i can settle for icbm | 10:46 |
| dpawlik | hehe | 10:47 |
| dpawlik | cleanup done | 10:47 |
| marios|ruck | thank you dpawlik | 10:47 |
| dpawlik | let's start a new era | 10:48 |
| marios|ruck | dpawlik: i'll close the bug later on once we see it running green | 10:48 |
| *** ysandeep is now known as ysandeep|brb | 10:49 | |
| marios|ruck | chem: o/ can you please checkout https://bugs.launchpad.net/tripleo/+bug/1998770 train minor update job promotion blocker - something must have merged late last week to cause it ... i will dig more later but greatful if you have a minute to check might ring bells | 10:51 |
| chem | marios|ruck: hey, looking | 10:52 |
| marios|ruck | chem: when/if you have time thanks not meant to be interrupt. it will end up on cix board just looking for early feedback | 10:52 |
| marios|ruck | chem: in your own time thank you | 10:52 |
| chem | marios|ruck: I think I know | 10:53 |
| marios|ruck | chem: looks to me some regression from our code TypeError: isinstance expected 2 arguments, got 1" | 10:53 |
| marios|ruck | chem: k thanks if you have any idea please leave comment in the bug i will revisit later and dig more if needed o/ thank you! | 10:54 |
| chem | marios|ruck: commented | 10:55 |
| marios|ruck | chem++ | 10:55 |
| chem | marios|ruck: extra code went in that review, I've tested it downstream and it passed ci, but something must trigger it in some case | 10:55 |
| dpawlik | marios|ruck: one more question: https://quay.rdoproject.org/repository/tripleozedcentos9/openstack-neutron-server-ovn should it be private or public? | 10:57 |
| *** ysandeep|brb is now known as ysandeep | 10:59 | |
| chem | marios|ruck: found it, doing the review | 11:02 |
| *** dviroel|afk is now known as dviroel | 11:05 | |
| marios|ruck | chem: perfect thank you | 11:06 |
| marios|ruck | dpawlik: i thnk they should all be public | 11:07 |
| marios|ruck | dpawlik: maybe an oversight? | 11:07 |
| *** rlandy|out is now known as rlandy|rover | 11:10 | |
| dpawlik | dunno | 11:10 |
| dpawlik | so I will make it public | 11:10 |
| marios|ruck | dpawlik: yes thanks | 11:10 |
| dpawlik | https://quay.rdoproject.org/repository/tripleozedcentos9/nop | 11:10 |
| dpawlik | this one I guess it is for tests | 11:10 |
| marios|ruck | dpawlik: yeah dont know what that is :) | 11:10 |
| dpawlik | :D | 11:11 |
| dpawlik | thanks | 11:11 |
| dpawlik | going to eat something | 11:11 |
| rlandy|rover | marios|ruck: hey want to sync after review time? | 11:12 |
| marios|ruck | rlandy|rover: yes | 11:14 |
| chem | marios|ruck: is there a way to trigger a test with the proposed patch ? | 11:15 |
| marios|ruck | chem: joining call now will check in a little bit | 11:15 |
| chem | marios|ruck: ack | 11:16 |
| rlandy|rover | frenzy_friday: can you join review time? | 11:28 |
| marios|ruck | https://review.rdoproject.org/r/c/config/+/46342 | 11:28 |
| *** pojadhav is now known as pojadhav|brb | 11:31 | |
| frenzy_friday | rlandy|rover, sorry missed the ping | 11:44 |
| *** yadnesh is now known as yadnesh|afk | 11:45 | |
| marios|ruck | chem: you should be able to use testproject for tripleo-ansible as usual | 11:55 |
| marios|ruck | chem: let me know ifyou want me to post after my call | 11:56 |
| marios|ruck | https://review.opendev.org/c/openstack/tripleo-ansible/+/866565 | 11:56 |
| chem | marios|ruck: "as usual" is totally alien to me ..., but I think I have a link from you | 11:56 |
| marios|ruck | chem: let me know if you want help/me to do no problem | 11:56 |
| rlandy|rover | marios|ruck: https://review.rdoproject.org/r/c/rdo-jobs/+/45625 | 11:59 |
| chem | marios|ruck: I took a naive approach, can you check it makes sense https://code.engineering.redhat.com/gerrit/c/testproject/+/437007 ? | 12:04 |
| marios|ruck | chem: why downstream, will post something in a minute thanks | 12:09 |
| frenzy_friday | akahat, rlandy|rover , Voted on the rdo decommission patches | 12:14 |
| rlandy|rover | frenzy_friday: thanks - so in your opinion, that is safe to go? | 12:23 |
| *** pojadhav|brb is now known as pojadhav | 12:23 | |
| *** yadnesh|afk is now known as yadnesh | 12:26 | |
| frenzy_friday | rlandy|rover, yep | 12:27 |
| marios|ruck | food brb | 12:28 |
| rlandy|rover | dviroel: chandankumar: made some progress on the hive stage stuff yesterday | 12:28 |
| frenzy_friday | dasm|off, hey, do you know if http://10.0.11.209/ - the staging cockpit is down? I cannot ssh the vm | 12:28 |
| rlandy|rover | dviroel: stuck on the cluster claim - NoClusters pending | 12:28 |
| rlandy|rover | will need your eye pls some time this afternoon | 12:29 |
| chandankumar | rlandy|rover: cool, Sandeep and I are still figuring out how to make zuul jobs required in gh pr | 12:29 |
| rlandy|rover | chandankumar: as soon as I have the claim right, will chat with dviroel about how best to put this in a zuul job | 12:30 |
| rlandy|rover | then we can play in our regular tenant | 12:30 |
| chandankumar | rlandy|rover: sounds good :-) | 12:30 |
| rlandy|rover | yay team work | 12:30 |
| dviroel | rlandy|rover: ok, i can check what is hapenning in my afternoon | 12:32 |
| rlandy|rover | dviroel: ack - will ping you - thanks | 12:32 |
| akahat | frenzy_friday, thanks! | 12:40 |
| *** frenzy_friday is now known as frenzy_friday|food | 12:40 | |
| *** arxcruz is now known as arx|2023 | 13:08 | |
| rlandy|rover | arx|2023: enjoy your time off | 13:13 |
| dpawlik | marios|ruck: just FYI still out of space on Quay. Checking why GC did not cleanup deleted images | 13:36 |
| marios|ruck | dpawlik: thanks | 13:37 |
| *** dasm|off is now known as dasm | 13:38 | |
| dasm | o/ | 13:38 |
| dasm | frenzy_friday|food: o/ i was removing mine VMs Friday afternoon. I might've nuked yours too. Lemme check | 13:39 |
| dasm | frenzy_friday|food: In my cmd history I don't see removal of your VM | 13:40 |
| frenzy_friday|food | dasm, ah ok. No, I was trying to access the staging VM (the one that you had set up). So that is deleted | 13:40 |
| frenzy_friday|food | Np, I'll test on .235 | 13:40 |
| dasm | ok | 13:41 |
| rlandy|rover | arx|2023: akahat: bhagyashris: chandankumar: dasm: dviroel: frenzy_friday|food: jm1[m]: ysandeep: pojadhav: that time of year again - compliance and ethics | 13:50 |
| rlandy|rover | pls check your inbox | 13:51 |
| rlandy|rover | for the link | 13:51 |
| rlandy|rover | please be nice to me and do it :) | 13:51 |
| rlandy|rover | thank you all | 13:51 |
| pojadhav | rlandy|rover, sure | 13:51 |
| ysandeep | rlandy|rover, ack o/ | 13:51 |
| dasm | poor marios|ruck he doesn't have compliance and ethics... He's missing so much! (sarcasm) | 13:58 |
| rlandy|rover | marios|ruck is so ethical, he's above it | 13:59 |
| dasm | :) | 13:59 |
| dasm | Ethics over 9000! | 13:59 |
| bhagyashris | rlandy|rover, ack 0/ | 14:00 |
| pojadhav | rlandy|rover, rcastillo, dviroel : scrum time | 14:00 |
| marios|ruck | rlandy|rover: dasm: ;) my lawyer says I'm not allowed to discuss this any further after the last incident (it was a _donation_ , NOT a bribe but he didn't like that) | 14:03 |
| dasm | marios|ruck: :) | 14:06 |
| *** yadnesh is now known as yadnesh|away | 14:21 | |
| dpawlik | marios|ruck: temporary changing the pruner job to 6 days | 14:29 |
| marios|ruck | dpawlik: ok | 14:29 |
| marios|ruck | dpawlik: mm 6 days is pretty tight though | 14:29 |
| marios|ruck | dpawlik: ok but lets update as soon as we can? | 14:30 |
| dpawlik | we can increase later | 14:33 |
| dpawlik | so far we don't know if there is a bug in quay and GC | 14:34 |
| dpawlik | or the Quay registry really needs to have bigger volume than 1 TB | 14:34 |
| dpawlik | but it would be odd due there is a lot free space on old registry | 14:34 |
| *** pojadhav is now known as pojadhav|afk | 14:38 | |
| akahat | reviewbot, add to review list https://review.rdoproject.org/r/c/config/+/46348 | 14:50 |
| reviewbot | I have added your review to the Review list | 14:50 |
| chandankumar | see ya! | 14:52 |
| *** ysandeep is now known as ysandeep|dinner | 14:52 | |
| rlandy|rover | marios|ruck: periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-network-master passed now | 14:55 |
| rlandy|rover | periodic-tripleo-ci-centos-9-standalone-full-tempest-api-network-wallaby failed again | 14:55 |
| marios|ruck | rlandy|rover: thanks, yeah i saw it was running and was waiting for it. not very consistent in the last 3 fails so good | 14:56 |
| dpawlik | marios|ruck: one more question: comparing to old registry, are you pushing now more container images or images are bigger? | 14:56 |
| rlandy|rover | marios|ruck: ok - will rekick the other three failing job | 14:56 |
| rlandy|rover | jobs | 14:56 |
| rlandy|rover | maybe we get a master promo | 14:56 |
| marios|ruck | dpawlik: not that am aware of for either more or bigger | 14:56 |
| rlandy|rover | will use same tp | 14:57 |
| rlandy|rover | marios|ruck: ha - wallaby c9 network is not in need of promo | 14:58 |
| marios|ruck | :/ | 14:59 |
| rlandy|rover | dpawlik: welcome back!!!! we missed you | 15:06 |
| dpawlik | hey rlandy|rover! Thanks \o/ | 15:35 |
| dpawlik | marios|ruck: I updated the launchapad bug. The Quay got a bug, that was fixed in new version | 15:35 |
| dpawlik | I will make a tests today | 15:36 |
| dpawlik | so maybe tomorrow Santa will bring luck and the upgrade will be not problematic | 15:36 |
| *** pojadhav|afk is now known as pojadhav | 15:40 | |
| * dviroel lunch | 16:00 | |
| *** dviroel is now known as dviroel|lunch | 16:00 | |
| rlandy|rover | marios|ruck: hey - probably your EoD soon | 16:12 |
| rlandy|rover | where should I pick up? | 16:12 |
| marios|ruck | rlandy|rover: https://code.engineering.redhat.com/gerrit/c/tripleo-ansible/+/437044 (and help upstream through gate before merging that?) | 16:13 |
| rlandy|rover | wallaby network still ut - although rr doesn;t show it | 16:13 |
| marios|ruck | rlandy|rover: for the 17.1/9 d/stream cherrypick | 16:13 |
| *** ysandeep|dinner is now known as ysandeep | 16:13 | |
| rlandy|rover | marios|ruck: I can click submit there if needed - is that what you want? https://code.engineering.redhat.com/gerrit/c/tripleo-ansible/+/437044? | 16:14 |
| marios|ruck | rlandy|rover: fulton tracking see rhos-delivery | 16:14 |
| rlandy|rover | marios|ruck: also wrt 17.1 ... all clear other than jenkins ... https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-periodic-integration-rhos-17.1-rhel9&skip=0 | 16:14 |
| rlandy|rover | ack | 16:14 |
| *** ysandeep is now known as ysandeep|out | 16:15 | |
| marios|ruck | rlandy|rover: also wallaby/8 still having problems promoting cc frenzy_friday|food dpawlik credentials issue? http://promoter.rdoproject.org/promoter_logs/centos8_wallaby.log unauthorized: Could not find robot with username: tripleozedcentos9+cirobot and supplied password.\nGet \"https://quay.rdoproject.org/v2/tripleowallabycentos8/openstack-barbican-keystone-listener/manifests/ | 16:16 |
| frenzy_friday|food | marios|ruck, ohhh! Need to update tokens on promoter too! | 16:17 |
| frenzy_friday|food | Lemme find out where, from bitwarden | 16:17 |
| dpawlik | marios|ruck: temporary stop the service | 16:17 |
| dpawlik | making backup and triggering upgrade to new version | 16:18 |
| marios|ruck | rlandy|rover: need sthing else from me? | 16:20 |
| frenzy_friday|food | marios|ruck, I will update the tokens generated today on the promoter. But why is wallaby 8 looking for the zed username? | 16:20 |
| rlandy|rover | marios|ruck: don't think so - I'll just follow thrugh on trying to get 17.1 patch into integration line | 16:20 |
| rlandy|rover | marios|ruck: master network shoudl get through | 16:20 |
| rlandy|rover | following wallaby network | 16:21 |
| rlandy|rover | rest should be ok | 16:21 |
| rlandy|rover | marios|ruck: anything else I missed? | 16:21 |
| frenzy_friday|food | dpawlik, you mean you are restarting the quay right? | 16:22 |
| dpawlik | yes | 16:22 |
| frenzy_friday|food | cool | 16:22 |
| dpawlik | frenzy_friday|food: doing backup + upgrade | 16:24 |
| *** frenzy_friday|food is now known as frenzy_friday | 16:24 | |
| marios|ruck | rlandy|rover: yes and train waiting on the minor update fix | 16:25 |
| marios|ruck | rlandy|rover: o/ | 16:25 |
| rlandy|rover | marios|ruck++ thanks | 16:26 |
| rlandy|rover | have a good night | 16:26 |
| marios|ruck | frenzy_friday: will followup tomorrow | 16:28 |
| marios|ruck | rlandy|rover: have a good one | 16:29 |
| frenzy_friday | marios|ruck, thanks, 0/ | 16:29 |
| dpawlik | upgrade done | 16:30 |
| dpawlik | cc frenzy_friday, marios|ruck | 16:30 |
| frenzy_friday | dpawlik, thnaks. I can access quay now | 16:30 |
| dpawlik | onlt 4.2 GB available | 16:30 |
| dpawlik | hope it will cleanup soon | 16:30 |
| *** marios|ruck is now known as marios|out | 16:35 | |
| dpawlik | frenzy_friday: it can happen that it will be out of space once again. If yes, tomorrow I will add small volume | 16:46 |
| frenzy_friday | ack, thanks | 16:46 |
| dpawlik | it is growing up very very slow | 16:46 |
| *** dviroel|lunch is now known as dviroel | 16:51 | |
| dpawlik | quay service has been restarted due it is applying few changes. Should be ok now | 16:59 |
| rlandy|rover | lunch - brb | 17:03 |
| dviroel | rlandy|rover: looking at hive config | 17:13 |
| rlandy|rover | dviroel: will need to send you the one I am using | 17:14 |
| rlandy|rover | dviroel: I am using a modification of the one chandankumar sent | 17:15 |
| dviroel | rlandy|rover: i see stage-1 | 17:15 |
| rlandy|rover | dviroel: ack | 17:15 |
| rlandy|rover | oc apply -f stage-1-claim.yaml worked | 17:16 |
| rlandy|rover | to define claim | 17:16 |
| rlandy|rover | must have missed a step to actually claim the cluster | 17:16 |
| rlandy|rover | or start it | 17:16 |
| dviroel | rlandy|rover: "stage-1 1 0 0 oooci.ccitredhat.com stage-imageset-497" | 17:17 |
| dviroel | no clusters created yet | 17:17 |
| rlandy|rover | dviroel: ack | 17:17 |
| dviroel | rlandy|rover: because there is an issue with configs | 17:17 |
| rlandy|rover | dviroel: pasting config | 17:17 |
| dviroel | not needed | 17:18 |
| dviroel | oc get cp stage-1 -o json | 17:18 |
| dviroel | gives the current config | 17:18 |
| dviroel | if you look at the end, on status | 17:18 |
| dviroel | "message": "{\"BrokenByCloud\":[],\"BrokenBySyntax\":[],\"Missing\":[\"stage-1\"]}", | 17:18 |
| dviroel | stage-1 should be you customization | 17:18 |
| dviroel | $ oc get clusterdeploymentcustomization | 17:19 |
| dviroel | rlandy|rover: you need to create a ClusterDeploymentCustomization | 17:19 |
| rlandy|rover | dviroel: I have one .. pasting | 17:20 |
| rlandy|rover | in pvt | 17:20 |
| frenzy_friday | rlandy|rover, I am leaving for the day. I have updated the new tokens on thr promoter. Lemme know if it still complains about unauthorized tokens | 17:44 |
| rlandy|rover | frenzy_friday++ thanks | 17:45 |
| rlandy|rover | will check | 17:45 |
| *** dviroel is now known as dviroel|afk | 17:52 | |
| rlandy|rover | 2022-12-05 17:44:29,950 p=2359170 u=promoter n=ansible | localhost : ok=10 changed=2 unreachable=0 failed=1 skipped=3 rescued=0 ignored=0 | 17:53 |
| rlandy|rover | was the last fail | 17:53 |
| rlandy|rover | frenzy_friday; let's see what the next run does | 17:53 |
| rlandy|rover | k - now actually trying to have lunch | 17:54 |
| rlandy|rover | biab | 17:54 |
| *** jpena is now known as jpena|off | 17:58 | |
| rlandy|rover | unauthorized: Could not find robot with username | 18:13 |
| -opendevstatus- NOTICE: The lists.opendev.org and lists.zuul-ci.org sites will be offline briefly for migration to a new server | 20:00 | |
| *** dviroel|afk is now known as dviroel | 22:01 | |
| * dviroel o/ | 22:01 | |
| dasm | dviroel: o/ | 22:01 |
| * dviroel we won o/ | 22:04 | |
| dasm | dviroel: congrats! | 22:21 |
| dviroel | \o/ | 22:22 |
| * dasm => offline | 22:51 | |
| dasm | o/ | 22:51 |
| *** dasm is now known as dasm|off | 22:51 | |
| rlandy|rover | dviroel: yaya | 23:37 |
| rlandy|rover | knew you would :) | 23:37 |
| *** rlandy|rover is now known as rlandy|out | 23:37 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!