Thursday, 2021-11-11

opendevreviewDavid Vallee Delisle proposed openstack/tripleo-ansible master: Making sure we use the first gateway of the list  https://review.opendev.org/c/openstack/tripleo-ansible/+/81750001:06
opendevreviewMerged openstack/tripleo-ansible stable/wallaby: Correct how we fetch certificate for cephadm/rgw  https://review.opendev.org/c/openstack/tripleo-ansible/+/81688001:12
opendevreviewMerged openstack/tripleo-heat-templates master: Add rgw max attributes constraints  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81739501:37
bshepharnoama_:03:16
bshephar[root@overcloud-dellcompute-0 qemu]# virsh -c qemu:///system list03:16
bshephar Id   Name                State03:16
bshephar-----------------------------------03:16
bshephar 2    instance-00000005   running03:16
*** ysandeep|out is now known as ysandeep03:16
bshephar 40   instance-00000507   running03:16
bshephar 65   instance-00000520   running03:16
bshephar 67   instance-00000522   running03:16
bshephar 68   instance-00000523   running03:16
bshephar 69   instance-00000524   running03:16
bshephar 70   instance-00000525   running03:16
bshephar 77   instance-0000052b   running03:16
bshepharYou can run it directly from the host03:16
bshepharOr from the nova_compute container:03:17
bshephar[root@overcloud-dellcompute-0 qemu]# podman exec nova_compute virsh -c qemu:///system list03:17
bshephar Id   Name                State03:17
bshephar-----------------------------------03:17
bshephar 2    instance-00000005   running03:17
bshephar 40   instance-00000507   running03:17
bshephar 65   instance-00000520   running03:17
bshephar 67   instance-00000522   running03:17
bshephar 68   instance-00000523   running03:17
bshephar 69   instance-00000524   running03:17
bshephar 70   instance-00000525   running03:17
bshephar 77   instance-0000052b   running03:17
*** ysandeep is now known as ysandeep|afk04:06
opendevreviewJohn Fulton proposed openstack/python-tripleoclient master: Support custom network names with deployed ceph  https://review.opendev.org/c/openstack/python-tripleoclient/+/81713104:49
*** ysandeep|afk is now known as ysandeep04:55
opendevreviewMerged openstack/tripleo-heat-templates master: Add note indicating only Controllers use network VIPs  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/79083305:31
opendevreviewMerged openstack/os-net-config master: Add ifcfg-* scripts on boot for Mellanox NIC interface  https://review.opendev.org/c/openstack/os-net-config/+/81383805:31
*** ykarel__ is now known as ykarel05:32
opendevreviewFrancesco Pantano proposed openstack/tripleo-heat-templates stable/wallaby: Add rgw max attributes constraints  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81753406:51
opendevreviewchandan kumar proposed openstack/tripleo-ci master: [WIP] use tripleo-repos in CS9 main deployment  https://review.opendev.org/c/openstack/tripleo-ci/+/81745506:59
Tengumarios: heya! will check that ptmx pile of things today. Hopefully I'll come with a solution NOT involving modifying the mode/context on the host (remember: in those affected containers, /dev is bind-mounted, meaning any edition onto it is reported on the host)07:01
mariosack Tengu add us to the review? so should we hold on chandankumar evil (666) workaround or should we go with that? [EDIT]: yes i just read your comments @ https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/817141/2#message-cb4df8d19a0a5d50fe3c19fd197f78a04958213f :D07:08
mariosTengu: hmm bogdan isn't around yet? 07:08
marioswe need to bribe him about his -107:08
Tenguchandankumar: care to update https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/817141 commit message and push "workaround"? I can update it in order to be clearer on the /dev/ptmx vs /dev/pts/ptmx thing as well.07:11
chandankumarTengu: will do that, just rechecked the patch07:11
chandankumar*testproject patch 07:12
Tenguchandankumar: ok!07:12
TenguI'll follow, but if I don't act on the review within an hour, just ping me - I'll dig into the selinux arcanes of the containers.07:12
chandankumarsure!07:14
Tenguwokay... so. apparently, the native selinux context of iscsid is "iscsid_t" - which sounds kind of legit. let's see if I can start a container with this context now.07:27
Tengufirst: tea. 07:28
*** pojadhav is now known as pojadhav|afk07:52
opendevreviewchandan kumar proposed openstack/tripleo-quickstart-extras master: [workaround]Set permission for /dev/pts/ptmx to 666  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81714108:06
chandankumarTengu: marios ^^08:06
opendevreviewlikui proposed openstack/python-tripleoclient master: Replace deprecated assertEquals  https://review.opendev.org/c/openstack/python-tripleoclient/+/81737808:06
Tenguchandankumar++ thanks - I'm a bit picky on that one, but... let's be 100% clear :)08:07
opendevreviewchandan kumar proposed openstack/tripleo-ansible master: Explicitly define infra_image in containers.conf  https://review.opendev.org/c/openstack/tripleo-ansible/+/81720208:08
*** pojadhav|afk is now known as pojadhav08:30
Tenguchandankumar: hmm. for now I don't have cs9 things in my lab - guess I'll have to use a testproject thing for my tests. I don't remember how to do so though :/.08:46
Tenguchandankumar: 1- push a change to the actual repo; 2- then find that testproject thing, make a dumb job with a depends-on [1] ?08:46
chandankumarTengu: yes, you can push the change to actual repo08:47
chandankumarTengu: then put depends on there 08:47
chandankumarcurrent test example is https://review.rdoproject.org/r/c/testproject/+/3492608:47
chandankumarlet me know which job to test it08:48
chandankumarI will put the depends on there08:48
chandankumargit clone "https://review.rdoproject.org/r/testproject"08:48
chandankumarand cd testproject 08:48
chandankumarand add .zuul.yaml 08:48
chandankumarwith in .zuul.yaml put the desired job or 08:48
chandankumarand do git-review08:49
chandankumaryou are done08:49
Tenguchandankumar++ thanks - tweaking a patch first.08:49
chandankumarTengu: marios has documented this https://docs.openstack.org/tripleo-docs/latest/ci/reproduce-ci.html08:49
chandankumarsorry better this one https://docs.openstack.org/tripleo-docs/latest/ci/chasing_promotions.html#hack-the-promotion-with-testproject08:50
opendevreviewCedric Jeanneret proposed openstack/tripleo-heat-templates master: Try to move iscsid container to its native context  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81752208:50
Tenguah, thanks08:50
* Tengu bookmarks08:50
Tenguchandankumar: what's the test job for the chmod 0666?08:52
Tenguah, seems to be the one you actually linked.08:52
Tenguok08:52
chandankumarperiodic-tripleo-ci-centos-9-containers-multinode-master08:52
chandankumaryes just remove tqe patch08:53
chandankumaradd your patch there08:53
Tenguchandankumar: I'd rather not block your own tests.08:54
Tengulet's see.08:54
Tenguhttps://review.rdoproject.org/r/c/testproject/+/3669808:55
Tengubrb - breakfast.08:56
chandankumargreat, thanks!08:57
Tenguguess I'll have to start building cs9 containers.09:07
chandankumaryou can pull it from rdo registry?09:08
Tengutoo slow :)09:08
*** ysandeep is now known as ysandeep|lunch09:09
chandankumarTengu: I will ask arxcruz to move the containers to quay once we have promotion09:09
Tengustill - I have a thing building master containers here.09:09
Tengujust a matter of moving one of my NUC to cs9, and get a namespace for cs9 to not conflict with the others.09:10
TenguI'll probably take some time during my PTO for that.09:10
chandankumarok09:11
Tenguand maybe my node provisioning issue will also be solved with cs9 containers.... though I have some doubts.09:11
Tengusomething changed, and I'm now unable to provision my OC nodes - and I don't understand why.09:11
chandankumarTengu: Updating one of my patch and will recheck yours one09:13
opendevreviewchandan kumar proposed openstack/tripleo-ci master: Lay down EL9 compose repos before TQ starts  https://review.opendev.org/c/openstack/tripleo-ci/+/81745509:13
opendevreviewJiri Podivin proposed openstack/tripleo-ci master: Adding VF directories into the log collection list  https://review.opendev.org/c/openstack/tripleo-ci/+/81739209:17
opendevreviewMerged openstack/puppet-tripleo master: [Pure Storage] Add new params for FlashArray Cinder driver  https://review.opendev.org/c/openstack/puppet-tripleo/+/81683509:19
*** ysandeep|lunch is now known as ysandeep09:38
opendevreviewMarios Andreou proposed openstack/tripleo-ci master: Wire up centos9 master buildimage jobs for check and gate  https://review.opendev.org/c/openstack/tripleo-ci/+/81510809:42
opendevreviewAdriano Petrich proposed openstack/validations-libs master: Add the community validation paths  https://review.opendev.org/c/openstack/validations-libs/+/81583209:53
opendevreviewShnaidman Sagi (Sergey) proposed openstack/tripleo-ansible master: Use shell module to generate keys  https://review.opendev.org/c/openstack/tripleo-ansible/+/81721310:00
sshnaidmhi, all, who can review octavia related patch? ^10:01
*** ykarel is now known as ykarel|lunch10:05
jpodivinmarios: hi. I've got the time info about the validations catalogue execution. 10:10
jpodivinmarios: all things taken into account, with CLI tests and 63 validations executed, we are hitting about 5-6 mins. 10:12
jpodivinmarios: compared to the job total >3 hrs it's negligible. 10:13
mariosjpodivin: nice 10:16
mariosjpodivin: can you get some logs/pointers together that shows this and maybe consider bringing to our next tuesday community call? 10:16
mariosjpodivin: or whenever you folks want to sync again10:17
jpodivinmarios: sure, np, we will probably go on the sync next week. 10:17
jpodivinalthough I will be on pto so matbu will have handle it. 10:17
jpodivinmarios: the links to logs are in the hackmd along with couple of words about further options. I'm not sure how long will the execution log stay up, so I've made a backup of the main logfile in the pastebin. 10:22
opendevreviewHarald Jensås proposed openstack/tripleo-common master: Deactivate LV's before disconnecting nbd device  https://review.opendev.org/c/openstack/tripleo-common/+/81745610:25
mariosjpodivin: thanks10:26
opendevreviewHarald Jensås proposed openstack/tripleo-quickstart master: Re-enable protected resource types  https://review.opendev.org/c/openstack/tripleo-quickstart/+/80770410:27
*** dviroel|out is now known as dviroel10:45
opendevreviewamolkahat proposed openstack/openstack-tempest-skiplist master: Remove entry for Queens  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/81490310:48
opendevreviewJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Adapt standalone-upgrade to run tripleo operator  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81755210:49
opendevreviewamolkahat proposed openstack/openstack-tempest-skiplist master: Added LoadBalancerScenarioTest in skiplist  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/81700210:51
*** rlandy is now known as rlandy|ruck11:09
opendevreviewDouglas Viroel proposed openstack/tripleo-repos master: yum-config: add support to download repo from url  https://review.opendev.org/c/openstack/tripleo-repos/+/81621111:09
opendevreviewJose Luis Franco proposed openstack/tripleo-ansible stable/ussuri: Add back testing for tripleo-ansible modules.  https://review.opendev.org/c/openstack/tripleo-ansible/+/81753911:19
*** ykarel|lunch is now known as ykarel11:19
opendevreviewJose Luis Franco proposed openstack/tripleo-ansible stable/ussuri: Add tripleo_dnf_stream Ansible module.  https://review.opendev.org/c/openstack/tripleo-ansible/+/81755411:26
opendevreviewamolkahat proposed openstack/openstack-tempest-skiplist master: Remove entry for Queens  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/81490311:49
opendevreviewMerged openstack/tripleo-ci-health-queries master: Updating no key value pair query for c9 sc001  https://review.opendev.org/c/openstack/tripleo-ci-health-queries/+/81685111:53
opendevreviewJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Adapt standalone-upgrade to run tripleo operator  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81755212:40
opendevreviewShnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart master: Run CI with Ansible 2.11  https://review.opendev.org/c/openstack/tripleo-quickstart/+/76948012:41
opendevreviewShnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Run CI with Ansible 2.11 (extras)  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81698812:41
opendevreviewShnaidman Sagi (Sergey) proposed openstack/tripleo-quickstart-extras master: Run CI with Ansible 2.11 (extras)  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81698812:42
*** amoralej|off is now known as amoralej12:49
rlandy|ruckhjensas_: sshnaidm: https://review.opendev.org/c/openstack/tripleo-common/+/817456 - looks like we're good to go here13:01
sshnaidmrlandy|ruck, yeah, finally..13:02
Tenguchandankumar: https://logserver.rdoproject.org/98/36698/1/check/periodic-tripleo-ci-centos-9-containers-multinode-master/feab77b/logs/undercloud/home/zuul-worker/overcloud_deploy.log.txt.gz   that's yet another error, is it? because this time it seems to be running out of PTY on the UC node... ?13:10
Tenguor am I missing data?13:10
jpodivinI have a patch for the tripleo-ci. 13:10
jpodivinextending log collection to stuff produced by VF https://review.opendev.org/c/openstack/tripleo-ci/+/81739213:11
Tenguah. nope. that's the neutron.agent.dhcp.agent. so this IS in a container.13:11
chandankumarTengu: it is the same issue https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81714113:11
chandankumarhttps://launchpad.net/bugs/195017613:11
Tenguor.. wait. 13:11
Tenguchandankumar: that sounds more like "on the node" than within a container....13:12
jpodivininput would be most appreciated ^^^^^13:12
chandankumarTengu: then just workaround is enough?13:12
chandankumarworkaround fixes the issue13:13
Tenguchandankumar: lemme check in the log of this job.13:13
Tengubecause if it's "just" on the CI, yeah, that chmod would be enough - though..... iirc it's a permission denied.13:14
chandankumarTengu: the same issue is mentioned in bandini cs9 doc also13:14
Tengugrumpf...13:15
Tenguchandankumar: nope.13:21
Tengunot the same.13:21
Tenguhttps://logserver.rdoproject.org/98/36698/1/check/periodic-tripleo-ci-centos-9-containers-multinode-master/feab77b/logs/undercloud/var/log/extra/selinux_denials.txt.gz  no denials related to devpts_t13:21
Tenguchandankumar: I think there's a huge, huge, huge mix of issues around the tty things.13:22
Tenguespecially since the error reported in the doc seems to related to sshd, NOT container_t13:22
bandiniTengu: the doc has not been updated. The issue is another container messes up the /dev/...ptmx. Workaround to avoid it for the time being https://gitlab.cee.redhat.com/osp17/rhel9-playground/-/commit/24ae8f32f3ff5505ae3c311a6144eb3cf7f23a9413:23
Tengubandini: pfff..13:24
Tenguthat's just.... raaahhh13:24
bandininot sure i follow13:24
Tengubandini: I'm also lost.13:25
bandinithe problem is the permissions on /dev/pts/ptmx are wrong. Chances are it is due to one of those containers that bind mounts /dev. Selinux might or might not be involved. I.e. it needs root causing13:26
Tenguyeah. so. probably NOT selinux (at least for https://logserver.rdoproject.org/98/36698/1/check/periodic-tripleo-ci-centos-9-containers-multinode-master/feab77b/logs/undercloud/home/zuul-worker/overcloud_deploy.log.txt.gz)13:26
bandinithe workaround was just to get the deployment going and find the next issues13:26
Tengubut yeah. the fact we're mounting /dev:/dev in containers is a source of potential issues.13:26
bandiniyeah selinux might or might not be involved. once enabled we could suddenly ssh to the uc, so we assumed it was at least partially involved13:27
bandinithat might or might not be the case13:27
Tengu"yay"13:27
bandinis/enabled/disabled/ me--13:28
chandankumarI have new issue now https://logserver.rdoproject.org/26/34926/13/check/periodic-tripleo-ci-centos-9-containers-multinode-master/b11a222/logs/subnode-1/home/zuul-worker/dlrn.log.txt.gz if want to take a look13:28
chandankumar OSError: out of pty devices13:28
bandinichandankumar: that is issue 81 13:29
bandinior surely looks a lot like it13:29
chandankumarbandini: then I think we need to do the same on all nodes13:30
bandiniwcould be yeah13:31
chandankumarTengu: sorry for the bug mess .13:34
*** NobodyCam_ is now known as NobodyCam13:43
*** mwhahaha_ is now known as mwhahaha13:43
*** donnyd_ is now known as donnyd13:43
cloudnullmornings 13:44
*** JqckB_ is now known as JqckB13:45
*** gmann_ is now known as gmann13:45
*** gouthamr_ is now known as gouthamr13:45
hjensas_rlandy|ruck: cool, let's hope that was the last bug related to modify-image for a while. :)13:45
*** erbarr_ is now known as erbarr13:46
*** johnsom_ is now known as johnsom13:46
*** simondodsley_ is now known as simondodsley13:46
opendevreviewMarios Andreou proposed openstack/tripleo-quickstart-extras master: Use whole_disk_images if this is passed through job vars  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81724313:46
*** pojadhav is now known as pojadhav|sick13:46
*** rcastillo|away is now known as rcastillo13:48
*** dviroel_ is now known as dviroel13:49
opendevreviewMarios Andreou proposed openstack/tripleo-quickstart master: Remove explicit set for whole_disk_image: true  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81631313:49
mariosrlandy|ruck: akahat|rover: ysandeep: chandankumar: ^^ 13:49
rlandy|ruckhjensas_: yeah - will close that bug out once we see some test results13:51
rlandy|ruckhjensas_: we have a few other whole_disk_image issues13:51
rlandy|ruckwill bring them to the next bug scrub if needed13:51
*** jpena|off is now known as jpena14:03
opendevreviewMerged openstack/tripleo-heat-templates stable/wallaby: Add rgw max attributes constraints  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81753414:14
opendevreviewMerged openstack/tripleo-quickstart master: Feature matrix regeneration  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81716514:14
opendevreviewJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Adapt standalone-upgrade to run tripleo operator  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81755214:28
*** ykarel is now known as ykarel|away14:46
sshnaidmslaweq, hi, do you know maybe who can help with octavia failure investigation? https://review.rdoproject.org/zuul/build/ccde99b64d9d4af78291cb81cbb290a314:47
opendevreviewJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Adapt standalone-upgrade to run tripleo operator  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81755214:51
opendevreviewJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Adapt standalone-upgrade to run tripleo operator  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81755214:53
*** mnaser_ is now known as mnaser14:58
opendevreviewJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Adapt standalone-upgrade to run tripleo operator  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81755215:01
opendevreviewJiri Podivin proposed openstack/validations-common master: CI task for execution of all packaged validations  https://review.opendev.org/c/openstack/validations-common/+/81704115:15
*** ysandeep is now known as ysandeep|out15:22
*** sshnaidm is now known as sshnaidm|afk15:27
opendevreviewJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Adapt standalone-upgrade to run tripleo operator  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81755215:34
opendevreviewSimon Dodsley proposed openstack/tripleo-heat-templates master: [Pure Storage] Add new FlashArray Cinder driver params  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81683115:51
*** soniya29|ruck is now known as soniya29|ruck|out15:54
*** dviroel is now known as dviroel|lunch16:13
opendevreviewSimon Dodsley proposed openstack/tripleo-heat-templates master: [Pure Storage] Add new FlashArray Cinder driver params  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81683116:30
*** dviroel|lunch is now known as dviroel16:58
*** marios is now known as marios|out17:02
rlandy|rucksshnaidm|afk: about octavia failure - scenario010 .... we have an open cix17:23
rlandy|ruckhttps://trello.com/c/tx4SwUQp/2109-cixlp1895248tripleociproa-master-loadbalancerscenariotesttestloadbalanceripv4crud-showloadbalancer-provisioningstatus-updated-to17:23
rlandy|ruckthat test has been failing for ages17:23
rlandy|ruckwe have it passing in periodic17:23
rlandy|ruckrunning the kvm test helps17:23
*** amoralej is now known as amoralej|off17:41
*** carloss_ is now known as carloss17:53
*** jpena is now known as jpena|off18:01
opendevreviewRonelle Landy proposed openstack/tripleo-quickstart master: Remove explicit set for whole_disk_image: true  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81631318:07
opendevreviewRonelle Landy proposed openstack/tripleo-quickstart master: Remove explicit set for whole_disk_image: true  https://review.opendev.org/c/openstack/tripleo-quickstart/+/81631318:24
*** dviroel is now known as dviroel|afk20:12
mwhahahacloudnull: what's the status of our libssh packaging stuff?21:26
cloudnulllibssh should be good 21:27
cloudnulland with the removal of our other deps we should be in the clear for deps 21:27
cloudnullhttps://paste.opendev.org/show/810956/21:28
cloudnullI've been going through the directord code-base and removing / reworking deps that we dont have packaged already. so hopefully it should be fairly straight forward to package everything21:32
mwhahahaok was thinking we might need a small library in tripleo common/client to handle some bulk actions for bootstraping/cli actions21:39
cloudnullmwhahaha I think we could extend the directord lib integration to support bootstrapping. atm it just runs orchestrations, but pulling bootstrap.Bootstrap(args=args) should be fairly straight forward. 21:45
mwhahahacloudnull: yea that' dbe fine too. i was looking at the admin command21:53
opendevreviewRonelle Landy proposed openstack/tripleo-quickstart-extras master: Do not use whole_disk_image for libvirt image upload  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81765122:27
opendevreviewTakashi Kajinami proposed openstack/tripleo-heat-templates stable/wallaby: Keystone: Remove unused container_puppet_tasks  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/81754823:05
opendevreviewTakashi Kajinami proposed openstack/puppet-tripleo master: Clean up unused heat_admin_* parameters  https://review.opendev.org/c/openstack/puppet-tripleo/+/81765323:09
opendevreviewAlex Schultz proposed openstack/tripleo-common master: task-core file generation  https://review.opendev.org/c/openstack/tripleo-common/+/79872123:23
*** rlandy|ruck is now known as rlandy|ruck|afk23:32

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!