| opendevreview | OpenStack Release Bot proposed openstack/devstack-plugin-ceph stable/xena: Update .gitreview for stable/xena https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/811074 | 06:45 |
|---|---|---|
| opendevreview | OpenStack Release Bot proposed openstack/devstack-plugin-ceph stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/811075 | 06:45 |
| *** jpena|off is now known as jpena | 07:29 | |
| frickler | yoctozepto: sorry for not adding you as co-author directly, I should've thought of that myself. also likely we want that job n-v like the other platform jobs | 09:00 |
| frickler | also why in $%&/= does this start failing with concurrency=2 now, too? seems this is very sensitive on the ordering of tempest tests? | 09:01 |
| frickler | also why is debian at the same time both so much faster and so much more memory hungry than others? | 09:01 |
| *** bhagyashris is now known as bhagyashris|rover | 09:32 | |
| *** bhagyashris is now known as bhagyashris|rover | 10:38 | |
| *** dviroel|out is now known as dviroel | 11:30 | |
| *** jpena is now known as jpena|lunch | 11:32 | |
| *** pojadhav is now known as pojadhav|brb | 11:35 | |
| opendevreview | Christian Schwede proposed openstack/tempest master: Fix IPv6 localhost replacement in Swift container sync test https://review.opendev.org/c/openstack/tempest/+/810681 | 11:47 |
| yoctozepto | frickler: yeah, it was sensitive to it before | 11:57 |
| yoctozepto | and the other question also bugs me | 11:57 |
| yoctozepto | it probably explains the first one though | 11:58 |
| yoctozepto | different speed - different races | 11:58 |
| yoctozepto | i.e., the whole CI runs on luck ;-) | 11:58 |
| opendevreview | Soniya Murlidhar Vyas proposed openstack/tempest master: This patch creates the network v2.0 floating-ips-port-forwarding client. https://review.opendev.org/c/openstack/tempest/+/810015 | 11:59 |
| opendevreview | Lee Yarwood proposed openstack/grenade master: zuul: Add neutron-trunk to base grenade job https://review.opendev.org/c/openstack/grenade/+/811117 | 12:15 |
| soniya29 | kopecmartin, gmann, please review these patches - https://review.opendev.org/c/openstack/patrole/+/800226, https://review.opendev.org/c/openstack/patrole/+/795548 whenever you have time. | 12:22 |
| *** jpena|lunch is now known as jpena | 12:25 | |
| *** pojadhav|brb is now known as pojadhav | 12:33 | |
| monica_pardhi | We are trying to execute ironic-tempest-plugin standalon test case(BaremetalDriverDirectWholedisk), where we are observing timeout error while pinging the deployed instance. Tried by increasing ping_timeout ,still observed the same issue. | 12:51 |
| monica_pardhi | Any suggestion to overcome this issue? | 12:52 |
| kopecmartin | gmann: i forgot there is a public holiday tomorrow in my country, should I cancel the office hour? | 14:24 |
| yoctozepto | we have one discussion point with frickler (mariadb) but it's not urgent | 14:36 |
| gmann | kopecmartin: i am ok to cancel. | 15:08 |
| gmann | monica_pardhi> if you see this msg. can you point me the exact test so that we can see if that use configurable (build_interval or so) for that ping. | 15:09 |
| *** redrobot is now known as Guest1129 | 15:09 | |
| frickler | yoctozepto: regarding debian, I think I'll revert to serial and bump the job timeout to 2.5h for now, wdyt? | 15:16 |
| yoctozepto | frickler: I think it's the only reasonable choice for now | 15:22 |
| opendevreview | Dr. Jens Harbott proposed openstack/devstack master: Enable running on Debian Bullseye https://review.opendev.org/c/openstack/devstack/+/789083 | 15:26 |
| frickler | yoctozepto: btw., did you ever talk to neutron about what is missing for OVN on Debian? | 15:27 |
| yoctozepto | frickler: I talked to Debian because I know it's the packages that are missing :D | 15:27 |
| yoctozepto | zigo said they are packaged but stuck in verification queues | 15:28 |
| yoctozepto | but it was some time ago | 15:28 |
| yoctozepto | they might have moved since | 15:28 |
| frickler | yoctozepto: ah, o.k., so might be solved by running from source or using another repo possibly. but let's first see for a bit how the current job fares I guess | 15:32 |
| yoctozepto | frickler: yeah, I doubt lack of OVN is what's bringing this madness ;-) | 15:35 |
| zigo | frickler: yoctozepto: OVN is still in the NEW queue: https://ftp-master.debian.org/new.html | 15:40 |
| yoctozepto | thanks zigo | 15:40 |
| yoctozepto | as I said, there is no extreme rush to get it released | 15:40 |
| zigo | It got rejected once, so I had to re-upload. Hopefully, it will go through soon... | 15:40 |
| yoctozepto | fingers crossed! | 15:40 |
| yoctozepto | frickler: ooh, so it was oom killer also with 2? I did not realise! | 16:03 |
| yoctozepto | then I wonder what it is | 16:03 |
| yoctozepto | as we don't really spawn any big vms | 16:04 |
| amodi | artom lyarwood rtm now? https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/786979 | 16:15 |
| artom | amodi, yeah, lemme take a look, just sorting out this CIX | 16:16 |
| amodi | sure | 16:17 |
| lyarwood | amodi: you might want to throw a DNM patch on top of that so the test is actually running somewhere | 16:21 |
| opendevreview | Artom Lifshitz proposed openstack/tempest master: Add floating IP waiter https://review.opendev.org/c/openstack/tempest/+/811191 | 16:21 |
| amodi | lyarwood: oh good idea, yes will do! | 16:21 |
| lyarwood | and a quick nit on the zuul config for the job, I thought we agreed you only need to update the compute config? | 16:22 |
| amodi | lyarwood: so i replied on the patch, so u only want the $NOVA_CONF: section removed right? and also from the group-vars/devstack_local_conf/post-config/$NOVA_CONF: ? | 16:35 |
| lyarwood | amodi: yeah | 16:35 |
| amodi | lyarwood: ack | 16:36 |
| *** jpena is now known as jpena|off | 16:38 | |
| opendevreview | Archit Modi proposed openstack/whitebox-tempest-plugin master: Add Rbd direct download test https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/786979 | 16:39 |
| opendevreview | Archit Modi proposed openstack/whitebox-tempest-plugin master: DNM: Run whitebox-devstack-ceph-compute-local-ephemeral https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/811193 | 16:39 |
| opendevreview | Dr. Jens Harbott proposed openstack/devstack master: DNM: debugging ooms https://review.opendev.org/c/openstack/devstack/+/811203 | 17:00 |
| frickler | yoctozepto: I'm starting to think that maybe instances aren't getting cleaned up properly or too slow. there's like 8 qemu processes in one of the oom logs. I'm going to hold a node with ^^ for further analysis | 17:00 |
| lyarwood | I'm trying to enable the neutron-trunk service in Nova's multinode grenade job but for whatever reason the devstack plugins from the neutron repo itself don't appear to load | 17:20 |
| lyarwood | does anyone know where or how devstack finds these plugins during a deployment? | 17:21 |
| clarkb | lyarwood: you've explicitly enabled the plugin in the localrc? I believe the enable_plugin directive is what gives devstack the info to do the lookup | 17:22 |
| clarkb | it fetches the repo to its repo dir and then looks for devstack/ contents in that repo | 17:23 |
| lyarwood | clarkb: yeah it's enabled in the localrc for both the old and new versions and listed as an enabled service etc. | 17:23 |
| lyarwood | https://zuul.opendev.org/t/openstack/build/9e193cb9aef2459a8a2dfd3f57d6c6a6/log/controller/logs/old/devstacklog.txt is the initial stable/xena install | 17:23 |
| frickler | neutron service and neutron devstack_plugin isn't the same | 17:23 |
| gmann | yeah, it should be like that enable_plugin neutron https://opendev.org/openstack/neutron | 17:24 |
| lyarwood | right this is working in nova-next without that | 17:24 |
| * lyarwood checks again | 17:25 | |
| frickler | maybe neutron needs to move that service definition from neutron plugin into devstack proper | 17:25 |
| lyarwood | aaaaaaaah there it is | 17:25 |
| lyarwood | sorry it was missing the line gmann just pointed out, couldn't see the wood from the trees | 17:25 |
| yoctozepto | frickler: hmm, debian being faster due to being lazy with memory freeing? sounds feasible | 17:30 |
| opendevreview | Lee Yarwood proposed openstack/grenade master: WIP zuul: Add the neutron devstack plugin and neutron-trunk service https://review.opendev.org/c/openstack/grenade/+/811117 | 17:30 |
| lyarwood | thanks for that gmann btw | 17:40 |
| gmann | lyarwood: np! | 17:41 |
| ecsantos[m] | hi folks, I have a question regarding the devstack CI jobs | 17:49 |
| ecsantos[m] | the devstack job by default uses the ubuntu-focal label. this label's diskimage uses the element openstack-repos to clone the required repos | 17:50 |
| ecsantos[m] | I'm trying to reproduce a devstack job build in my local Zuul, but I'm having a problem | 17:50 |
| ecsantos[m] | openstack-repos defines that the directory to clone the repos should be /opt/git, as per here: https://opendev.org/openstack/project-config/src/branch/master/nodepool/elements/openstack-repos/extra-data.d/50-create-repo-list#L75 | 17:51 |
| ecsantos[m] | but the devstack job expects it to be in /opt/stack, as per here: https://opendev.org/openstack/devstack/src/branch/master/roles/run-devstack/tasks/main.yaml#L9 | 17:51 |
| ecsantos[m] | how did the openinfra folks make this work in the openinfra CI? | 17:51 |
| clarkb | ecsantos[m]: /opt/git is the repo cache on our nodes, not where devsatck wants the repos | 17:52 |
| clarkb | ecsantos[m]: our zuul jobs have a distinct repo setup step that will take the cached repos in the image and update them with the code that should be tested. Then put that in the appropriate location | 17:52 |
| clarkb | none of that should be required though. If you just run devsatck it will clone the repos | 17:53 |
| ecsantos[m] | clarkb: thanks clark! could you point me to the job which moves the repos to /opt/stack? I couldn't find it | 17:54 |
| ecsantos[m] | the devstack job uses the `ERROR_ONCLONE=yes` variable, so I believe it expects the repos to be available offline | 17:54 |
| ecsantos[m] | s/ERROR_ONCLONE/ERROR_ON_CLONE/ | 17:54 |
| clarkb | yes error on clone is there in the zuul run jobs to ensure that only zuul managed code isused otherwise you can test the wrong stuff. | 17:55 |
| clarkb | ecsantos[m]: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/prepare-workspace-git then https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/mirror-workspace-git-repos then https://opendev.org/openstack/devstack/src/branch/master/roles/setup-devstack-source-dirs | 17:58 |
| clarkb | youcan see those all running if you look at a devstack job in zuul | 17:58 |
| clarkb | the first role creates the workspace and the repos. The second one puts the zuul specific code that we want to test in place. The third is devstack's copy as necessary role to ensure the repos are where it wants them | 18:00 |
| ecsantos[m] | clarkb: thanks again! I see the problem, for some reason my Zuul isn't using these zuul-jobs roles (using Software Factory), but now I have a direction | 18:03 |
| ecsantos[m] | having this same error locally: https://zuul.opendev.org/t/openstack/build/2b59223b1c8047929b967ad52fdaed19 | 19:54 |
| ecsantos[m] | but no changes were merged today on openstack/devstack | 19:56 |
| -opendevstatus- NOTICE: Gerrit and Zuul services are being restarted briefly for configuration and code updates but should return to service momentarily | 20:10 | |
| ecsantos[m] | weird, it complains about placement API not running, when it is: https://paste.openstack.org/show/809630/ | 20:21 |
| rosmaita | i'm seeing those failure ecsantos[m] mentioned on all our devstack-based jobs | 21:02 |
| rosmaita | looks like around 1600utc, devstack-based jobs started failing at /stack.sh:main:1455 ... nova-status upgrade check is failing because "Placement API does not seem to be running." | 21:02 |
| rosmaita | https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-storage&job_name=grenade&job_name=openstacksdk-functional-devstack&branch=master&branch=stable%2Fxena | 21:03 |
| rosmaita | anyone have any ideas? | 21:03 |
| clarkb | https://opendev.org/openstack/nova/src/branch/master/nova/cmd/status.py#L178-L192 is where that error message comes from | 21:11 |
| clarkb | https://opendev.org/openstack/keystoneauth/src/branch/master/keystoneauth1/exceptions/http.py#L147-L155 means it got a 404 from the endpoint request? | 21:13 |
| clarkb | https://zuul.opendev.org/t/openstack/build/2b59223b1c8047929b967ad52fdaed19/log/controller/logs/screen-placement-api.txt#257 I'm going to assume that is the issue | 21:15 |
| clarkb | note the url | 21:15 |
| clarkb | almost looks like someone is trying to append a / to the path but replaced the last character in the url with / instead of appending | 21:17 |
| clarkb | I quickly get lost in the keystone auth stuff (really really wish the client libs were easier to consume) so I'm not sure where that is happening | 21:17 |
| clarkb | I'll let nova know in case this is something on their end | 21:18 |
| rosmaita | ok, thanks | 21:18 |
| rosmaita | maybe in placement | 21:18 |
| rosmaita | doesn't look like anything merged recently in nova proper | 21:18 |
| *** dviroel is now known as dviroel|out | 21:29 | |
| opendevreview | Ghanshyam proposed openstack/devstack master: WIP: testing sync mode https://review.opendev.org/c/openstack/devstack/+/811245 | 23:51 |
| opendevreview | Ghanshyam proposed openstack/devstack master: Log all the services endpoint at the end of stack.sh https://review.opendev.org/c/openstack/devstack/+/811246 | 23:59 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!