Tuesday, 2021-10-12

niteshHi All, i've few queries on Barbican-tempest-plugin, can some one help me on this05:29
opendevreviewSoniya Murlidhar Vyas proposed openstack/tempest master: Need to add netstat logging in the background  https://review.opendev.org/c/openstack/tempest/+/81359007:35
opendevreviewSoniya Murlidhar Vyas proposed openstack/tempest master: This patch creates the network v2.0 floating-ips-port-forwarding client.  https://review.opendev.org/c/openstack/tempest/+/81001507:48
fricklernitesh: just leave your questions here and wait for someone to reply08:30
niteshI don't have an L3 agent in my setup and when i try to run Barbican-tempest-plugin all the testcases are failing. When i see the tempest code at https://github.com/openstack/tempest/blob/25.0.0/tempest/lib/common/dynamic_creds.py#L343 , i see the router creation is happening. Is it Mandatory to have L3 agent? My tempest.conf is https://paste.openstack.org/show/809908/  let me know if i miss any configuration09:10
fricklerkopecmartin: gmann: ^^ I seem to remember that there was some discussion about this use case some time ago already, do you happen to know more details?09:25
*** pojadhav is now known as pojadhav|brb11:27
*** pojadhav|brb is now known as pojadhav12:34
*** thelounge94 is now known as redrobot13:02
*** redrobot is now known as thelounge9413:04
*** thelounge94 is now known as redrobot13:04
kopecmartinnitesh: what traceback does it fail with? 13:23
kopecmartindid it fail during router creation?13:23
kopecmartinare you asking whether the router creation is required?13:23
fricklerkopecmartin: aiui, yes, there are deployments that are not able to create routers, they only use provider networks13:29
niteshkopemartin, the traceback error is https://paste.openstack.org/show/809922/13:35
niteshi've some 13 testcases, For every test case it is going to https://github.com/openstack/tempest/blob/25.0.0/tempest/lib/common/dynamic_creds.py#L343 and failing as L3 is not present in my setup13:36
fricklernitesh: ah, that code references the option that you'll likely need: auth.create_isolated_networks=false13:40
kopecmartinyeah: https://opendev.org/openstack/tempest/src/branch/master/tempest/config.py#L7113:41
fricklersee also the last paragraph in https://docs.openstack.org/tempest/latest/configuration.html#with-dynamic-credentials13:42
fricklerbtw., I think one should remove the mention of nova-network in that section?13:42
gmannfrickler: nitesh kopecmartin as of now tempest create all network resource including router if neutron service is enabled or create_isolated_networks is enabled as frickler mentioned13:45
gmannbut create_isolated_networks  false means you will rely on default network in your env.13:46
niteshThank you guys. So conclusion is that if i " auth.create_isolated_networks=false" in my tempest.conf. It will solve my issue right ?13:52
gmannnitesh: if your env has single network otherwise server creation can fail with 'multiple network' issue.13:54
gmannyou can try13:54
gmannwhat this option does is 1. stop tempest to create any network resource 2. rely on existing network in testing env to be used for server creation 13:55
kopecmartin#startmeeting qa14:00
opendevmeetMeeting started Tue Oct 12 14:00:11 2021 UTC and is due to finish in 60 minutes.  The chair is kopecmartin. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
opendevmeetThe meeting name has been set to 'qa'14:00
kopecmartin#link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours14:00
kopecmartinthe agenda ^^^14:00
soniya29hello14:00
kopecmartinhi soniya2914:01
kopecmartinlet's start14:02
kopecmartin#topic Announcement and Action Item (Optional)14:02
yoctozeptoo/14:02
kopecmartinPTG is gonna be the next week, but we'll talk about that in a minute14:02
kopecmartinhi yoctozepto14:02
kopecmartin#topic Xena Priority Items progress14:02
kopecmartin#link https://etherpad.opendev.org/p/qa-xena-priority14:02
kopecmartinany progress on this front?14:03
kopecmartinnothing from my side unfortunately14:03
gmanno/14:03
gmannnothing from me14:03
kopecmartin#topic OpenStack Events Updates and Planning14:04
kopecmartin#link https://etherpad.opendev.org/p/qa-yoga-ptg14:04
kopecmartinPTG is next week14:04
kopecmartinif you haven't proposed any topic yet, feel free to do so, rather sooner than later14:05
kopecmartina reminder:14:05
kopecmartinwe have booked the following slots: 13-15 UTC October 18th and 15-17 UTC October 19th14:05
soniya29kopecmartin, sure14:05
kopecmartinspeaking about the PTG, the QA office hour next week is cancelled due to PTG 14:06
gmann+114:06
kopecmartinwe can talk in qa sessions :) 14:06
kopecmartinmoving on14:06
kopecmartin#topic Gate Status Checks14:06
kopecmartin#link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking)14:07
kopecmartinany issues to bring up?14:07
kopecmartinI don't see any urgent patches14:07
gmannI think it seems green. 14:07
kopecmartinthat's great14:07
kopecmartin#topic Periodic jobs Status Checks14:07
kopecmartin#link https://zuul.openstack.org/builds?job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&job_name=tempest-full-train-py3&pipeline=periodic-stable14:07
kopecmartin#link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic14:07
kopecmartinall green too it seems (one failure, although that seems to be random)14:08
kopecmartin#topic Sub Teams highlights14:08
kopecmartinany updates, highlights we need to discuss today ?14:08
kopecmartinChanges with Review-Priority == +114:09
kopecmartin#link https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking)14:09
kopecmartinthere are none14:09
kopecmartin#topic Open Discussion14:11
gmannone issue nitesh brought before meeting about env not having L3 network and how to run tests as router cannot be created without L314:11
gmannanswer is to stop the network resource creation and rely on env existing network14:11
gmannslaweq ^^ is this right approach from neutron perspective right?14:12
kopecmartinthis is the option in tempest btw: https://opendev.org/openstack/tempest/src/commit/a7bcabc8976e6e646d5e4379e3289b43586261c1/tempest/config.py#L7114:12
slaweqgmann: if You have provider network configured, You can plug vms directly to that network14:13
slaweqand then they should be accessible using fixed ips directly14:13
gmannyeah, that is Tempest expecation and tempest do not create any network resource14:13
fricklerwow, you are fast. I wanted to bring up the issue of pip installs failing with latest setuptools14:13
fricklerthis is hidden in CI by our use of pre-built, cached wheels14:13
fricklerbut an increasing number of people show up failing to install devstack locally because of that14:14
gmannslaweq: thanks for confirmation 14:14
slaweqyw14:14
fricklerit also affects running unit tests14:14
gmannfrickler: we are facing that on stable right? 14:14
fricklerno, on master too14:14
frickleror especially there14:15
fricklerI made a requirements job withour our ci-built wheels14:15
kopecmartinis it this bug?14:15
kopecmartin#link https://bugs.launchpad.net/devstack/+bug/194239614:15
fricklerhttps://review.opendev.org/c/openstack/requirements/+/81329214:15
frickleryes, that bug14:16
fricklerehm, no14:16
fricklerhttps://bugs.launchpad.net/cinder/+bug/194634014:16
fricklerit is mostly an issue with oslo.vmware and designate14:17
fricklerbut also some further uninstallable libs I listed in https://review.opendev.org/c/openstack/requirements/+/81330214:18
fricklerfor stable branches, capping setuptools and possibly pip might be a solution14:18
gmannyeah,14:18
fricklerfor master, we need to replace the outdated libs14:18
fricklerfor suds-jurko, there is a suds-community fork, which will hopefully release 1.0.0 with a fix soon14:19
fricklerfor the other libs, I don't know a solution yet14:19
fricklerI'm also thinking we should do a periodic devstack job than runs without prebuilt wheels, so that we can detect this situation14:20
gmann+1 on job14:20
kopecmartinthat is a good idea14:20
kopecmartinmaybe we could detect these kind of things sooner14:21
gmannbut if it does and on replacement (like current siutaiton) what we do? ask projects/requirement team to remove the deps? 14:21
gmann*no replacement14:21
kopecmartinthere isn't other option than waiting until the libs will make their code compatible with newer setuptools/pip, is there?14:22
fricklerthose libs mostly seem to be abandoned14:22
kopecmartinoh14:22
fricklerlike suds-jurko hasn't been updated in some years14:22
gmannyeah14:22
fricklerwhich makes them good candidates for replacement anyway, yes14:22
gmannseems 8 project using it https://codesearch.opendev.org/?q=suds-jurko&i=nope&literal=nope&files=&excludeFiles=&repos=14:24
gmannI think adding such job and then notify requirement/projects  team to start thinking about replacement/remove-deps can be good way forward14:25
kopecmartinyup, i agree 14:25
gmannand reply on prebuilt wheels until then but it does not solve the local devstack issue14:25
gmannrely 14:26
gmannin that case we can add var to in the setuptool/pip for local env. frickler ?14:26
gmann* to pin the setuptool/pip14:26
fricklerhmm, I think the mirrors are accessible from the outside, but not sure if we want to point people to that14:27
fricklerin the current case, just removing oslo.vmware from nova/cinder/designate reqs is enough14:27
fricklerbecause most deployments don't actually use it14:27
gmannbut we need to disable corresponding tests too right?14:28
gmannand it might be hard to find all unit/functional tests corresponding to vmware driver in nova until we see all failing one 14:29
fricklerI didn't check that, just verified that devstack installation finishes14:29
fricklerI can try to build that job later and maybe also a corresponding tox job14:30
fricklerthen we can see in nova what actually is failing14:30
gmannfrickler: if we remove it from requirement repo then we have nova.cinder unit test jobs14:30
gmannyah14:30
gmannfunctional too14:30
kopecmartinanything else to add?14:33
kopecmartinanything else for the open discussion? 14:33
gmannnothing from me 14:34
kopecmartin#topic Bug Triage14:35
kopecmartin#link https://etherpad.opendev.org/p/qa-bug-triage-xena14:35
kopecmartini recorded the numbers as always ^^14:35
kopecmartini have 2 bugs to bring up14:35
kopecmartin#link https://bugs.launchpad.net/tempest/+bug/194632114:35
kopecmartin 'jsonschema' has no attribute 'compat' 14:35
kopecmartinwe have a fix for master14:35
kopecmartin#link https://review.opendev.org/c/openstack/tempest/+/81280414:35
kopecmartinwhat will we do with older releases? we can't modify tempest there :/ 14:36
gmannthat is fixed by this #link https://review.opendev.org/c/openstack/devstack/+/81209214:36
gmannwe need to use the latest compatible version on stable branches where we use old tempest14:36
kopecmartinthat makes sense14:37
kopecmartinok then, gmann please review this when you have a moment14:38
gmannwe have a set of compatible tempest version for any stable branch so using any version from that range works14:38
kopecmartin#link https://review.opendev.org/c/openstack/tempest/+/81280414:38
gmannah sure, i thought I did :) will re-review14:38
kopecmartinthanks 14:38
kopecmartinanother bug is this one:14:39
kopecmartin#link https://bugs.launchpad.net/tempest/+bug/194549514:39
kopecmartintest_rebuild_server fails with mismatch - fixed and floating ip type after rebuild14:39
gmanndone14:39
kopecmartinthere is a race condition in the test 14:39
kopecmartini'm not sure how to approach that14:39
kopecmartinwe don't have any waiter for a floating ip to wait until it's attached to a vm14:39
kopecmartinas mentioned in the comments , there is a possibility that the ip doesn't get assigned between lines 193 and 214 and that will result in the mismatch14:43
kopecmartin#link https://opendev.org/openstack/tempest/src/commit/669900f622ce46c62260910d2cf9fa1d32216191/tempest/api/compute/servers/test_server_actions.py#L19314:43
kopecmartin#link https://opendev.org/openstack/tempest/src/commit/669900f622ce46c62260910d2cf9fa1d32216191/tempest/api/compute/servers/test_server_actions.py#L21414:43
gmannmay be we can use os-interface nova APi to poll that?14:44
gmann#link https://docs.openstack.org/api-ref/compute/?expanded=show-port-interface-details-detail14:44
kopecmartinso basically to create a waiter - whenever the server has a floating ip attached before the rebuild, wait until the floating ip is attached after the rebuild and then continue14:46
gmannyeah, if this is race on FIP attachment. but i have not checked the log14:47
kopecmartinthat's the only way i can image we could get to the mismatch error 14:48
kopecmartinok ,i'll try to create that waiter and let's see14:49
kopecmartinthanks for the consult 14:49
kopecmartinthis all from my side14:49
kopecmartinif there isn't anything else, i think we can close today's office hour14:50
gmannnothing from me, we can close14:50
kopecmartinthank you all and see you at the PTG next week o/14:50
gmannthanks kopecmartin  14:50
kopecmartin#endmeeting14:50
opendevmeetMeeting ended Tue Oct 12 14:50:52 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:50
opendevmeetMinutes:        https://meetings.opendev.org/meetings/qa/2021/qa.2021-10-12-14.00.html14:50
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/qa/2021/qa.2021-10-12-14.00.txt14:50
opendevmeetLog:            https://meetings.opendev.org/meetings/qa/2021/qa.2021-10-12-14.00.log.html14:50
*** poojajadhav is now known as pojadhav15:31
opendevreviewMerged openstack/whitebox-tempest-plugin master: test_volume_encryption: Correctly lookup the rbd user from XML  https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/81305615:49
opendevreviewSophie Huang proposed openstack/tempest master: Set compute API microversion in volume API testing  https://review.opendev.org/c/openstack/tempest/+/81367617:15
opendevreviewMerged openstack/devstack master: Fix updating setuptools in Centos  https://review.opendev.org/c/openstack/devstack/+/79706920:43

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!