00:00:35 #startmeeting qa 00:00:36 Meeting started Thu Oct 24 00:00:35 2019 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. 00:00:37 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 00:00:39 The meeting name has been set to 'qa' 00:00:41 who all here today 00:01:24 ping masayukig 00:03:49 hi 00:05:40 let's start 00:05:54 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours 00:06:00 today agenda ^ 00:06:38 #topic Train Priority Items progress 00:06:52 #link https://etherpad.openstack.org/p/qa-train-priority 00:07:18 How to make tempest-full stable - gmann 00:07:26 this is completed except glance 00:07:51 cool 00:08:01 i already pinged glacne ptl and replied on patch also but no review - https://review.opendev.org/#/c/671548/ 00:08:28 i am marking this completed and leave on glance if they still want that patch 00:08:49 OpenStack-health improvement - masayukig 00:08:54 masayukig: same status ? 00:09:07 yeah, the same 00:09:14 ok 00:09:15 Planning for Patrole Stable release - gmann 00:09:31 we released patrole for train but not as satble 00:09:33 stable 00:10:23 there are less active contributors in patrole in this release. I will plan something during ussuri and see if anything left to release as stable 00:10:36 Keystone system-scope testing - gmann 00:10:47 this is PTG topic also which will discuss in shanghai 00:10:57 Document the QA process or todo things for releases, stable branch cut etc - gmann 00:11:36 no update on this. I did not get time to do so but good to do during shanghai PTG 00:11:44 new whitebox plugin for tempest - artom 00:12:03 Oh snap, I'm actually here?! 00:12:09 (It's 20:15 here) 00:12:11 #link https://review.opendev.org/#/c/656890/ 00:12:25 artom: perfect. so we need to review this ^^ spec 00:12:43 Thanks for bringing it up :) 00:12:46 #action gmann masayukig to review the spec 656890 00:13:01 sure 00:13:06 artom: thanks for pushing the spec, we will make sure get that review soon 00:13:17 Ack, it's appreciated :) 00:13:18 Tempest Plugin sanity check among cross tempest plugins - masayukig 00:13:24 masayukig: this is done right ? 00:13:40 gmann: yeah, I think it's completed 00:14:16 ok 00:14:29 doc patch is there which we can review in regular review - https://review.opendev.org/#/c/674121/ 00:14:34 marked that as completed 00:14:47 fix scenario tests not to use admin cred for non-admin API call - Shi Yan 00:15:02 I do not remember any update in this. masayukig did you see any review ? 00:15:44 gmann: well, I probably saw a patch but not sure 00:16:07 ok. i will keep this open. anyways this is something can be identified case by case 00:16:15 masayukig: if you find the patch please add that in etherpad 00:16:22 scenario manager stable interface - 00:16:23 gmann: for sure 00:16:30 masayukig: thanks 00:16:37 no progress or volunteer for this. 00:17:01 This is good item to publish to shanghai event where there might be few new contributors 00:17:10 Data plane tester - 00:17:24 doug has not submitted the spec yet 00:18:16 This is last tracking for train topics, in PTG we will start the ussuri items and pending things here will go on ussuri cycle 00:18:21 #topic OpenStack Events Updates and Planning 00:18:32 Next: Shanghai Summit & PTG 00:18:43 Summit & PTG Planning: 00:18:58 Forum: #link https://etherpad.openstack.org/p/PVG-forum-qa-ops-user-feedback 00:19:12 this is for QA forum sessions 00:19:17 PTG: #link https://etherpad.openstack.org/p/shanghai-ptg-qa 00:19:21 this is ptg etherpad 00:19:37 add the topic you want to discuss 00:19:59 sure 00:20:02 i have sent both of those on ML 00:20:16 ++ 00:20:16 i talked to zhufl over email and she would not be able to join PTG. 00:20:32 i was hoping we could have meet her f2f this time 00:20:52 oh.. 00:21:10 we have Wednesday full day for QA and we will plan onboarding sessions in that 00:21:35 Sub Teams highlights (Sub Teams means individual projects under QA program) 00:21:39 Tempest 00:22:27 I have started the ML thread for dropping the py2.7 from Tempest 00:22:29 #link http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010317.html 00:22:57 i have not read the reply yet. I will do and respond tomorrow. but masayukig feel free to add if anything missing 00:23:12 thanks, yeah, I'll check it 00:23:20 masayukig: thanks 00:23:36 that is all from today from my side. 00:23:42 masayukig: anything you have ? 00:24:17 nope so far 00:24:25 also I would like to cancel the next office hour because of PTG after that 00:24:40 +1 yeah 00:24:56 can someone remind me, where can I see the per test timeout in a CI job? is it a tempest config setting or a zuul job def setting? 00:24:58 and sometime mid of next week, we can plan the final schedule of PTG on qa channel itself as it is mostly two of us 00:25:34 gmann: ok 00:25:57 melwitt: that is in tox. 00:25:59 https://github.com/openstack/tempest/blob/master/tox.ini#L61 00:26:21 that is for tempest-all job which using 'all' tox env 00:26:24 gmann: thanks! the only place I didn't look, feels like haha 00:27:01 melwitt: and for other job/tox env it is default from here - https://github.com/openstack/tempest/blob/master/tox.ini#L21 00:27:34 I'm trying to figure out which gate bug a patch hit and I see a test ran for 260 seconds and I was trying to verify whether test timeout was related 00:27:36 thank you 00:28:31 masayukig: does 30th work for you to plan the final schedule of PTG ? or 31st ? 00:28:42 sorry 31st is my flight 00:28:48 29th or 30th work for me 00:29:32 gmann: both work for me 00:30:22 masayukig: ok let's do it on 30th. ping me once you start your day. I will be online by this time usually 00:30:58 gmann: ok, do you mean 30th in JST? 00:31:16 masayukig: yeah 00:31:54 around 0 UTC (your 9 AM or early ) 00:32:10 ok, np 00:32:13 around 0 UTC (your 9 AM or early ) 00:32:22 almost same slot of this office hour 00:32:25 ok 00:32:33 ping me if i am afk 00:32:48 yeah, that's convenient :) 00:32:52 ok 00:33:07 anything else to discuss 00:33:53 nope from my side 00:34:06 ok let's close 00:34:08 thanks 00:34:12 #eendmeeting 00:34:14 thanks! 00:35:27 melwitt: and you can also define the timeout on job side also with this var 'tempest_test_timeout' - https://opendev.org/openstack/tempest/src/branch/master/roles/run-tempest/tasks/main.yaml#L45 00:35:53 gmann: ah thanks 00:36:16 i mean if it has to be other than default. as of now only ironic job use this and all other gate jobs are with default timeout what we have in tox 00:37:30 not seeing anything that would add up to 260 seconds xD I dunno.... gonna have to ask mriedem_afk again. I feel like I vaguely remember 260 seconds being something 00:38:05 which job ? 00:38:33 https://zuul.opendev.org/t/openstack/build/afbd0c78abe44554a9ee9dc6f2e4515b/log/job-output.txt#34008 00:39:19 I feel like I remember a long time ago mriedem_afk had pointed out [260.434169s] ... FAILED the 260 seconds time elapsed 00:47:29 melwitt: that job using 'all' tox env where timeout is 1200 by default - https://opendev.org/openstack/devstack-plugin-ceph/src/branch/master/.zuul.yaml#L25 00:47:52 gmann: thank you! 00:48:32 guess it's just ssh timeout (like it says). I don't know what I was thinking 00:49:44 doesn't seem to match any of the existing ssh bugs on http://status.openstack.org/elastic-recheck/ tho 00:50:13 yeah 00:53:00 that is not good for my ocd 01:12:11 wanglbj proposed openstack/tempest master: Modify the error fake date https://review.opendev.org/689955 01:28:28 Merged openstack/devstack master: Fix benign epmd@0.0.0.0.socket failure https://review.opendev.org/673010 01:48:43 linanbj proposed openstack/tempest master: Modify the response example in test_routers_client https://review.opendev.org/690779 01:54:56 linanbj proposed openstack/tempest master: Modify the response example in test_routers_client https://review.opendev.org/690779 02:18:37 linanbj proposed openstack/tempest master: Modify the response example in test_routers_client https://review.opendev.org/690779 02:35:22 linanbj proposed openstack/tempest master: Modify the response example in test_subnetpools_client https://review.opendev.org/690785 03:26:11 Merged openstack/tempest master: tempest-plugin-sanity: respect upper-constraints https://review.opendev.org/689950 05:50:12 Michal Skalski proposed openstack/tempest master: Fallback to tenant_id if project_id not found https://review.opendev.org/690809 06:50:43 pengyuesheng proposed openstack/patrole master: Switch to Ussuri jobs https://review.opendev.org/690862 11:58:36 Lajos Katona proposed openstack/tempest master: Check port allocation for VM with resource request https://review.opendev.org/690934 14:32:05 Phil Sphicas proposed openstack/tempest master: Update test_port_list_filter_by_ip pass conditions https://review.opendev.org/676852 14:47:40 Matt Riedemann proposed openstack/devstack master: Pad an extra 1GiB onto tempest flavor root disk https://review.opendev.org/690991 21:46:30 Matt Riedemann proposed openstack/devstack master: Pad an extra 1GiB onto tempest flavor root disk https://review.opendev.org/690991 23:37:59 hello, asking for review on this devstack change to fix vnc libvirt/qemu configuration to work with multinode jobs https://review.opendev.org/690469 23:56:50 melwitt: did you see mriedem's comment? the job will ned updating too (but +2 on the devstack fix) 23:57:22 clarkb: yes, he fixed the job (proposed in the nova patch) and all is good to go 23:57:27 thanks! 02:13:09 Xuan Yandong proposed openstack/bashate master: Switch to Ussuri jobs https://review.opendev.org/691144 02:16:28 melwitt: +A 02:16:54 thanks gmann 02:22:27 linanbj proposed openstack/tempest master: Modify the response example in test_routers_client https://review.opendev.org/690779 02:27:15 linanbj proposed openstack/tempest master: Modify the response example in test_subnetpools_client https://review.opendev.org/690785 02:27:37 linanbj proposed openstack/tempest master: Modify the response example in test_routers_client https://review.opendev.org/690779 03:29:41 zhangboye proposed openstack/os-testr master: Switch to Ussuri jobs https://review.opendev.org/691176 04:16:11 hi 04:16:29 is there any impact on nova testcases in py37? 04:16:58 in nova-powervm all the test in py36 passes where as in py37 it times out 04:18:39 out of 100+ test not running i could identify 3 tests [ 04:18:40 TestPowerVMDriver,TestISCSIAdapter,TestVSCSIAdapter] are failing and it times out during setup itself 04:18:44 can someone help me 05:59:09 wanglbj proposed openstack/tempest master: Modify the error fake date https://review.opendev.org/689955 06:18:24 zhangboye proposed openstack/openstack-health master: Switch to Ussuri jobs https://review.opendev.org/691201 07:16:10 pengyuesheng proposed openstack/tempest master: Switch to Ussuri jobs https://review.opendev.org/691238 07:16:21 pengyuesheng proposed openstack/tempest-stress master: Switch to Ussuri jobs https://review.opendev.org/691239 12:10:56 Merged openstack/devstack master: Remove n-novnc service requirement for TLS configuration https://review.opendev.org/690469 13:46:21 Balazs Gibizer proposed openstack/tempest master: Check port allocation for VM with resource request https://review.opendev.org/690934 13:52:18 Matt Riedemann proposed openstack/devstack stable/train: Remove n-novnc service requirement for TLS configuration https://review.opendev.org/691314 15:01:55 Matt Riedemann proposed opendev/elastic-recheck master: Add query for ssh fail bug 1849857 https://review.opendev.org/691374 15:01:55 bug 1849857 in OpenStack-Gate "ssh fails to guest in tempest run, "Starting dropbear sshd: WARN: generating key of type ecdsa failed!" in guest console log" [Undecided,New] https://launchpad.net/bugs/1849857 15:24:10 Merged opendev/elastic-recheck master: Add query for ssh fail bug 1849857 https://review.opendev.org/691374 15:24:11 bug 1849857 in OpenStack-Gate "ssh fails to guest in tempest run, "Starting dropbear sshd: WARN: generating key of type ecdsa failed!" in guest console log" [Undecided,Confirmed] https://launchpad.net/bugs/1849857 15:24:12 Merged opendev/elastic-recheck master: Hard-code os-brick into TestQueries.openstack_projects https://review.opendev.org/689039 15:29:37 Merged opendev/elastic-recheck master: Remove H233 from flake8 ignore list https://review.opendev.org/689041 15:29:39 Merged opendev/elastic-recheck master: Fix link formatting in CONTRIBUTING.rst https://review.opendev.org/689043 21:14:52 Jim Rollenhagen proposed openstack/devstack stable/ocata: Ignore TEMPEST_BRANCH in test_refs https://review.opendev.org/691434 22:35:53 Ghanshyam Mann proposed openstack/devstack stable/queens: Add Zuul v3 native unit-tests https://review.opendev.org/691449 22:38:21 Ghanshyam Mann proposed openstack/devstack stable/pike: Add Zuul v3 native unit-tests https://review.opendev.org/691450 01:57:56 Merged openstack/devstack stable/train: Remove n-novnc service requirement for TLS configuration https://review.opendev.org/691314 19:20:47 Rodolfo Alonso Hernandez proposed openstack/tempest master: Add a default network when creating a server https://review.opendev.org/682964 22:39:59 Phil Sphicas proposed openstack/tempest master: Update test_port_list_filter_by_ip pass conditions https://review.opendev.org/676852 01:04:13 Xuan Yandong proposed openstack/bashate master: Stop testing python2.7 https://review.opendev.org/691542 02:07:06 wanglbj proposed openstack/tempest master: Modify the fake data https://review.opendev.org/691547 03:38:33 wanglbj proposed openstack/tempest master: Modify the fake data https://review.opendev.org/691547 04:58:04 zhangboye proposed openstack/openstack-health master: Stop testing python2.7 https://review.opendev.org/691577 05:04:06 I have started devstack, and after some time I tried to reach to Keystone via request to 05:04:06 http://10.0.2.15/identity/v2.0 05:04:06 and I am getting Error 500. 05:04:06 What would you recommend me to do? 05:47:39 gmann: quite a few changes have been pushed to repos to stop py2 testing that are broken, I gave some -1s, see https://review.opendev.org/#/q/owner:xuanyandong%2540inspur.com+status:open - this includes client library changes already in phase 1. There're others that do the same, not sure what can be done here 06:08:31 zhangboye proposed openstack/os-testr master: Stop testing python2.7 https://review.opendev.org/691592 06:25:28 zhangboye proposed openstack/tempest-lib master: Stop testing python2.7 https://review.opendev.org/691602 06:43:17 gmann: also many devstack/tempest job in master are still on py2, should they be switched as well? Or is that part of the get rid of legacy goal? 10:28:50 wanglbj proposed openstack/tempest master: Modify the fake data https://review.opendev.org/691547 13:39:59 Lajos Katona proposed openstack/tempest master: Check port allocation for VM with resource request https://review.opendev.org/690934 17:33:33 Ghanshyam Mann proposed openstack/devstack stable/pike: Ignore TEMPEST_BRANCH in test_refs https://review.opendev.org/691725 17:33:55 Ghanshyam Mann proposed openstack/devstack stable/pike: Add Zuul v3 native unit-tests https://review.opendev.org/691450 17:35:29 AJaeger: yeah i saw those patches, I commented about goal patch and need to go with the plan schedule. from TC side we are reaching each projects about the discussed schedule so that project core team take care of such patches. 17:36:56 AJaeger: RE on devstack and tempest py2.7 jobs. we will drop those in phase-2 as per schedule. 17:37:58 gmann: thanks! 14:02:06 afazekas: please, have a look at this when you have a moment https://review.opendev.org/#/c/615558/ .. tell me what you think then 18:07:19 Matt Riedemann proposed opendev/elastic-recheck master: Add query for test_vcpu_to_pcpu_reshape fail bug 1850514 https://review.opendev.org/691933 18:07:19 bug 1850514 in OpenStack Compute (nova) "ReshapeForPCPUsTest.test_vcpu_to_pcpu_reshape intermittently fails with "Cannot 'migrate' instance while it is in vm_state building"" [Medium,Confirmed] https://launchpad.net/bugs/1850514 20:21:52 Merged opendev/elastic-recheck master: Add query for test_vcpu_to_pcpu_reshape fail bug 1850514 https://review.opendev.org/691933 20:21:52 bug 1850514 in OpenStack Compute (nova) "ReshapeForPCPUsTest.test_vcpu_to_pcpu_reshape intermittently fails with "Cannot 'migrate' instance while it is in vm_state building"" [Medium,In progress] https://launchpad.net/bugs/1850514 - Assigned to Matt Riedemann (mriedem) 22:00:26 Matt Riedemann proposed openstack/devstack master: DNM: default DATABASE_QUERY_LOGGING=True https://review.opendev.org/691995 22:04:21 Matt Riedemann proposed openstack/grenade master: DNM: Enable DATABASE_QUERY_LOGGING https://review.opendev.org/687376 00:02:36 gmann: ping 00:02:52 masayukig: hi 00:03:22 gmann: PTG schedule? 00:03:40 masayukig: ok. give me 10 min. 00:03:52 gmann: sure, np 00:34:59 masayukig: hi sorry for late 00:35:13 gmann: hi 00:35:19 https://etherpad.openstack.org/p/shanghai-ptg-qa 00:35:26 do you have any other topic to add ? 00:35:45 we have projects updates (15 min this time) on wed itself 00:37:33 We also have the QA tooling session on Monday, right? 00:37:47 masayukig: you mean feedback sessions ? 00:38:05 gmann: yeah, https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24401/users-operators-adoption-of-qa-tools-plugins 00:38:07 yes. L5 00:38:39 ah, yes 00:39:34 I don't have any other topic so far 00:39:46 ok 00:41:34 ok I am writing the time on etherpad and then we can discuss 00:41:41 thanks. ah, do we have a Ussuri cycle planning session? 00:44:01 masayukig: oh, i did not add that, can you add 00:44:13 we need to check the backlog also 00:44:18 so 30 min needed 00:46:49 masayukig: added your name as owner 00:46:58 masayukig: done with slots, please check/edit 00:47:33 gmann: thanks, sure 01:01:24 gmann: added the planning & priority session etherpad 01:03:15 masayukig: thanks, 2 free slot if anything come on runtime 01:03:36 gmann: ok, cool 01:03:45 masayukig: also i am preparing the project update slides and will share with you by tomorrow my EOD 01:04:31 gmann: thanks. I think it can be based on the last slides. 01:04:45 yeah 01:09:28 masayukig: anything else on schedule things ? 01:09:45 gmann: nothing from my side 01:10:37 ok. 14:36:34 Matt Riedemann proposed openstack/devstack stable/train: DNM: default DATABASE_QUERY_LOGGING=True https://review.opendev.org/692155 14:37:11 Matt Riedemann proposed openstack/grenade master: DNM: Enable DATABASE_QUERY_LOGGING https://review.opendev.org/687376 14:44:48 this is a pretty simple grenade change to run placement's online_data_migrations as part of the upgrade, been sitting awhile if we have some grenade cores that can take a look https://review.opendev.org/#/c/631614/ 15:17:54 David Sedlák proposed openstack/tempest master: Fixes race condition in test_add_remove_fixed_ip https://review.opendev.org/692168 18:13:40 Matt Riedemann proposed opendev/elastic-recheck master: Add query for nova functional test fail bug 1850682 https://review.opendev.org/692203 18:13:40 bug 1850682 in OpenStack Compute (nova) "functional tests in rocky randomly fail with "Build of instance was re-scheduled: Cannot modify readonly field uuid"" [Undecided,Confirmed] https://launchpad.net/bugs/1850682 21:34:15 Rick Bartra proposed openstack/patrole master: Fix resource leak from 'test_create_image_from_volume_backed_server' https://review.opendev.org/692233 21:36:03 Rick Bartra proposed openstack/patrole master: Fix resource leak from 'test_create_image_from_volume_backed_server' https://review.opendev.org/692233 10:30:53 Rodolfo Alonso Hernandez proposed openstack/tempest master: Add a default network when creating a server https://review.opendev.org/682964 10:32:19 Rodolfo Alonso Hernandez proposed openstack/tempest master: Add a default network when creating a server https://review.opendev.org/682964 11:57:39 Merged openstack/tempest master: Modify the error fake date https://review.opendev.org/689955 13:42:30 Merged opendev/elastic-recheck master: Add query for nova functional test fail bug 1850682 https://review.opendev.org/692203 13:42:30 bug 1850682 in OpenStack Compute (nova) "functional tests in rocky randomly fail with "Build of instance was re-scheduled: Cannot modify readonly field uuid"" [Undecided,Confirmed] https://launchpad.net/bugs/1850682 13:58:58 Martin Kopec proposed openstack/tempest master: Add a new filter to auth.py https://review.opendev.org/692375 14:31:18 clarkb: i've noticed that http://download.cirros-cloud.net is slow as hell, at least right now - has infra ever talked about setting up a mirror for devstack jobs? 14:34:26 though looking here it is fast https://zuul.opendev.org/t/openstack/build/51f73a92576d4873adc4b1c87e2caea8/log/controller/logs/devstacklog.txt.gz#22309 14:34:32 just took me 5.5 minutes though 14:35:06 oh must not have wget'ed it 15:17:14 mriedem: wecache cirros on every disk image 15:17:24 in /opt/cache/files 15:18:42 ah ok cool 16:11:16 Dan Smith proposed openstack/devstack master: DNM: Enable some database connection debug https://review.opendev.org/692402 18:53:02 Brian Haley proposed openstack/devstack master: Fix brctl calls https://review.opendev.org/692429 00:56:16 What are some elegant ways to search through previous, old IRC messages before I joined the channels? Do I have to download the log and grep through it or there're some better way to do this? 00:56:34 Have anyone done this before? 06:14:20 zhufl proposed openstack/tempest master: Deprecate spice_console and rdp_console config items https://review.opendev.org/692491 06:30:54 gugug proposed openstack/openstack-health master: tox: Keeping going with docs https://review.opendev.org/692495 07:01:07 gugug proposed openstack/openstack-health master: tox: Keeping going with docs and remove unnecessary requirements https://review.opendev.org/692495 07:01:55 gugug proposed openstack/openstack-health master: tox: Keeping going with docs and remove unnecessary requirements https://review.opendev.org/692495 07:19:37 Anastasiya Zhyrkevich proposed openstack/tempest master: Replace all-plugin to all in tempest plugin usage docs https://review.opendev.org/692506 07:22:14 Hello, Tempest Team, can you please advice, who can review my small proposal for the documentation? 09:10:07 mriedem: clarkb: re cirros downloads, smoser has an ongoing problem with the dreamhost hosting it seems, see https://bugs.launchpad.net/cirros/+bug/1849327 09:10:07 Launchpad bug 1849327 in CirrOS "Download cirros using http fails" [Undecided,New] 09:10:41 #endmeeting