13:30:59 <gmann> #startmeeting qa
13:30:59 <openstack> Meeting started Tue Jan 14 13:30:59 2020 UTC and is due to finish in 60 minutes.  The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:31:00 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:31:02 <openstack> The meeting name has been set to 'qa'
13:31:14 <gmann> who all here today ?
13:31:30 <bigdogstl> Hello Doug is here
13:31:42 <tosky> o/
13:31:42 <kopecmartin> hi o/
13:32:53 <gmann> hello everyone. let's start
13:32:56 <gmann> #link https://wiki.openstack.org/wiki/BugTriage
13:33:05 <gmann> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours
13:33:09 <gmann> this is agenda ^^
13:33:19 <gmann> #topic Announcement and Action Item (Optional)
13:34:02 <gmann> we have two AI from previous meeting.
13:34:07 <gmann> 1. kopecmartin need to check and report the unit test coverage for cleanup CLI.
13:34:16 <gmann> 2. gmann to add the community goal tracking for QA in office hour agenda.
13:34:41 <gmann> i have not done that at least. I will do that today so re-adding it.
13:34:53 <gmann> #action gmann to add the community goal tracking for QA in office hour agenda.
13:35:04 <gmann> kopecmartin: how about you?
13:35:12 <kopecmartin> gmann: i did checked the numbers and tempest/cmd/cleanup.py has unit test coverage == 25% and cleanup_service.py 90%
13:35:41 <gmann> ok.
13:36:24 <kopecmartin> cleanup_service.py coverage can be improved, but I'm afraid that unit tests for cleanup.py would be too complicated , the code contains basically for loops and overall controll of the whole cleanup process, it would be hard to test it by unit tests i think
13:36:46 <kopecmartin> cleanup.py should be tested by functional tests in my opinion
13:37:07 <kopecmartin> we have a zuul role for that up fo review, so we can start by that :)
13:37:20 <kopecmartin> that's all to my action item
13:38:07 <gmann> kopecmartin: i agree but we can at least add unit test for its arg etc so that changing them or their behavior can be caught
13:38:25 <gmann> i remember we broke those with very silly changes.
13:38:32 <kopecmartin> gmann: ok, that sounds reasonable
13:39:37 <gmann> kopecmartin: we have BP to track the CLI unit tests.
13:39:39 <gmann> #link https://blueprints.launchpad.net/tempest/+spec/tempest-cli-unit-test-coverage
13:40:15 <gmann> I will assign this to you now and also feel free to add notes in that.
13:40:38 <gmann> done.
13:40:56 <gmann> #topic Ussuri Priority Items progress
13:41:11 <gmann> #link https://etherpad.openstack.org/p/qa-ussuri-priority
13:41:58 <gmann> first two items from masayukig who is not here.
13:42:04 <gmann> RBAC testing strategy (reader/member/admin X project/domain/system)
13:42:20 <gmann> i have not reviewed this yet. This is priority item for us.
13:42:44 <gmann> I should be spending some time in this week for this.
13:42:45 <bigdogstl> we are trying to get patrole running but it has resource leaks
13:42:57 <bigdogstl> patrole reviews https://review.opendev.org/#/q/owner:rick.bartra%2540att.com+status:open+project:openstack/patrole
13:43:03 <bigdogstl> to fix some of them
13:43:37 <gmann> bigdogstl: yeah, i have reviewed most of them i think
13:44:00 <gmann> bigdogstl: gate should be fin now as Rick fix for admin job is merged.
13:44:18 <gmann> we will come into that during patrole section
13:44:20 <bigdogstl> yes thank you for helping him out on that
13:44:26 <bigdogstl> ok
13:44:30 <gmann> next item is- py2.7 drop plan for QA
13:44:50 <gmann> this is in progress for devstack and not started for Tempest.
13:45:02 <gmann> Tempest will be during start of Feb as per TripleO request.
13:45:07 <gmann> CI
13:45:43 <gmann> skipping few as no progress on them.
13:45:45 <gmann> Improving Tempest cleanup
13:45:56 <gmann> kopecmartin: is working on that.
13:46:15 <gmann> - Bug triage status report before office hour
13:46:30 <gmann> kopecmartin: did I send you the etherpad links for this ?
13:46:35 <gmann> i think no..
13:46:49 <kopecmartin> no, at least i haven't receive them
13:47:09 <gmann> ok.
13:47:17 <gmann> I will do right after meeting
13:47:44 <gmann> #topic OpenStack Events Updates and Planning
13:48:17 <gmann> next event is Vancouver summit/PTG. I have not started the planning.  I will do during April end or May.
13:48:40 <gmann> #topic Sub Teams highlights (Sub Teams means individual projects under QA program)
13:48:44 <gmann> Tempest
13:49:07 <gmann> #link https://review.opendev.org/#/q/project:openstack/tempest+status:open
13:49:12 <gmann> these are open review.
13:50:06 <tosky> is this the moment when we try to highlight some reviews, or is it for the open discussion part, as the agenda seems to hint?
13:50:31 <gmann> efried fix is merged to explicit telling from tests to create the isolated network for create server request - https://review.opendev.org/#/c/701792/
13:50:42 <gmann> tosky: you can if it is Tempest
13:51:01 <tosky> just a reminder about a new scenario (mostly volume) test: https://review.opendev.org/#/c/695967/
13:51:24 <kopecmartin> i'd like to bring this up https://review.opendev.org/#/c/692375/, it's very important
13:51:33 <bigdogstl> Skip adding host to aggregate with az if host already in other zone https://review.opendev.org/#/c/673342/
13:52:24 <gmann> tosky: ok, i think cinder tempest plugin test patch is abandon now so i can review on Tempest one. I was waiting as same tests was added in both place
13:52:43 <tosky> gmann: yes, the other patch was abandoned some time ago
13:52:51 <gmann> kopecmartin: sorry about that. I will check that today for sure.
13:53:05 <kopecmartin> gmann: thanks
13:54:10 <gmann> bigdogstl: is host with AZ detected automatically or with config ?
13:54:25 <gmann> i have not looked into review comments in detail. seems masayukig left some comment
13:55:03 <bigdogstl> I will have Rick address them
13:55:55 <gmann> ok. adding that in review list
13:56:09 <gmann> anything else on Tempest ?
13:56:38 <bigdogstl> Not at this time most of our effort is in patrole
13:56:57 <gmann> Patrole.
13:57:01 <gmann> bigdogstl: go ahead.
13:57:08 <bigdogstl> patrole reviews https://review.opendev.org/#/q/owner:rick.bartra%2540att.com+status:open+project:openstack/patrole
13:57:16 <gmann> agenda is old for Patrole. I will update for next meeting
13:57:56 <gmann> bigdogstl: i tried to review almost all open and zuul happy patches.
13:58:08 <gmann> one i need to investigate more is -  #link https://review.opendev.org/#/c/676979/
13:58:52 <gmann> i suspect keeping tenant_id in cred during cleanup can have risk of stale tenant_id
13:58:55 <bigdogstl> Ok I will have Rick review the comments and keep pushing them
13:59:53 <gmann> he added the investigation on bug and review and the workaround also.
14:00:49 <gmann> which I need to check in details and find some mechanism if we can restore the user_id and tenant_id during self._initial.
14:01:20 <gmann> not cleaning them during clean_cred() is something does not sounds good.
14:01:39 <gmann> anyways, i will add comment on bug and review.
14:02:30 <bigdogstl> ok
14:02:31 <gmann> other item in patrole is. I missed to add the Train job for Patrole.
14:02:34 <gmann> #link https://review.opendev.org/#/c/701271/
14:02:49 <lyarwood> https://review.opendev.org/#/q/topic:negative_multiattach_swap_volume_tests is ready for review if anyone has time, not sure if this is the correct place in the meeting to bring this up.
14:02:52 <gmann> this is somehow failing which means some policies might have changed.
14:03:17 <rick_bartra> if you get a chance, can you please take a look at https://review.opendev.org/#/c/692233/ again. It is for the volumes and volume snapshots, I replied back to @gmann comments
14:03:37 <gmann> lyarwood: thanks. this is correct time and place to bring.  I am adding those in list.
14:04:03 <rick_bartra> correction: volumes and volume snapshots resource leaks*
14:04:46 <lyarwood> gmann: wonderful thanks
14:04:56 <gmann> rick_bartra: create_test_server does not delete the volume.
14:05:18 <lyarwood> gmann: can I also bring up https://review.opendev.org/#/c/702089/ for devstack-gate, final bit of n-cauth clean up for stable/rocky
14:06:40 <rick_bartra> @gmann if it doesn't I am not exactly sure where the volume is deleted. I will need to dig further, but do know the volume is deleted when the volume snapshot is cleaned up in the test
14:07:12 <gmann> lyarwood: +2. cellv1 is legacy job which i did not check. all other job since rocky use devstack service things
14:08:18 <gmann> rick_bartra: ok. because once you fix is merged I am thinking to move the volume cleanup part in create_test_server becasue we can easily detect the created volume there for volume backed server request
14:08:51 <gmann> and we can do that during class level so we do not need to worry about if tests need those volume and should be cleanup after detach etc.
14:09:09 <rick_bartra> gmann, sounds good. I will find where it is happening and reply back to the change review
14:09:53 <gmann> rick_bartra: thanks. on L308 comment. did you find that after running single tests or set of tempest set.
14:11:04 <rick_bartra> gmann, I found it with both. We found resource leaks running downstream, then when I ran this test by itself the same leaks happened
14:12:16 <gmann> ok. kopecmartin does your patch of adding cleanup CLI call in gate job log the leaked resource somewhere ?
14:12:57 <gmann> rick_bartra: kopecmartin is adding the cleanup CLI in upstream job which should be able to detect all these resource leak.
14:13:19 <gmann> #link https://review.opendev.org/#/c/698610/
14:13:33 <kopecmartin> gmann: yes, it should be printed out to the console
14:14:00 <gmann> kopecmartin: ok and what about init saved file ?
14:14:04 <rick_bartra> gmann, thanks will check it out
14:14:09 <bigdogstl> Should leaks cause the gate to fail?
14:14:24 <bigdogstl> or at least a gaol to head that way?
14:14:42 <kopecmartin> init_saved state is created before tempest execution https://review.opendev.org/#/c/698610/8/playbooks/devstack-tempest.yaml and after tempest execution tempest cleanup --dry-run is called
14:14:50 <gmann> bigdogstl: not initially till we fix all and it pass everything.  but later we should fail and detect the leakage during addition of tests itself
14:15:00 <kopecmartin> but the role allows to change dry run to actual run, it depends on the needs
14:15:04 <gmann> bigdogstl: yeah. that is overall goal.
14:15:05 <bigdogstl> sounds great thanks
14:16:18 <gmann> kopecmartin: yeah, can we log init_saved state also as part of that role. it should be few resource in that which devstack create.
14:16:56 <gmann> kopecmartin: let me check the review in detail and then i can comment exact logging i am thinking.
14:17:08 <gmann> rick_bartra: for that volume things. let me check locally also.
14:17:17 <gmann> thanks for fixing all those rick_bartra
14:17:27 <tosky> if we moved away from Patrole, there is a Python 3 fix for subunit2html: https://review.opendev.org/#/c/700778/ - which goes back to the discussion about moving subunit2html somewhere else
14:17:30 <rick_bartra> gmann, thank you for all the reviews
14:17:38 <gmann> moving next...
14:18:31 <gmann> tosky: somewhere else ?
14:18:57 <gmann> i did not see comment about moving
14:19:15 <tosky> I mean, from the topics; first Tempest, then Patrole, then...
14:19:16 <tosky> :)
14:19:38 <gmann> ohk :)
14:20:10 <gmann> there was discussion of moving subunit2html somewhere else during PTG - L20 - https://etherpad.openstack.org/p/qa-ussuri-priority
14:20:20 <gmann> the sentence matches that topic :)
14:20:55 <gmann> grenade
14:21:11 <gmann> this to check for grenade zuulv3 native base jobs
14:21:13 <tosky> erm, I'm guilty; I planned to look and the failures yesterday
14:21:34 <tosky> but then the world feel apart and I moved to other things; I still plan to check and fix the jobs this week
14:21:41 <gmann> tosky: you mentioned plugin things is failing for that job ?
14:21:43 <tosky> so status as last week
14:21:49 <gmann> phk
14:21:51 <gmann> phk
14:21:52 <gmann> ohk
14:22:15 <gmann> tosky: thanks again for working on those.
14:22:16 <tosky> yes; both testing jobs which involves plugin are failing, so I guess there is a more general issue somewhere; something that happened in the last months
14:22:38 <tosky> I will keep everyone updated when I manage to make them working again
14:22:56 <gmann> cool.
14:23:10 <gmann> #topic Bug Triage
14:23:36 <gmann> i have not prepared the etherpad for kopecmartin and sonia who volunteer for bug triage.
14:24:06 <gmann> but we have 49 New bug in Tempest. #link https://bugs.launchpad.net/tempest/+bugs?search=Search&field.status=New
14:24:53 <gmann> which is too much i think and we need to triage them on priority. I will try to keep 30 min for bug triage from next office hour.
14:25:32 <gmann> #topic Critical Reviews
14:25:45 <gmann> any other critical review than we already discussed ?
14:26:21 <gmann> #topic Open Discussion
14:26:27 <gmann> 4 min left if any other topic
14:27:53 <gmann> if nothing. let's close it.
14:27:59 <gmann> thanks everyone for joining.
14:28:05 <gmann> #endmeeting