| *** rcernin has joined #openstack-qa | 00:21 | |
| *** chaconpiza has quit IRC | 00:38 | |
| *** jparoly has quit IRC | 01:00 | |
| gmann | dansmith: its teardown or after that in addcleanup. for in between of test cleanup, we need to do it explicitly via delete APis | 01:41 |
|---|---|---|
| *** hamalq has quit IRC | 01:52 | |
| *** hamalq has joined #openstack-qa | 01:52 | |
| *** priteau has quit IRC | 02:03 | |
| *** rh-jelabarre has quit IRC | 02:21 | |
| *** redrobot has quit IRC | 02:45 | |
| *** hemanth_n has joined #openstack-qa | 02:49 | |
| *** redrobot has joined #openstack-qa | 03:50 | |
| *** gcheresh has joined #openstack-qa | 04:37 | |
| *** vishalmanchanda has joined #openstack-qa | 04:40 | |
| *** ralonsoh has joined #openstack-qa | 05:27 | |
| frickler | this job seems to have encountered some sporadic error in the async code, if anyone want to take a closer look, that'd be nice https://zuul.opendev.org/t/openstack/build/5ef56833a8964ffe8e473abd802d1f05 | 05:31 |
| *** sboyron_ has joined #openstack-qa | 05:57 | |
| *** slaweq has joined #openstack-qa | 06:00 | |
| *** sboyron has joined #openstack-qa | 06:01 | |
| *** sboyron_ has quit IRC | 06:01 | |
| *** ysandeep|away is now known as ysandeep | 06:17 | |
| *** eolivare has joined #openstack-qa | 06:25 | |
| openstackgerrit | MaAoyu proposed openstack/tempest-lib master: setup.cfg: Replace dashes with underscores https://review.opendev.org/c/openstack/tempest-lib/+/788221 | 07:03 |
| *** hamalq has quit IRC | 07:03 | |
| yoctozepto | frickler: looks like keystone is bootstrapped twice, yet logs do not show it, hmm | 07:05 |
| *** jpena|off is now known as jpena | 07:21 | |
| *** rcernin has quit IRC | 07:25 | |
| *** rpittau|afk is now known as rpittau | 07:36 | |
| *** hamalq has joined #openstack-qa | 07:39 | |
| openstackgerrit | Min Li proposed openstack/tempest master: test_server_multinode.TestServerMultinode. test_schedule_to_all_nodes will create a vm on all the compute nodes included in az, but when there is an ironic type hypervisor, the test fails. https://review.opendev.org/c/openstack/tempest/+/788417 | 07:45 |
| *** ysandeep is now known as ysandeep|lunch | 08:02 | |
| *** hamalq has quit IRC | 08:02 | |
| *** lucasagomes has joined #openstack-qa | 08:03 | |
| *** hamalq has joined #openstack-qa | 08:09 | |
| *** rcernin has joined #openstack-qa | 08:21 | |
| *** rcernin has quit IRC | 08:32 | |
| openstackgerrit | Lee Yarwood proposed openstack/devstack master: cinder: Default CINDER_ISCSI_HELPER to tgtadm on Bionic https://review.opendev.org/c/openstack/devstack/+/788429 | 08:32 |
| lyarwood | ^ if any devstack cores are around, this is a follow up to https://review.opendev.org/c/openstack/devstack/+/779624 as it looks like various projects are still using dsvm bionic based jobs that aren't in devstack's gate | 08:33 |
| *** brinzhang0 has joined #openstack-qa | 08:39 | |
| *** brinzhang_ has quit IRC | 08:42 | |
| openstackgerrit | sean mooney proposed openstack/tempest master: Revert "Revert "don't reuse servers in test_server_actions"" https://review.opendev.org/c/openstack/tempest/+/788437 | 09:21 |
| *** priteau has joined #openstack-qa | 09:23 | |
| *** sean-k-mooney has joined #openstack-qa | 09:24 | |
| *** ysandeep|lunch is now known as ysandeep | 09:26 | |
| *** rcernin has joined #openstack-qa | 09:39 | |
| *** hamalq has quit IRC | 09:44 | |
| *** tosky has joined #openstack-qa | 09:45 | |
| frickler | lyarwood: do you happen to have a list of those jobs? I'd like to follow up on this as we intend to drop support for bionic in devstack master completely | 09:47 |
| openstackgerrit | Victoria Martinez de la Cruz proposed openstack/devstack-plugin-ceph master: Bump up Ceph to Pacific and Ganesha to v3.5 https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/787774 | 09:48 |
| lyarwood | frickler: you can use http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22Failed%20to%20start%20rtslib-fb-targetctl.service%5C%22 | 09:48 |
| lyarwood | frickler: or to get an actual list use to command line client | 09:48 |
| * lyarwood tries quickly | 09:49 | |
| frickler | lyarwood: the logstash query is fine I guess. much too much legacy hidden there IMO | 09:51 |
| lyarwood | yeah agreed, I knew about the nova grenade job that I've been trying to update for ages now but the others were a surprise | 09:56 |
| openstackgerrit | Merged openstack/devstack stable/queens: Fix for downloading uwsgi on centos in lib/apache https://review.opendev.org/c/openstack/devstack/+/736189 | 09:57 |
| *** vishalmanchanda has quit IRC | 10:00 | |
| *** vishalmanchanda has joined #openstack-qa | 10:11 | |
| *** gfidente is now known as gfidente|off | 10:17 | |
| *** whoami-rajat_ has joined #openstack-qa | 10:22 | |
| *** dtantsur is now known as dtantsur|brb | 10:32 | |
| soniya29 | kopecmartin, gmann, we have py39 released but tempest and plugins doesn't seems to use it | 10:43 |
| soniya29 | kopecmartin, gmann, Do we need to drop py38 within tempest? | 10:49 |
| frickler | soniya29: not sure what your idea is, but py36 and py38 are still the required versions to support https://governance.openstack.org/tc/reference/runtimes/xena.html | 10:52 |
| soniya29 | frickler, but latest available version of python is py39 which is currently not supported | 10:54 |
| yoctozepto | we generally have non-voting jobs with py39 to ensure early compat | 10:55 |
| *** brinzhang_ has joined #openstack-qa | 10:59 | |
| soniya29 | yoctozepto, so do we need to wait to update tempest tox? | 11:01 |
| *** brinzhang0 has quit IRC | 11:02 | |
| sean-k-mooney | soniya29: you cant drop py38 within tempest | 11:02 |
| sean-k-mooney | soniya29: py38 is part of the xena runtime defintion | 11:03 |
| soniya29 | sean-k-mooney, yeah got it | 11:03 |
| sean-k-mooney | https://github.com/openstack/governance/blob/master/reference/runtimes/xena.rst | 11:03 |
| sean-k-mooney | oh frickler said that already | 11:03 |
| soniya29 | sean-k-mooney, yeah | 11:03 |
| sean-k-mooney | have eventlets fix teh dnspython and py39 issues yet | 11:04 |
| sean-k-mooney | its been a couple of months since i tried it but in january you could not run nova or neutron propelry with py39 | 11:04 |
| soniya29 | sean-k-mooney, okay | 11:05 |
| sean-k-mooney | looks like there was a new release march 3rd 0.30.2 | 11:06 |
| sean-k-mooney | no that does not fix the compat issues | 11:07 |
| sean-k-mooney | https://github.com/eventlet/eventlet/issues/619 and https://github.com/eventlet/eventlet/issues/632 are still unresovled :( | 11:09 |
| yoctozepto | soniya29: it should be at zuul level | 11:22 |
| yoctozepto | if you test xena runtimes | 11:22 |
| yoctozepto | then py39 gets tested non-voting | 11:22 |
| *** jparoly has joined #openstack-qa | 11:27 | |
| soniya29 | yoctozepto, ohh | 11:29 |
| yoctozepto | this is about unit tests btw | 11:29 |
| yoctozepto | and tempest is usually run using distro-specific | 11:30 |
| yoctozepto | so on ubuntu focal it is py38 | 11:30 |
| yoctozepto | we probably can't afford covering all supported py versions in tempest tests | 11:30 |
| sean-k-mooney | i think fedora is the only distro we have in teh gate with 39 by default? debian and ubunut can proably install it but its not there default | 11:31 |
| *** jpena is now known as jpena|lunch | 11:31 | |
| yoctozepto | debian bullseye has it as default | 11:31 |
| yoctozepto | focal? I don't think it has py39 at all | 11:31 |
| yoctozepto | I mean, in the builtin repos | 11:32 |
| sean-k-mooney | ya looking it has to come from a ppa | 11:32 |
| sean-k-mooney | at least on 20.04 | 11:33 |
| yoctozepto | mhm | 11:33 |
| sean-k-mooney | its in 20.10 | 11:34 |
| sean-k-mooney | in universe | 11:34 |
| sean-k-mooney | 21.04 defaults to 3.9 | 11:34 |
| sean-k-mooney | and has 3.10 in universe so we proably wont get 3.9+ tempest jobs untill centos 9 stream is out in the Y cycle | 11:35 |
| sean-k-mooney | although we coudl do it today if we really needed too | 11:36 |
| yoctozepto | yeah, if we | 11:36 |
| yoctozepto | it's usually more useful to bump the lower bound (i.e. drop 3.6 now to start with 3.8) than the upper | 11:37 |
| yoctozepto | to be able to use 3.8+ features | 11:37 |
| sean-k-mooney | yep | 11:38 |
| sean-k-mooney | droping 3.5 gave use fstrings | 11:38 |
| yoctozepto | bumping upper brings only misery :D | 11:38 |
| yoctozepto | yes, and they are cool | 11:38 |
| yoctozepto | even more if you one would normally use them | 11:38 |
| *** dtantsur|brb is now known as dtantsur | 11:41 | |
| *** rh-jelabarre has joined #openstack-qa | 11:49 | |
| *** dviroel|afk is now known as dviroel | 11:51 | |
| frickler | yoctozepto: soniya29: gmann: actually this should become xena now and already have been wallaby before instead of victoria I guess? https://opendev.org/openstack/tempest/src/branch/master/zuul.d/project.yaml#L6 | 11:53 |
| amodi | lyarwood: hi, re: https://review.opendev.org/c/openstack/whitebox-tempest-plugin/+/786979/8/whitebox_tempest_plugin/api/compute/test_rbd_direct_download.py#51, can u suggest if cmd = "sh -c \"zgrep 'Attempting to export RBD image[:]"... is the right way to grep for logs, the zuul job has a different structure so i can't grep for the logs at the same location | 11:57 |
| lyarwood | amodi: you would need to use journalctl | 11:59 |
| yoctozepto | frickler: ooh, indeed! | 12:00 |
| yoctozepto | if that is what soniya29 observed | 12:00 |
| yoctozepto | then it is solved by bumping to xena | 12:00 |
| yoctozepto | normally, it's done by bot on release | 12:01 |
| yoctozepto | for wallaby https://review.opendev.org/c/openstack/tempest/+/757253 | 12:02 |
| amodi | lyarwood: just saw the comments, thanks! will implement | 12:11 |
| *** ysandeep is now known as ysandeep|afk | 12:12 | |
| frickler | yoctozepto: doesn't look like that was a bot. I'll propose a patch, didn't have one for tempest in a while ;) | 12:12 |
| yoctozepto | frickler: because tempest does not release this way | 12:13 |
| yoctozepto | so bot does not cover it | 12:13 |
| openstackgerrit | Victoria Martinez de la Cruz proposed openstack/devstack-plugin-ceph master: Bump up Ceph to Pacific and Ganesha to v3.5 https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/787774 | 12:13 |
| yoctozepto | hence no xena patch | 12:13 |
| openstackgerrit | Dr. Jens Harbott proposed openstack/tempest master: Bump the openstack-python3 jobs to xena https://review.opendev.org/c/openstack/tempest/+/788480 | 12:16 |
| frickler | yoctozepto: ah, right, that explains why sdk was in the same situation last week | 12:17 |
| *** jpena|lunch is now known as jpena | 12:19 | |
| lyarwood | anyone working on the memory_tracker.service failures on master? | 12:28 |
| * lyarwood writes up a bug | 12:30 | |
| *** whoami-rajat_ is now known as whoami-rajat | 12:32 | |
| *** hemanth_n has quit IRC | 12:32 | |
| yoctozepto | frickler: afaik, sdk should be getting this help from bot | 12:35 |
| soniya29 | yoctozepto, frickler, okay thanks! | 12:35 |
| yoctozepto | but may be mistaken | 12:35 |
| yoctozepto | lyarwood: /me not aware | 12:35 |
| lyarwood | https://bugs.launchpad.net/devstack/+bug/1926434 - I've seen it a few times today | 12:37 |
| openstack | Launchpad bug 1926434 in devstack "devstack@memory_tracker.service: Main process exited, code=exited, status=1/FAILURE" [Undecided,New] | 12:37 |
| *** nweinber has joined #openstack-qa | 12:45 | |
| frickler | lyarwood: does that lead to actual job failures or is this just warning-like, like the worlddump messages? | 12:47 |
| lyarwood | frickler: job failure AFAICT | 12:50 |
| lyarwood | frickler: exit_trap runs just after this failure | 12:51 |
| lyarwood | might be missing something if this is async | 12:51 |
| *** ysandeep|afk is now known as ysandeep | 12:56 | |
| *** iurygregory has quit IRC | 12:59 | |
| tosky | frickler, lyarwood : sooo... about bumping the size of the cinder volume to 30G, can we somehow find a solution? | 13:00 |
| frickler | lyarwood: ah, o.k.. I think this might be https://review.opendev.org/c/openstack/devstack/+/620198 , could be worth to revisit that patch, not sure why it would suddenly happen so often, possibly indeed triggered by switching to async | 13:00 |
| frickler | tosky: I just dropped my -1, so you can go ahead with that | 13:04 |
| gmann | soniya29: yeah, we need to test as per cycle defined testing runtime. but we do have a n-v job for py39 testing which will be future cycle testing runtime | 13:07 |
| soniya29 | gmann, okay, so we will have py9 for future | 13:08 |
| gmann | as voting job, yes | 13:08 |
| gmann | py3.9 | 13:08 |
| tosky | frfo | 13:08 |
| tosky | frickler: thanks! | 13:08 |
| soniya29 | gmann, ack | 13:13 |
| *** eliadcohen has joined #openstack-qa | 13:22 | |
| *** iurygregory has joined #openstack-qa | 13:35 | |
| kopecmartin | gmann: when you have a moment https://review.opendev.org/c/openstack/tempest/+/783140 | 14:09 |
| dansmith | gmann: ack, I added manual cleanup between tests which got me closer, thanks | 14:13 |
| openstackgerrit | Dr. Jens Harbott proposed openstack/devstack master: Handle disappearing pids in mlock_report.py https://review.opendev.org/c/openstack/devstack/+/620198 | 14:15 |
| frickler | lyarwood: ^^ actually after looking a bit, I'm pretty sure that ianw's patch is the right fix for your issues. rebased and added the bug reference | 14:18 |
| lyarwood | frickler: ack thanks, I'll review shortly | 14:18 |
| dansmith | gmann: did you tell me once that tests within a single class in tempest will run serially? | 14:27 |
| *** dave-mccowan has joined #openstack-qa | 14:36 | |
| openstackgerrit | Dan Smith proposed openstack/tempest master: Add tests for glance unified quotas https://review.opendev.org/c/openstack/tempest/+/788346 | 14:43 |
| gmann | dansmith: yes, within class tests are serial run so safe to use shared resources. | 14:43 |
| dansmith | gmann: hrm, okay I don't understand why I can't repro these failures then | 14:43 |
| gmann | dansmith: which one? | 14:44 |
| dansmith | on the patch I'm working on ^ | 14:44 |
| dansmith | I just split them out into separate classes as a test, but if you're right about serializing the runs, it shouldn't matter | 14:44 |
| gmann | dansmith: quota is little different that way, as updating quota can impact on overall on service. did you use lock for those to avoid issues of other' test class run in parallel? | 14:46 |
| dansmith | ah crap, guess what? | 14:46 |
| dansmith | I just realized my devstack patch doesn't point at the top of my glance stack :X | 14:46 |
| dansmith | me not so smart | 14:46 |
| openstackgerrit | Dan Smith proposed openstack/devstack master: Configure glance unified limit quotas https://review.opendev.org/c/openstack/devstack/+/788056 | 14:48 |
| gmann | dansmith: ok and you can use lock also if there are more than on test class changing the quota | 14:48 |
| dansmith | gmann: within a dedicated quota class I should have an isolated tenant right? | 14:48 |
| gmann | like we do for compute quota test | 14:48 |
| dansmith | dedicated quota *test* class I mean | 14:48 |
| gmann | dansmith: if that is just one class having all image quota test then fine. | 14:49 |
| dansmith | yeah, okay I should be good then I think | 14:49 |
| dansmith | I had my devstack patch pointing to not the top of my glance stack, | 14:49 |
| gmann | dansmith: but anyone add new one in tempest or glance-tempest-plugin then it can create issue | 14:49 |
| dansmith | and I was like "why is it not enforcing this one quota item?" <-- because I didn't run against all my patches | 14:49 |
| dansmith | gmann: I thought we created an destroyed a tenant for each test class, is that not right? | 14:50 |
| gmann | dansmith: yeah, in that case tenant level quota fine but if we have global quota or so. but not sure about image quota structure | 14:51 |
| dansmith | yeah, this is all tenant-level, so I should be good I think | 14:52 |
| gmann | ok, | 14:52 |
| openstackgerrit | Dan Smith proposed openstack/tempest master: Add tests for glance unified quotas https://review.opendev.org/c/openstack/tempest/+/788346 | 14:52 |
| *** rcernin has quit IRC | 15:01 | |
| openstackgerrit | Ghanshyam proposed openstack/patrole master: Move to Xena testing runtime template https://review.opendev.org/c/openstack/patrole/+/788511 | 15:06 |
| *** rcernin has joined #openstack-qa | 15:12 | |
| *** rcernin has quit IRC | 15:17 | |
| *** dviroel is now known as dviroel|lunch | 15:22 | |
| *** ysandeep is now known as ysandeep|away | 15:56 | |
| *** rcernin has joined #openstack-qa | 15:59 | |
| *** lucasagomes has quit IRC | 16:04 | |
| *** rcernin has quit IRC | 16:05 | |
| *** ysandeep|away is now known as ysandeep | 16:14 | |
| *** jpena is now known as jpena|off | 16:18 | |
| *** dviroel|lunch is now known as dviroel | 16:20 | |
| *** dtantsur is now known as dtantsur|afk | 16:22 | |
| *** ysirndjuro has joined #openstack-qa | 16:25 | |
| openstackgerrit | Stephen Finucane proposed openstack/devstack master: Handle disappearing pids in mlock_report.py https://review.opendev.org/c/openstack/devstack/+/620198 | 16:31 |
| *** rcernin has joined #openstack-qa | 16:38 | |
| *** rpittau is now known as rpittau|afk | 16:39 | |
| openstackgerrit | Victoria Martinez de la Cruz proposed openstack/devstack-plugin-ceph master: Bump up Ceph to Pacific and Ganesha to v3.5 https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/787774 | 16:40 |
| *** rcernin has quit IRC | 16:43 | |
| *** rcernin has joined #openstack-qa | 16:57 | |
| *** rcernin has quit IRC | 17:02 | |
| *** hamalq has joined #openstack-qa | 17:04 | |
| *** hamalq has quit IRC | 17:05 | |
| *** hamalq has joined #openstack-qa | 17:06 | |
| *** ysandeep is now known as ysandeep|away | 17:14 | |
| *** lbragstad has quit IRC | 17:37 | |
| *** lbragstad has joined #openstack-qa | 17:38 | |
| *** eolivare has quit IRC | 17:44 | |
| *** ralonsoh has quit IRC | 17:53 | |
| openstackgerrit | Merged openstack/tempest master: Bump the openstack-python3 jobs to xena https://review.opendev.org/c/openstack/tempest/+/788480 | 18:57 |
| clarkb | Does anyone recognize this error 'keystone.exception.ValidationError: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect.' when using devstack? The command that causes it to happen is `openstack --os-project-name demo --os-username demo server create --flavor cirros256 --image $cirros_image unmanaged-vm | 19:16 |
| clarkb | --network public` | 19:16 |
| clarkb | this error does not occur 100% of the time. Much less than that | 19:16 |
| clarkb | looks like we run that as admin `source /opt/devstack/openrc admin admin` occurs just before that server create | 19:18 |
| *** amodi has quit IRC | 19:20 | |
| clarkb | really old bugs indicate that the issue is often mitigated by specifying the domain name. However, devstack's openrc should already do that | 19:23 |
| clarkb | note this is with master devstack | 19:23 |
| clarkb | nova validate_networks() seems to the source of the problem. I wonder if validate_networks is somehow pruning things before askign keystone | 19:30 |
| *** vishalmanchanda has quit IRC | 19:40 | |
| *** sboyron has quit IRC | 19:41 | |
| *** nweinber has quit IRC | 19:51 | |
| *** lbragstad_ has joined #openstack-qa | 20:01 | |
| *** lbragstad has quit IRC | 20:03 | |
| *** dave-mccowan has quit IRC | 20:22 | |
| *** dave-mccowan has joined #openstack-qa | 20:25 | |
| *** jparoly has quit IRC | 20:35 | |
| *** paras333_ has quit IRC | 20:48 | |
| *** whoami-rajat has quit IRC | 20:52 | |
| *** amodi has joined #openstack-qa | 20:54 | |
| *** brinzhang0 has joined #openstack-qa | 20:59 | |
| *** brinzhang_ has quit IRC | 21:02 | |
| openstackgerrit | Merged openstack/devstack master: cinder: Default CINDER_ISCSI_HELPER to tgtadm on Bionic https://review.opendev.org/c/openstack/devstack/+/788429 | 21:24 |
| *** slaweq has quit IRC | 21:25 | |
| *** ysirndjuro has quit IRC | 21:35 | |
| *** gcheresh has quit IRC | 21:38 | |
| *** rcernin has joined #openstack-qa | 21:40 | |
| ianw | frickler: heh "Patchset 1 Nov 27, 2018 1:01 PM" ... i knew it would have it's day to shine :) | 21:42 |
| *** rh-jelabarre has quit IRC | 21:45 | |
| openstackgerrit | Dan Smith proposed openstack/tempest master: Add tests for glance unified quotas https://review.opendev.org/c/openstack/tempest/+/788346 | 21:58 |
| *** rcernin has quit IRC | 22:06 | |
| *** rcernin has joined #openstack-qa | 22:07 | |
| *** paras333_ has joined #openstack-qa | 22:10 | |
| *** paras333_ has quit IRC | 22:14 | |
| *** dviroel is now known as dviroel|away | 22:14 | |
| *** Yarboa has quit IRC | 22:33 | |
| *** Yarboa has joined #openstack-qa | 22:35 | |
| *** tosky has quit IRC | 22:48 | |
| *** Yarboa has quit IRC | 23:33 | |
| *** Yarboa has joined #openstack-qa | 23:35 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!