Tuesday, 2019-04-02

*** hongbin has joined #openstack-qa00:37
*** fnaval has quit IRC00:43
*** markvoelker has joined #openstack-qa00:55
*** yamamoto has joined #openstack-qa01:02
*** yamamoto has quit IRC01:07
*** dulek has quit IRC01:21
*** whoami-rajat has joined #openstack-qa01:31
*** dulek has joined #openstack-qa01:34
*** brinzhang has joined #openstack-qa01:35
*** brinzhang has quit IRC01:47
*** brinzhang has joined #openstack-qa01:48
*** lbragstad has joined #openstack-qa02:02
*** rcernin_ has joined #openstack-qa02:05
*** rcernin has quit IRC02:06
*** rcernin_ has quit IRC02:12
*** rcernin has joined #openstack-qa02:15
*** gkadam has joined #openstack-qa02:48
*** gkadam has quit IRC02:50
*** hongbin has quit IRC02:57
*** hongbin has joined #openstack-qa02:58
*** yamamoto has joined #openstack-qa03:06
*** hongbin has quit IRC03:12
*** hongbin has joined #openstack-qa03:14
*** samueldmq has quit IRC03:15
*** hongbin has quit IRC03:31
*** spsurya has joined #openstack-qa03:32
*** brinzhang has quit IRC03:45
*** brinzhang has joined #openstack-qa03:46
*** yamamoto has quit IRC04:06
*** yamamoto has joined #openstack-qa04:06
*** yamamoto has quit IRC04:06
*** yamamoto has joined #openstack-qa04:45
*** lbragstad has quit IRC05:33
*** pcaruana has joined #openstack-qa05:35
*** ramishra has joined #openstack-qa05:38
*** yamamoto has quit IRC05:48
*** markvoelker has quit IRC05:58
*** ccamposr has joined #openstack-qa06:09
*** sridharg has joined #openstack-qa06:12
*** markvoelker has joined #openstack-qa06:29
*** cshen has joined #openstack-qa06:40
*** slaweq has joined #openstack-qa06:44
*** gcheresh has joined #openstack-qa07:02
*** kopecmartin|off is now known as kopecmartin07:02
*** tosky has joined #openstack-qa07:09
*** yamamoto has joined #openstack-qa07:13
*** e0ne has joined #openstack-qa07:16
*** iurygregory has joined #openstack-qa07:18
*** rpittau|afk is now known as rpittau07:23
*** jpena|off is now known as jpena07:58
*** e0ne has quit IRC08:01
*** e0ne has joined #openstack-qa08:08
*** gkadam has joined #openstack-qa08:10
*** lucasagomes has joined #openstack-qa08:12
*** raissa has joined #openstack-qa08:17
*** zbr|pto is now known as zbr08:23
fricklerstill failed again. test_resize_volume_backed_server_confirm is one of the most unstable tests I'm seeing, maybe we should drop it from tempest-full?08:35
fricklerit's also interesting how centos+fedora+opensuse are failing the same set of compute tests, maybe some nova folk can have a deeper look there08:37
*** raissa has quit IRC08:51
*** manjeets has quit IRC08:56
*** manjeets has joined #openstack-qa08:57
*** gfidente has joined #openstack-qa08:57
*** gfidente has quit IRC08:59
*** manjeets has quit IRC09:05
*** e0ne has quit IRC09:07
*** rcernin has quit IRC09:11
*** gfidente has joined #openstack-qa09:13
*** e0ne has joined #openstack-qa09:15
*** dtantsur|afk is now known as dtantsur09:33
*** manjeets has joined #openstack-qa09:38
*** e0ne has quit IRC09:51
*** openstackgerrit has joined #openstack-qa09:55
openstackgerritMerged openstack/tempest master: Add missing validation for the compute/positive  https://review.openstack.org/64755409:55
*** zigo has joined #openstack-qa10:00
*** yamamoto has quit IRC10:01
openstackgerritJens Harbott (frickler) proposed openstack-dev/devstack master: Debug check_python3_support_for_package_remote  https://review.openstack.org/64909110:04
openstackgerritJens Harbott (frickler) proposed openstack-dev/devstack master: Remove crusty old python 3 package version logic  https://review.openstack.org/64909610:13
*** yamamoto has joined #openstack-qa10:14
*** yamamoto has quit IRC10:18
openstackgerritMerged openstack-dev/devstack stable/rocky: Fix installing tempest plugins  https://review.openstack.org/64333410:28
*** e0ne has joined #openstack-qa10:28
openstackgerritLuigi Toscano proposed openstack-dev/devstack master: sync-devstack-data: new argument devstack_data_base_dir  https://review.openstack.org/64927510:34
openstackgerritLuigi Toscano proposed openstack-dev/grenade master: Setup grenade-base, grenade, grenade-{py3,postgresql}  https://review.openstack.org/54893610:35
openstackgerritLuigi Toscano proposed openstack-dev/grenade master: DNM - grenade-heat-test job  https://review.openstack.org/63977410:35
*** e0ne has quit IRC10:45
*** e0ne has joined #openstack-qa10:52
*** yamamoto has joined #openstack-qa10:57
fricklermriedem: I think you've uncovered a severe gap in our py3 testing with https://review.openstack.org/64909710:57
fricklereven with USE_PYTHON3=True we setup venvs with py210:57
fricklerbecause stackrc reads the variable from localrc only after it has already set VIRTUALENV_CMD10:58
fricklerhttp://logs.openstack.org/49/636949/6/check/tempest-full-py3/782106a/job-output.txt.gz#_2019-03-29_19_03_02_48109410:58
fricklercompare this to the code being executed here https://opendev.org/openstack-dev/devstack/src/branch/master/stackrc#L146-L15110:59
*** yamamoto has quit IRC11:03
*** yamamoto has joined #openstack-qa11:03
openstackgerritJens Harbott (frickler) proposed openstack-dev/devstack master: Handle localrc early enough in stackrc  https://review.openstack.org/64927811:04
fricklerI'm assuming that this will actually break the py3 jobs, but we can iterate on that then and fix what we need to make a successfull real py3 job11:05
fricklerI wonder if we should even make a job that removes py27 just in order to have a proof that everything works with py311:06
*** yamamoto has quit IRC11:17
*** jpena is now known as jpena|lunch11:29
*** yamamoto has joined #openstack-qa11:42
*** e0ne has quit IRC11:56
*** yamamoto has quit IRC11:59
*** yamamoto has joined #openstack-qa12:00
openstackgerritVlad Gridin proposed openstack-dev/devstack stable/queens: Fix installing tempest plugins  https://review.openstack.org/64334112:01
*** e0ne has joined #openstack-qa12:03
*** yamamoto has quit IRC12:07
*** manjeets has quit IRC12:16
*** markvoelker has quit IRC12:25
*** markvoelker has joined #openstack-qa12:25
*** jpena|lunch is now known as jpena12:27
*** e0ne has quit IRC12:31
*** yamamoto has joined #openstack-qa12:33
*** brinzhang has quit IRC12:39
*** e0ne has joined #openstack-qa12:44
*** mriedem has joined #openstack-qa12:52
*** mjturek has joined #openstack-qa12:52
*** mjturek has quit IRC12:53
*** mjturek has joined #openstack-qa12:54
openstackgerritMartin Kopec proposed openstack/tempest master: Improve router deletion logging in tempest cleanup  https://review.openstack.org/64905312:55
*** raildo has joined #openstack-qa12:59
*** rh-jelabarre has joined #openstack-qa13:03
*** mriedem has left #openstack-qa13:08
*** mriedem has quit IRC13:08
*** mriedem has joined #openstack-qa13:09
*** lbragstad has joined #openstack-qa13:22
*** trident has quit IRC13:30
*** trident has joined #openstack-qa13:33
*** e0ne has quit IRC13:36
*** e0ne has joined #openstack-qa13:42
*** e0ne has quit IRC13:43
*** awaugama has joined #openstack-qa13:49
toskyso... I've tried enabling tls-proxy by default on the native grenade job but it fails horribly13:54
toskyand when I tried to compare with the existing jobs, I realized that maybe we don't have grenade jobs that enable tls-proxy13:55
toskyI would like to be proven wrong13:56
toskygmann: ^^13:56
*** yamamoto has quit IRC14:01
*** yamamoto has joined #openstack-qa14:04
*** yamamoto has quit IRC14:04
*** yamamoto has joined #openstack-qa14:05
openstackgerritJens Harbott (frickler) proposed openstack-dev/devstack master: Handle localrc early enough in stackrc  https://review.openstack.org/64927814:06
*** e0ne has joined #openstack-qa14:10
*** cshen has quit IRC14:33
*** gcheresh has quit IRC14:42
*** sridharg has quit IRC14:56
*** fnaval has joined #openstack-qa15:01
fricklerso I did some testing regarding the "pip fails because it cannot find wheel on our mirror" issue as seen here http://logs.openstack.org/78/649278/2/check/tempest-full-py3/a334730/job-output.txt.gz#_2019-04-02_14_32_36_42636115:05
fricklerI can reproduce this locally by using the same /etc/pip.conf that we place onto our nodes15:06
fricklerbut only when the venv contains pip==9.0.1, which stems from the ubuntu pkg python-pip-whl==9.0.1-2.3~ubuntu1 . even if I upgrade pip only slightly to 9.0.3 (=latest < 10 what we install in devstack), the error is resolved15:07
*** rh-jelabarre has quit IRC15:08
fricklerclarkb: fungi: mriedem: that's related to what we saw yesterday. ^^ probably the fix will be to update pip for every py3 venv we create15:09
clarkbfrickler: ya the issue is in the distro packaging of pip by way of a pip bug15:10
clarkbfrickler: https://github.com/pypa/pip/pull/6367 has more details and pip is asking for help testing a patch that I haven't been able to do15:11
*** e0ne has quit IRC15:11
clarkbI asked zigo on irc if zigo could test it but not response yet. zigo let me know if I/we can help with that more15:11
fungithe discussion also continued over to the debian-python ml where there's a good chance it'll be incorporated into buster15:14
fungiin which case it'll be easier to argue for a sru in ubuntu bionic as well15:14
frickleroh, so it's only the distro version that is broken, great. I can confirm that in my test when I downgrade to pip==9.0.1 via pip again, things work15:15
fricklerdo you know if there's an ubuntu/lp bug for that already? maybe jamespage / coreycb could help a bit, then15:16
openstackgerritJens Harbott (frickler) proposed openstack-dev/devstack master: Handle localrc early enough in stackrc  https://review.openstack.org/64927815:28
*** ccamposr has quit IRC15:33
zigoclarkb: Sorry, test what?15:34
zigopip?15:35
fricklerzigo: https://github.com/pypa/pip/pull/636715:35
zigoI thought it was tested ...15:35
zigo:/15:35
zigoclarkb: There's the package in Unstable that you can try already.15:35
fricklerzigo: maybe just lacking proper feedback to upstream?15:35
zigoLet me try now.15:35
clarkbzigo frickler mine 2as tested they have suggested an alternative15:36
clarkbthe alternative is more resilient long term to code changes but my patch should work as is15:36
clarkbthe ask is to test the alternative15:36
*** e0ne has joined #openstack-qa15:38
zigoclarkb: So, basically, I should test https://github.com/pypa/pip/commit/8ef3283fcf7c7c5537f305a374355edc96f40d3d instead of your patch?15:39
zigoclarkb: Today, the machine hosting my bouncer crashed, so I had to setup a new IRC bouncer, and therefore, I don't have the history...15:40
zigoSo, sorry to ask again, how do I check if pip works?15:40
clarkbzigo https://github.com/pypa/pip/commit/8ef3283fcf7c7c5537f305a374355edc96f40d3d and https://github.com/yan12125/pip/commit/7a29806a6b8eca0a1ec4e265d7b20610d5609742 together15:40
gmanntosky: yes, nova-next is the only job enable tls-proxy.15:40
toskygmann: oh, the only grenade job? I'd like to compare the settings15:41
toskyor, if you agree, I may just disable tls-proxy right now on the ne wgrenade job15:41
toskyand try to fix the issues later15:41
clarkbzigo: to check if pip works run `pip install --extra-index-url http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64/ --trusted-host mirror.dfw.rax.openstack.org -U pip`15:42
clarkbzigo: don't worry that that extra index url is for ubuntu wheels since pip will 404 anyway15:42
clarkbzigo: the condition we need to test is that the extra index url 404s for the package listed15:42
zigoclarkb: It does work with your patch for me ...15:42
gmanntosky: that is not grenade job, i think we do not have tls-proxy in current grenade job15:42
clarkbzigo: yup my patch works and should be fine. Pip is just asking that we test the alternative because the alternative would be better for long term code maintenance15:43
clarkbits a manageablity of the code issue not a concern of functionality in the current code state15:43
toskygmann: right, that looks indeed like a devstack job, but tls-proxy is enabled by default on all native zuulv3 devstack jobs15:43
zigoclarkb: Ok, so I just test it, but I don't need to package it in Debian, right?15:43
gmanntosky: ah yeah, that is enabled for zuulv3 natie jobs15:44
gmanntosky: we can leave as of now and keep a TODO there15:44
clarkbzigo: I think if pip switches to that alternative you'll eventually pull it in via code updates and then you can probably drop my patch that you already proposed15:45
clarkbzigo: so it isn't urgent but longer term you'll end up switching to it and life will be better15:45
zigoclarkb: Yeah, that's what I expect to do indeed.15:45
zigoBut since I have already sent the bugs to the Debian release team to fix things with your patch in both Stretch and Buster, I'd prefer not to revert that...15:45
clarkbzigo: I believe that to be ok. My patch works and is functional from what I can tell. Its just they don't like it because if things rename again in the future it will potentially break again15:46
clarkbthis is less of a concern for debian because debian mostly fixes the upstream source when they make a release15:46
openstackgerritLuigi Toscano proposed openstack-dev/grenade master: Setup grenade-base, grenade, grenade-{py3,postgresql}  https://review.openstack.org/54893615:47
openstackgerritLuigi Toscano proposed openstack-dev/grenade master: DNM - grenade-heat-test job  https://review.openstack.org/63977415:47
* zigo is testing right now ...15:48
*** iurygregory has quit IRC15:55
*** lucasagomes has quit IRC15:56
*** rh-jelabarre has joined #openstack-qa16:20
*** rpittau is now known as rpittau|afk16:24
*** dtantsur is now known as dtantsur|afk16:24
*** e0ne has quit IRC16:28
*** ramishra has quit IRC16:34
*** gfidente is now known as gfidente|afk16:45
*** gfidente|afk has quit IRC16:58
*** amodi has quit IRC16:58
*** gkadam has quit IRC17:02
*** artom has quit IRC17:22
*** rh-jelabarre has quit IRC17:25
*** jpena is now known as jpena|off17:28
*** rh-jelabarre has joined #openstack-qa17:29
*** amodi has joined #openstack-qa17:36
fricklerzigo: can you tell me the debian bug ids? I created a bug for ubuntu, too https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/182284217:37
openstackLaunchpad bug 1822842 in python-pip (Ubuntu) "pip fails to install package from source when extra-index-url returns 404" [Undecided,New]17:37
*** cshen has joined #openstack-qa17:37
fricklerclarkb: do you know a workaround for the issue? of course pip is also failing to update itself with the pip.conf we have in place http://logs.openstack.org/78/649278/3/check/tempest-full-py3/ee98517/job-output.txt.gz#_2019-04-02_17_35_42_62551517:42
clarkbfrickler: the workaround is to install pip using get-pip (basically don't use distro pip to install pip)17:43
fricklerclarkb: we do that, but for a venv generated with "python3 -m venv", it contains the distro pip again17:47
*** cshen has quit IRC17:48
clarkboh then we need to install virtualenv from pip first17:49
clarkbfrickler: ^17:50
*** kopecmartin is now known as kopecmartin|off18:04
*** e0ne has joined #openstack-qa18:04
*** artom has joined #openstack-qa18:14
openstackgerritJens Harbott (frickler) proposed openstack-dev/devstack master: Handle localrc early enough in stackrc  https://review.openstack.org/64927818:21
fricklerclarkb: we actually have that, so from testing locally, it should be enough to switch to actually use it for py3 venvs, too ^^18:22
clarkbfrickler: is the change at https://review.openstack.org/#/c/649278/4/inc/python necessary if using `virtualenv`?18:23
clarkbbut ya using pip installed virtualenv with -p should work great18:23
fricklerclarkb: it seems like I can indeed drop that again, too, good point18:26
openstackgerritJens Harbott (frickler) proposed openstack-dev/devstack master: Handle localrc early enough in stackrc  https://review.openstack.org/64927818:28
*** spsurya has quit IRC18:32
*** artom has quit IRC18:58
*** mjturek has quit IRC19:02
*** mjturek has joined #openstack-qa19:03
*** gmann is now known as gmann_afk19:04
*** e0ne has quit IRC19:20
awaugamaj19:22
*** rh-jelabarre has quit IRC19:26
*** rh-jelabarre has joined #openstack-qa19:30
*** awaugama has quit IRC19:30
zigofrickler: https://bugs.debian.org/83776419:34
openstackDebian bug 837764 in src:python-pip "python-pip: Using `--extra-index-url` results in `HTTPError: 404 Client Error: NOT FOUND`" [Important,Fixed]19:35
zigofrickler: The bug is closed because I uploaded to Sid, though it's currently blocked to migrate to Buster as per: https://qa.debian.org/excuses.php?package=python-pip19:36
zigoI opened a bugs against release.debian.org as per the procedure, and I'm waiting for the Debian release team members to reply with approval or rejection.19:36
zigoGiven that clarkb's patch are 2 lines of change, it's likely to be approved for both Buster and Stretch.19:37
zigoThough I do expect it to take a week or so...19:37
*** cshen has joined #openstack-qa19:43
*** cshen has quit IRC19:48
openstackgerritEric Fried proposed openstack-infra/elastic-recheck master: Update query for 1820892  https://review.openstack.org/64944520:05
*** mjturek has quit IRC20:11
*** artom has joined #openstack-qa20:17
*** mjturek has joined #openstack-qa20:22
*** markvoelker has quit IRC20:23
*** pcaruana has quit IRC20:30
*** whoami-rajat has quit IRC20:30
*** mjturek has quit IRC20:37
*** amodi has quit IRC21:01
*** slaweq has quit IRC21:07
openstackgerritMerged openstack-infra/elastic-recheck master: Update query for 1820892  https://review.openstack.org/64944521:16
*** slaweq has joined #openstack-qa21:24
*** slaweq has quit IRC21:28
*** cshen has joined #openstack-qa21:44
*** cshen has quit IRC21:48
*** fnaval has quit IRC22:10
*** rcernin has joined #openstack-qa22:25
openstackgerritMatt Riedemann proposed openstack-dev/devstack master: DNM: test nova with compute rpc api pinned to queens  https://review.openstack.org/64947022:38
*** slaweq has joined #openstack-qa22:38
*** slaweq has quit IRC22:43
openstackgerritmelanie witt proposed openstack-dev/devstack master: Configure console proxy ports in nova_cellN.conf  https://review.openstack.org/64947322:58
*** gmann_afk is now known as gmann23:11
*** tosky has quit IRC23:24
*** cshen has joined #openstack-qa23:44
*** cshen has quit IRC23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!