Tuesday, 2020-03-03

*** threestrands has joined #openstack-ansible00:03
*** weifan has joined #openstack-ansible00:05
*** weifan has quit IRC00:05
*** aedc has quit IRC00:50
*** owalsh has quit IRC00:54
*** aedc has joined #openstack-ansible00:58
*** owalsh has joined #openstack-ansible01:00
*** yolanda has quit IRC01:15
*** gyee has quit IRC02:08
*** joshualyle has quit IRC02:26
openstackgerritEmilien Macchi proposed openstack/openstack-ansible-os_tempest master: post_install: add missing onditional to workspace task  https://review.opendev.org/71089302:30
*** weifan has joined #openstack-ansible03:22
*** weifan has quit IRC03:26
*** weifan has joined #openstack-ansible03:28
*** weifan has quit IRC03:33
*** dave-mccowan has quit IRC04:04
*** raukadah is now known as chandankumar04:04
*** jamesdenton has quit IRC05:03
*** jamesdenton has joined #openstack-ansible05:03
*** vesper has quit IRC05:34
*** evrardjp has quit IRC05:35
*** evrardjp has joined #openstack-ansible05:35
*** vesper11 has joined #openstack-ansible05:37
*** macz_ has joined #openstack-ansible05:50
*** macz_ has quit IRC05:55
*** ansmith has quit IRC06:21
*** ansmith_ has joined #openstack-ansible06:21
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add compute node deployment script  https://review.opendev.org/70649107:01
*** fghaas has joined #openstack-ansible07:13
*** hwoarang has quit IRC07:18
*** yolanda has joined #openstack-ansible07:21
*** hwoarang has joined #openstack-ansible07:21
*** shyamb has joined #openstack-ansible07:25
*** cshen has joined #openstack-ansible07:31
*** gshippey has quit IRC07:33
*** cshen has quit IRC07:44
*** shyamb has quit IRC07:50
*** cshen has joined #openstack-ansible07:50
*** pcaruana has joined #openstack-ansible07:55
*** trikorasolns has joined #openstack-ansible07:56
*** cshen has quit IRC08:01
*** this10nly has joined #openstack-ansible08:03
*** pcaruana has quit IRC08:16
*** fghaas has quit IRC08:19
*** ansmith_ has quit IRC08:20
*** tosky has joined #openstack-ansible08:20
*** ansmith has joined #openstack-ansible08:21
*** jbadiapa has joined #openstack-ansible08:21
*** pcaruana has joined #openstack-ansible08:21
*** hwoarang has quit IRC08:27
*** hwoarang has joined #openstack-ansible08:28
*** stingrayza has joined #openstack-ansible08:34
*** this10nly has quit IRC08:36
*** rpittau|afk is now known as rpittau08:41
*** sshnaidm|pto has quit IRC08:45
*** ysastri has joined #openstack-ansible08:48
*** DanyC has joined #openstack-ansible08:50
*** DanyC has quit IRC08:50
*** DanyC has joined #openstack-ansible08:51
*** DanyC has quit IRC08:52
*** DanyC has joined #openstack-ansible08:53
*** DanyC has joined #openstack-ansible08:54
*** DanyC_ has joined #openstack-ansible08:54
*** shyamb has joined #openstack-ansible08:57
*** DanyC__ has joined #openstack-ansible08:57
*** DanyC has quit IRC08:58
*** sshnaidm|pto has joined #openstack-ansible08:59
*** DanyC_ has quit IRC08:59
*** cshen has joined #openstack-ansible09:03
*** threestrands has quit IRC09:09
*** udesale has joined #openstack-ansible09:14
*** pcaruana has quit IRC09:16
*** sshnaidm|pto is now known as sshnaidm09:17
*** fghaas has joined #openstack-ansible09:26
*** shyamb has quit IRC09:30
*** pcaruana has joined #openstack-ansible09:35
*** ysastri has quit IRC09:40
*** shyamb has joined #openstack-ansible09:40
*** pcaruana has quit IRC09:43
*** gshippey has joined #openstack-ansible09:54
*** pcaruana has joined #openstack-ansible09:55
jrossernoonedeadpunk: any ideas here https://zuul.opendev.org/t/openstack/build/de0f9a117fbc44b3855213c2ab552640/log/job-output.txt#2839-284010:11
jrossercentos7 distro package virtualenv is 15.1.0, but minimum 16.0.0 is needed10:12
jrosserso we have the distro package installed already, then the pip version gets installed anyway10:12
jrosserah dear - this suggests we are not cleaning the CI image properly either https://zuul.opendev.org/t/openstack/build/1acb9bf5a43743feb5470bb68120da43/log/job-output.txt#310710:15
noonedeadpunkNot sure I undesrtood what exactly requires 16.0.0, but in our codebase I can recall only this https://opendev.org/openstack/openstack-ansible/src/branch/master/test-requirements.txt#L1010:16
jrosserthis is all functional tests10:17
jrosseri'm not sure we have any constraints for that10:17
noonedeadpunkhm, maybe we do, since in u-c it's 20.0.7.... https://opendev.org/openstack/requirements/src/branch/master/upper-constraints.txt#L79110:17
jrosserthats basically "latest" though10:18
jrosserwe've got jobs that fail i think becasue we put 20.0.7 on the host on centos-710:19
jrosserwith pip10:19
*** ivve has joined #openstack-ansible10:27
*** spatel has joined #openstack-ansible10:27
*** spatel has quit IRC10:33
*** shyamb has quit IRC10:39
*** shyamb has joined #openstack-ansible10:40
*** masterpe has quit IRC10:56
*** fridtjof[m] has quit IRC10:56
*** openstackstatus has quit IRC10:59
*** shyamb has quit IRC10:59
*** waverider has joined #openstack-ansible11:00
*** fghaas has left #openstack-ansible11:04
*** sshnaidm has quit IRC11:07
noonedeadpunkjrosser: should I also dig into that to help with figuring out what's happening?11:10
*** shyamb has joined #openstack-ansible11:11
noonedeadpunkmgariepy: so, using coreos here https://review.opendev.org/#/c/710245/5 and https://review.opendev.org/#/c/708097/ seems to pass CI11:12
*** sshnaidm has joined #openstack-ansible11:12
noonedeadpunkand tempest11:12
*** sshnaidm has quit IRC11:13
*** sshnaidm has joined #openstack-ansible11:13
*** rpittau is now known as rpittau|bbl11:17
*** shyamb has quit IRC11:18
noonedeadpunkjrosser: as I really don't see anything at glance....11:19
jrossernoonedeadpunk: give me 10mins I have a vm you can look at11:20
*** fridtjof[m] has joined #openstack-ansible11:22
*** xakaitetoia has joined #openstack-ansible11:28
noonedeadpunkbtw, also would be great to get some reviews from cores https://review.opendev.org/#/q/is:open+owner:self+-label:Workflow-1+label:Verified11:29
*** DanyC__ has quit IRC11:33
*** DanyC has joined #openstack-ansible11:34
*** cshen has quit IRC11:35
*** cshen has joined #openstack-ansible11:37
*** cshen has quit IRC11:40
*** kvivan has quit IRC11:42
*** DanyC has quit IRC11:45
*** DanyC has joined #openstack-ansible11:46
*** trident has quit IRC11:46
*** DanyC has quit IRC11:47
*** DanyC has joined #openstack-ansible11:47
*** trident has joined #openstack-ansible11:49
*** ansmith has quit IRC11:53
jrossernoonedeadpunk: centos@38.108.68.15911:58
jrossernoonedeadpunk: what i'm not sure about is how we make the functional tests environment via tox consistent11:59
jrosserhttp://paste.openstack.org/show/790245/12:01
jrosserthis is reproducing exactly how the centos7 CI jobs are failing12:01
jrosserand i'm wondering if again it's becasue we have a mixture of distro pip/virtualenv and the one that gets installed as part of bringing up the tox environment12:02
*** cshen has joined #openstack-ansible12:02
noonedeadpunkso virtualenv is a tox requirement....12:03
noonedeadpunkhttps://github.com/tox-dev/tox/blob/master/setup.cfg#L4612:04
jrosserright - and it looks like we trigger the install of the requirements here https://github.com/openstack/openstack-ansible-tests/blob/master/tox.ini#L912:05
noonedeadpunkso 3.14.0 had 14.0.0+12:05
*** masterpe has joined #openstack-ansible12:06
noonedeadpunkjrosser: no, it comes from here https://opendev.org/openstack/openstack-ansible-tests/src/branch/master/run_tests_common.sh#L8512:06
noonedeadpunkso we can set tox<=3.14 .....12:07
noonedeadpunkat least for centos 712:07
jrosseroh youre right, yes thats where it comes from12:08
jrossershould we copy the concept of global-requirement-pins.txt from the integrated repo into the tests repo?12:11
jrosseri'm just really not sure what the right approach is here - theres a lot of complexity12:11
noonedeadpunksunset tests repo? :p12:12
noonedeadpunkif seriously - not sure...12:12
jrosserso much work to do to port all the tests to the integrated repo12:13
noonedeadpunklike we probably don't want to lock with old tox...12:13
jrossertheres two issues i think12:14
noonedeadpunkand only centos 7 don't have virtualenv 16+ from distro packages...12:14
jrosserversions not controlled12:14
jrosserand mixed distro / pip stuff12:14
jrossereven ubuntu is making a mixed install right now i think12:14
jrosserbut somehow thats working12:14
noonedeadpunkwe do mixed install as we don't (and can't) controll all of the requirements....12:14
noonedeadpunkwe can probably install tox inside virtualenv12:15
*** nicolasbock has joined #openstack-ansible12:15
noonedeadpunkso that it installed virtualenv inside virtualenv and didin't ruin our distro installed one...12:15
noonedeadpunkubuntu has virtualenv 16+ from packages, so it should just say ok, as requirement is satisfied...12:16
jrosseri was thinking something odd going on with that too https://zuul.opendev.org/t/openstack/build/1acb9bf5a43743feb5470bb68120da43/log/job-output.txt#310712:16
jrosserwhy /usr/local/lib......12:17
noonedeadpunkactually yes - it should be /usr/lib....12:18
jrosserso that smells like stuff left over in CI images12:18
jrosserbut also python2 :(12:18
jrosserso this is a very similar problem to the zuul CI images12:21
noonedeadpunkyes, but now it's we who are installing tox12:22
jrosserpip uninstall virtualenv; yum remove python-virtualenv; yum install python-virtualenv12:22
jrosser^ then virtualenv is working again12:22
noonedeadpunkI think that 20.0.7 is working as well, but it changes behaviour...12:23
noonedeadpunkor it's not...12:23
jrosserhuh it's mixed up stuffs i think12:24
jrosseryum remove python-virtualenv; pip install virtualenv12:24
jrosser^ now i got virtualenv 20.0.7 and it's working12:25
noonedeadpunkoh....12:25
jrosserand that gets installed by openstack_hosts https://zuul.opendev.org/t/openstack/build/de0f9a117fbc44b3855213c2ab552640/log/job-output.txt#410712:28
jrosserwhich is legitimate for actual deployments12:28
noonedeadpunkyes, ok, that's good point12:29
noonedeadpunkso in design we should follow what's the closest to the actual deployments12:29
jrosseryes, agree12:29
jrosserand the trouble is that tox wants a newer virtualenv than the distro i think?12:29
noonedeadpunkhowever, if ppl will just install tox on some host - it will be failing venv_build12:30
noonedeadpunkas tox will install more fresh virtualenv from pip...12:30
noonedeadpunkyes, but it's only issue for centos 712:30
noonedeadpunkas centos is ... stable?...12:31
noonedeadpunkso I'd say to workaround our tox installation for functional tests12:31
noonedeadpunkor we should return to virtualenv installation from pip, and use the version bumped in u-c12:32
noonedeadpunkbut I kinda believe that distro package usage was selected not without the reason12:33
*** aedc has quit IRC12:34
*** aedc has joined #openstack-ansible12:35
jrosserthe integrated repo went to distro only for these things for very good reason12:35
jrosserbut i think it's been happy accident that the functional tests have kept working12:35
noonedeadpunkso maybe let's add version control for tox here https://opendev.org/openstack/openstack-ansible-tests/src/branch/master/run_tests_common.sh#L85 ?12:37
jrosseryeah, perhaps we can do that selectively?12:37
noonedeadpunk++12:37
jrosserpull in a distro specific constraints file12:37
noonedeadpunkmaybe just do define var in some switch/case?12:38
jrosseractually yes we already have ${ID} which could drive that12:39
noonedeadpunkyeah12:39
jrosserbut it would need to be centos-7 only, not 812:39
noonedeadpunkand VERSION12:39
noonedeadpunkso just add if statement for centos/rhel12:40
jrosseri'm just going to get lunch but yes i'll try to make a patch for that12:41
jrosserthanks for talking through it12:41
noonedeadpunksure thing! if you wish I can work on that12:42
*** fghaas has joined #openstack-ansible12:45
*** fghaas has quit IRC12:47
*** fghaas has joined #openstack-ansible12:48
*** DanyC has quit IRC12:49
*** aedc has quit IRC12:50
*** DanyC has joined #openstack-ansible12:50
*** aedc has joined #openstack-ansible12:51
*** DanyC has quit IRC12:55
*** rpittau|bbl is now known as rpittau13:04
*** alvinstarr has joined #openstack-ansible13:11
*** DanyC has joined #openstack-ansible13:17
*** udesale_ has joined #openstack-ansible13:17
*** udesale has quit IRC13:18
*** udesale_ has quit IRC13:24
*** udesale has joined #openstack-ansible13:24
openstackgerritEmilien Macchi proposed openstack/openstack-ansible-os_tempest master: post_install: add missing onditional to workspace task  https://review.opendev.org/71089313:29
*** udesale has quit IRC13:31
*** udesale has joined #openstack-ansible13:31
openstackgerritMerged openstack/openstack-ansible master: Add missing horizon dashboard variables  https://review.opendev.org/71077313:32
*** ansmith has joined #openstack-ansible13:35
*** cshen has quit IRC13:35
*** joshualyle has joined #openstack-ansible13:42
*** joshualyle has quit IRC13:44
strattaojamesdenton, I just wanted to say thanks for your help. You pointed me in the right direction and was able to overcome our OVS related issues. Then was able to discover that we needed some neutron client tunings for connecting to rabbit. With those overrides in place, our region seems to be stable and happy. Thanks again for your help!13:46
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-tests master: Pin version of tox used for Centos-7 tests  https://review.opendev.org/71101513:52
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-tests master: Pin version of tox used for Centos-7 tests  https://review.opendev.org/71101513:53
jamesdentonsweet! mind sharing what your tunings were?13:53
*** udesale has quit IRC13:58
*** cshen has joined #openstack-ansible13:58
*** udesale has joined #openstack-ansible13:58
*** zigo has quit IRC13:58
*** pcaruana has quit IRC14:09
openstackgerritErik Berg proposed openstack/openstack-ansible stable/rocky: How to upgrade from xenial to bionic on rocky  https://review.opendev.org/71080314:16
strattaosure thing! Here are the overrides we set in user_variables.yml: http://paste.openstack.org/show/790251/14:27
*** zigo has joined #openstack-ansible14:31
*** spatel has joined #openstack-ansible14:36
openstackgerritErik Berg proposed openstack/openstack-ansible stable/rocky: How to upgrade from xenial to bionic on rocky  https://review.opendev.org/71080314:38
*** fghaas has left #openstack-ansible14:39
*** spatel has quit IRC14:40
openstackgerritErik Berg proposed openstack/openstack-ansible stable/rocky: pin ceph packages to release o=Ubuntu on rocky/bionic  https://review.opendev.org/70842015:02
*** spatel has joined #openstack-ansible15:05
*** KeithMnemonic has quit IRC15:16
*** chandankumar is now known as raukadah15:18
*** DanyC has quit IRC15:21
*** DanyC has joined #openstack-ansible15:31
openstackgerritEmilien Macchi proposed openstack/openstack-ansible-os_tempest master: post_install: add missing onditional to workspace task  https://review.opendev.org/71089315:32
openstackgerritEmilien Macchi proposed openstack/openstack-ansible-os_tempest master: post_install: add missing onditional to workspace task  https://review.opendev.org/71089315:33
EmilienMcloudnull: ^15:33
*** udesale_ has joined #openstack-ansible15:35
*** udesale has quit IRC15:37
*** dave-mccowan has joined #openstack-ansible15:41
*** dave-mccowan has quit IRC15:46
*** spatel has quit IRC15:51
spotzEmilienM: You've got the -1 do you want us to hold off on reviewing?15:55
EmilienMspotz: probably for now15:55
EmilienMuntil i remove -w15:55
EmilienMthanks15:55
spotzNP:)15:55
*** DanyC has quit IRC15:58
noonedeadpunk#startmeeting openstack_ansible_meeting16:00
openstackMeeting started Tue Mar  3 16:00:54 2020 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
noonedeadpunk#topic office hours16:00
*** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)"16:00
openstackThe meeting name has been set to 'openstack_ansible_meeting'16:01
*** openstack changes topic to "office hours (Meeting topic: openstack_ansible_meeting)"16:01
*** DanyC has joined #openstack-ansible16:03
*** fghaas has joined #openstack-ansible16:05
*** DanyC has quit IRC16:18
*** DanyC has joined #openstack-ansible16:21
*** cshen has quit IRC16:27
*** gyee has joined #openstack-ansible16:27
*** udesale_ has quit IRC16:29
*** waverider has quit IRC16:30
noonedeadpunkok, so from news is that Rocky has almost entered EM for us https://review.opendev.org/#/c/709875/16:31
noonedeadpunkthe last release fixing gates has been merged16:32
noonedeadpunkhowever there was an issue there https://review.opendev.org/#/c/710781/16:32
noonedeadpunkbut not sure if we should block EM as rocky shouldn't care too much about py3 for redhat....16:33
*** macz_ has joined #openstack-ansible16:39
jrosseri don't think thats too much of an issue16:43
*** ivve has quit IRC16:44
*** macz_ has quit IRC16:44
*** macz_ has joined #openstack-ansible16:47
*** fghaas has quit IRC16:59
ebbexnoonedeadpunk, jrosser: How is the "git cache" supposed to work? https://review.opendev.org/#/c/708115/16:59
ebbexTo me, it looks like it's checking for a path that maybe should get bind-mounted into a container under zuul, but it's contents are probably empty?17:00
noonedeadpunk#endmeeting17:06
*** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://bit.ly/2SAcGAn"17:06
openstackMeeting ended Tue Mar  3 17:06:03 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:06
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2020/openstack_ansible_meeting.2020-03-03-16.00.html17:06
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2020/openstack_ansible_meeting.2020-03-03-16.00.txt17:06
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2020/openstack_ansible_meeting.2020-03-03-16.00.log.html17:06
noonedeadpunkebbex: that's pretty tough question tbh... as I can hardly recall how it was working...17:06
*** rpittau is now known as rpittau|afk17:10
*** pcaruana has joined #openstack-ansible17:34
*** evrardjp has quit IRC17:35
*** evrardjp has joined #openstack-ansible17:35
jrosserebbex: it might be /opt/git/opendev.org/openstack/<repo_name> now17:45
*** DanyC has quit IRC18:07
*** DanyC has joined #openstack-ansible18:08
*** trikorasolns has quit IRC18:11
gshippeywould be super helpful for me to get this merged https://review.opendev.org/#/c/710758/ 👼18:14
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-tests master: Pin version of tox used for Centos-7 tests  https://review.opendev.org/71101518:16
openstackgerritJonathan Rosser proposed openstack/ansible-role-python_venv_build master: Verify version number for virtualenv>=20.0.0  https://review.opendev.org/70748018:19
*** DanyC has quit IRC18:24
*** DanyC has joined #openstack-ansible18:30
*** ansmith_ has joined #openstack-ansible18:31
*** ansmith has quit IRC18:34
noonedeadpunkjrosser: gshippey how appropriate is backporting such new functionality at the end of the next cycle?18:43
*** xakaitetoia has quit IRC18:44
jrosseroh errr yes18:44
jrosserthat’s not right18:44
noonedeadpunkkinda I'd love to see it, but yeah...18:46
jrossercould you take a look at https://review.opendev.org/#/c/710758/18:48
jrosserhttps://review.opendev.org/707480 seems to be passing centos 7 with a depends-on18:48
jrosseroh I mean look at https://review.opendev.org/71101518:52
openstackgerritJonathan Rosser proposed openstack/ansible-role-python_venv_build master: [ussuri][goal] Updates for python 2.7 drop  https://review.opendev.org/71059418:59
*** andrewbonney has quit IRC19:09
strattaojamesdenton FYI those overrides came from here: https://docs.mirantis.com/mcp/q4-18/mcp-deployment-guide/advanced-config/tune-rabbitmq-perf.html19:23
jamesdentonahh cool, thanks19:23
*** pcaruana has quit IRC19:33
*** fghaas has joined #openstack-ansible19:47
*** ansmith has joined #openstack-ansible20:01
*** ansmith_ has quit IRC20:03
*** aedc has quit IRC20:11
*** aedc has joined #openstack-ansible20:11
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-nspawn_container_create master: Add vars file for debian buster  https://review.opendev.org/71110420:12
*** tosky has quit IRC20:22
*** trident has quit IRC20:23
*** trident has joined #openstack-ansible20:26
*** aedc_ has joined #openstack-ansible20:27
*** aedc has quit IRC20:27
*** fghaas has quit IRC20:29
*** DanyC has quit IRC20:44
*** DanyC has joined #openstack-ansible20:44
*** DanyC has quit IRC20:49
*** nicolasbock has quit IRC20:58
*** ansmith_ has joined #openstack-ansible20:59
*** ansmith has quit IRC21:02
*** DanyC has joined #openstack-ansible21:04
*** joshualyle has joined #openstack-ansible21:16
*** joshualyle has quit IRC21:20
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-tests master: Do not compress log files before uploading  https://review.opendev.org/71111021:27
*** ansmith_ has quit IRC21:32
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-nspawn_hosts master: Fix debian buster variables  https://review.opendev.org/71111221:37
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-nspawn_container_create master: Add vars file for debian buster  https://review.opendev.org/71110421:37
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-nspawn_container_create master: [ussuri][goal] Updates for python 2.7 drop  https://review.opendev.org/71061321:38
*** owalsh has quit IRC22:03
*** owalsh has joined #openstack-ansible22:03
*** ansmith_ has joined #openstack-ansible22:10
*** jbadiapa has quit IRC22:12
openstackgerritEmilien Macchi proposed openstack/openstack-ansible-os_tempest master: post_install: add missing onditional to workspace task  https://review.opendev.org/71089323:05
*** owalsh_ has joined #openstack-ansible23:25
*** owalsh has quit IRC23:28

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!