Tuesday, 2019-01-29

*** wolverineav has joined #openstack-infra00:02
*** eernst has quit IRC00:03
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add prepare-workspace-openshift role  https://review.openstack.org/63140200:16
*** sdake has quit IRC00:23
*** sdake has joined #openstack-infra00:24
openstackgerritClark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant  https://review.openstack.org/63358600:25
*** hongbin has joined #openstack-infra00:27
pabelangerclarkb: ianw_pto: just confirmed fedora-29, non network manager still work. But haven't looked more into current failures00:28
*** kei-ichi has joined #openstack-infra00:29
clarkbcool I'm still trying to sort out the last issues on that puppet change for running an opendev zuul web frontend00:33
clarkbI'm oh so very close but now running into the specific behaviors of puppet hashes in erb templates when accessed via ruby00:33
*** wolverineav has quit IRC00:34
*** wolverineav has joined #openstack-infra00:35
*** jcoufal has quit IRC00:36
*** edmondsw has quit IRC00:38
*** trown is now known as trown|outtypewww00:39
*** wolverineav has quit IRC00:39
pabelangerokay, fedora-29 running on limestone current00:40
pabelangerI haven't seen one of ovh today00:40
pabelangerHmm, still having issues accessing github.com from limestone00:45
*** gyee has quit IRC00:51
*** wolverineav has joined #openstack-infra00:52
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Fall back to legacy networking for fedora-29  https://review.openstack.org/63363100:52
pabelangerclarkb: ^should allow fedora-29 on all providers, until ianw_pto is back to look more into network manager.  We in ansible-network are still using legacy network and images still boot properly on vexxhost / limestone00:54
*** smarcet has quit IRC00:54
*** smarcet has joined #openstack-infra00:55
*** wolverineav has quit IRC00:56
*** armax has quit IRC00:58
*** jamesmcarthur has joined #openstack-infra01:04
*** jamesmcarthur has quit IRC01:09
*** yamamoto has joined #openstack-infra01:09
*** markvoelker has joined #openstack-infra01:10
*** rlandy is now known as rlandy|bbl01:15
*** markvoelker has quit IRC01:15
*** hongbin has quit IRC01:18
*** hongbin has joined #openstack-infra01:20
*** rh-jelabarre has quit IRC01:24
*** sdake has quit IRC01:28
*** dulek has quit IRC01:28
*** bhavikdbavishi has joined #openstack-infra01:33
*** ruffian_sheep has joined #openstack-infra01:33
*** tosky has quit IRC01:34
*** armax has joined #openstack-infra01:36
*** wolverineav has joined #openstack-infra01:37
*** bhavikdbavishi has quit IRC01:51
*** bhavikdbavishi has joined #openstack-infra01:51
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: model: remove unused addImpliedBranchMatcher procedure  https://review.openstack.org/63364301:53
*** bobh has quit IRC01:53
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: model: remove unused job's BranchMatcher procedures  https://review.openstack.org/63364301:56
*** ddurst has joined #openstack-infra01:58
*** bobh has joined #openstack-infra02:00
*** yamamoto has quit IRC02:03
*** sdake has joined #openstack-infra02:31
*** sdake has quit IRC02:37
*** bhavikdbavishi has quit IRC02:37
*** whoami-rajat has joined #openstack-infra02:37
*** sdake has joined #openstack-infra02:37
*** yamamoto has joined #openstack-infra02:41
*** yamamoto has quit IRC02:46
*** psachin has joined #openstack-infra02:52
*** ykarel|away has joined #openstack-infra02:55
*** armstrong has quit IRC02:57
*** sdake has quit IRC02:58
*** sdake has joined #openstack-infra02:59
*** bgmccollum has quit IRC03:00
*** bgmccollum has joined #openstack-infra03:02
*** rlandy|bbl is now known as rlandy03:04
*** rlandy has quit IRC03:04
*** apetrich has quit IRC03:15
*** jamesmcarthur has joined #openstack-infra03:23
*** jamesmcarthur has quit IRC03:24
*** jamesmcarthur has joined #openstack-infra03:24
*** yamamoto has joined #openstack-infra03:25
*** sdake has quit IRC03:26
*** sdake has joined #openstack-infra03:29
*** yamamoto has quit IRC03:34
*** sdake has quit IRC03:42
*** ykarel|away is now known as ykarel03:42
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: config: add playbooks to job.toDict()  https://review.openstack.org/62134303:48
*** anteaya has quit IRC03:49
*** ykarel is now known as ykarel|afk03:50
*** bhavikdbavishi has joined #openstack-infra03:52
*** udesale has joined #openstack-infra04:02
*** yamamoto has joined #openstack-infra04:06
*** ykarel|afk is now known as ykarel04:06
*** bobh has quit IRC04:11
*** diablo_rojo has quit IRC04:12
*** jamesmcarthur has quit IRC04:15
*** jamesmcarthur has joined #openstack-infra04:16
*** bobh has joined #openstack-infra04:18
*** jamesmcarthur has quit IRC04:21
*** chkumar|out is now known as chandankumar04:22
*** bobh has quit IRC04:27
*** eernst has joined #openstack-infra04:32
*** bhavikdbavishi has quit IRC04:38
*** bhavikdbavishi has joined #openstack-infra04:39
*** sdake has joined #openstack-infra04:49
*** spsurya has joined #openstack-infra05:05
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: scheduler: add job's variants to the rpc job_list method  https://review.openstack.org/57347305:16
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: switch jobs list to a tree view  https://review.openstack.org/63343705:16
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add jobs list filter  https://review.openstack.org/63365205:16
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: scheduler: add job's tags to the rpc job_list method  https://review.openstack.org/63365305:16
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add tags to jobs list  https://review.openstack.org/63365405:16
*** auristor has quit IRC05:20
*** auristor has joined #openstack-infra05:20
*** ruffian_sheep has quit IRC05:21
*** bhavikdbavishi has quit IRC05:27
*** udesale has quit IRC05:29
*** jtomasek has joined #openstack-infra05:36
*** eernst has quit IRC05:38
*** sdake has quit IRC05:41
*** sdake has joined #openstack-infra05:49
*** udesale has joined #openstack-infra05:51
*** hwoarang has quit IRC05:52
*** bhavikdbavishi has joined #openstack-infra05:53
*** hwoarang has joined #openstack-infra05:55
*** udesale has quit IRC05:59
*** udesale has joined #openstack-infra06:00
*** jtomasek has quit IRC06:02
*** ykarel has quit IRC06:09
*** edmondsw has joined #openstack-infra06:19
*** markvoelker has joined #openstack-infra06:20
*** hongbin has quit IRC06:21
*** ykarel has joined #openstack-infra06:21
*** jamesmcarthur has joined #openstack-infra06:24
*** markvoelker has quit IRC06:24
*** noama has joined #openstack-infra06:24
*** mandre is now known as mandre|off06:25
*** jamesmcarthur has quit IRC06:28
tobiashinfra-root: are there dns problems in some providers? I think I remember several job failures due to 'Temporary failure in name resolution' during the last days. Example log is here: http://logs.openstack.org/43/633643/2/check/tox-py36/e3d3969/job-output.txt.gz#_2019-01-29_02_04_11_79985806:29
tobiashthis one happened in rax-dfw06:33
*** ramishra has joined #openstack-infra06:33
*** strigazi has quit IRC06:42
*** strigazi has joined #openstack-infra06:43
*** ccamacho has quit IRC06:44
*** kjackal has joined #openstack-infra06:50
AJaegercorvus: https://review.openstack.org/#/c/632785 is not handled by zuul - I think we merged 633591 too soon, should have waited for 785 to merge first - shouldn't it? Or how do you want to force-merge?07:01
*** quiquell|off is now known as quiquell07:01
*** wolverineav has quit IRC07:03
*** wolverineav has joined #openstack-infra07:04
*** wolverineav has quit IRC07:09
*** udesale has quit IRC07:10
*** udesale has joined #openstack-infra07:18
*** ccamacho has joined #openstack-infra07:20
*** udesale has quit IRC07:22
*** udesale has joined #openstack-infra07:26
*** yamamoto has quit IRC07:28
*** yamamoto has joined #openstack-infra07:32
*** pgaxatte has joined #openstack-infra07:33
*** kjackal has quit IRC07:34
*** wolverineav has joined #openstack-infra07:34
*** kjackal has joined #openstack-infra07:37
*** wolverineav has quit IRC07:38
*** slaweq has joined #openstack-infra07:43
*** aojea has joined #openstack-infra07:46
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: config: add tenant.toDict() method and REST endpoint  https://review.openstack.org/62134407:48
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add config page  https://review.openstack.org/63366707:48
*** e0ne has joined #openstack-infra07:59
*** shardy has joined #openstack-infra08:02
*** rpittau has joined #openstack-infra08:04
*** apetrich has joined #openstack-infra08:09
*** sdake has quit IRC08:11
*** ginopc has joined #openstack-infra08:11
*** shardy has quit IRC08:11
*** shardy has joined #openstack-infra08:12
*** dulek has joined #openstack-infra08:15
*** wolverineav has joined #openstack-infra08:15
*** kjackal has quit IRC08:16
*** kjackal_v2 has joined #openstack-infra08:16
*** kopecmartin|off is now known as kopecmartin08:19
*** wolverineav has quit IRC08:19
*** markvoelker has joined #openstack-infra08:20
*** gfidente has joined #openstack-infra08:21
*** gfidente has quit IRC08:25
*** ginopc has quit IRC08:26
*** bnemec has joined #openstack-infra08:29
*** jpena|off is now known as jpena08:33
*** ramishra_ has joined #openstack-infra08:34
*** ramishra has quit IRC08:35
*** electrofelix has joined #openstack-infra08:35
*** yamamoto has quit IRC08:35
*** yamamoto has joined #openstack-infra08:36
*** peschk_l has quit IRC08:36
*** janki has joined #openstack-infra08:36
chandankumarHello08:39
chandankumarcan we hold the node openstack-ansible-functional-centos-7 node next time we retrigger it we need to debug a tempest failure on centos7 for os_tempest side/08:40
*** tosky has joined #openstack-infra08:42
fricklerchandankumar: which project/patch do you need this for?08:45
chandankumarfrickler: https://review.openstack.org/#/c/63320808:46
*** tkajinam has quit IRC08:48
*** pcaruana has joined #openstack-infra08:51
*** ykarel is now known as ykarel|lunch08:52
elodhi, infra team! I've found a small issue last week around "editable git installed packages" and created a minor fix for it: https://review.openstack.org/#/c/633049/08:53
elodcould someone review it? :)08:53
*** priteau has joined #openstack-infra08:54
*** markvoelker has quit IRC08:54
tobias-urdininfra-root: ping on https://review.openstack.org/#/c/632699/08:54
elodwithout it there are some (stable) job failures (and some cases it doesn't cause real problem however does not install deps for editable packages)08:54
fricklerchandankumar: ok, hold set up, you'll get a node once that job fails. which ssh key shall we use to give you access?08:56
AJaegertobias-urdin: could you ask pabelanger to review 632699, please?08:58
fricklerelod: that fix looks to me like it will need some extended testing, would it be possible for you to pin pip<19 as a workaround?09:00
*** jpich has joined #openstack-infra09:01
tobias-urdinAJaeger: ack09:02
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: config: add playbooks to job.toDict()  https://review.openstack.org/62134309:04
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: config: add tenant.toDict() method and REST endpoint  https://review.openstack.org/62134409:04
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add config page  https://review.openstack.org/63366709:04
*** jtomasek has joined #openstack-infra09:06
*** gfidente has joined #openstack-infra09:06
chandankumarfrickler: sending in pm09:09
elodfrickler AJaeger : i've tested only the function itself, but i'll do a depends on test now, thanks for the review!09:13
elodfrickler: i can create pip<19 pin workaround if you think that is needed09:14
*** soumplis has quit IRC09:19
*** ykarel|lunch is now known as ykarel09:20
*** panda is now known as panda|numb09:28
*** dtantsur|afk is now known as dtantsur09:35
*** derekh has joined #openstack-infra09:36
*** luizbag has joined #openstack-infra09:40
*** ginopc has joined #openstack-infra09:44
*** markvoelker has joined #openstack-infra09:51
*** kashyap has joined #openstack-infra09:53
kashyapHi folks, this patch _keeps_ getting some kind of time out for the last 10 days: https://review.openstack.org/#/c/630980/ (libvirt: A few miscellaneous items related to "native TLS")09:53
kashyapAnd the failures are not related to my patch at all.09:53
kashyapI was pointed out this: http://logs.openstack.org/80/630980/7/gate/tempest-full/5b6ebba/ara-report/09:54
kashyap... where the 'run-devstack' took 30 minutes.09:54
kashyapAny hints on how to put that patch out of its misery?09:54
*** baysao has joined #openstack-infra10:01
*** zaneb has quit IRC10:16
openstackgerritMerged openstack-infra/irc-meetings master: Change of agenda url for weekly meetings  https://review.openstack.org/63346610:17
*** yboaron has joined #openstack-infra10:20
ttxAJaeger: any chance you could review https://review.openstack.org/#/c/631032/ ? I need that repo up to propose changes to it10:21
ttxjust needs the second +2a10:21
*** ramishra_ has quit IRC10:23
ttxor frickler ^10:23
*** markvoelker has quit IRC10:24
*** adriancz has joined #openstack-infra10:24
*** rf0lc0 has joined #openstack-infra10:27
*** rfolco has quit IRC10:27
fricklerttx: hmm, since this is the first repo in a new namespace, I'd like to wait for corvus' vote on that one10:28
*** ramishra has joined #openstack-infra10:30
ttxfrickler: sure. He did comment no a previous patchset but did not vote on the final one.10:30
openstackgerritIvan Kolodyazhny proposed openstack-infra/project-config master: Stop project gating before retirement  https://review.openstack.org/63121810:30
openstackgerritIvan Kolodyazhny proposed openstack-infra/project-config master: Stop project gating before retirement  https://review.openstack.org/63121810:31
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add roles usage information to the build page  https://review.openstack.org/63369710:43
*** bhavikdbavishi has quit IRC10:45
*** baysao has quit IRC10:49
fricklerchandankumar: slaweq: root@158.69.65.144 is your held node. please ping infra-root once you are finished with debugging10:49
*** rf0lc0 has quit IRC10:53
chandankumarfrickler: sure10:53
slaweqfrickler: thx10:54
fricklerttx: clarkb: corvus: added some comment regarding uniqueness of repo names vs. gerrit group names to https://review.openstack.org/#/c/631032/11:00
*** ssbarnea|bkp2 has quit IRC11:01
*** ssbarnea|rover has joined #openstack-infra11:02
*** janki has quit IRC11:02
*** rfolco has joined #openstack-infra11:03
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add roles usage information to the build page  https://review.openstack.org/63369711:04
*** udesale has quit IRC11:09
*** yamamoto has quit IRC11:14
chandankumarjrosser: slaweq is already there in the box11:20
chandankumarslaweq: jrosser can help on osa11:20
*** markvoelker has joined #openstack-infra11:21
*** bhavikdbavishi has joined #openstack-infra11:39
chandankumarfrickler: we are done with debugging on the nodes11:51
chandankumarfrickler: you can remove the hold11:51
chandankumarfrickler: thanks for the help :-)11:51
*** markvoelker has quit IRC11:53
*** yamamoto has joined #openstack-infra11:54
*** yamamoto has quit IRC12:04
*** kukacz has quit IRC12:04
*** kukacz has joined #openstack-infra12:04
*** lucasagomes is now known as lucas-hungry12:07
*** fdegir is now known as fdegir_12:10
*** agopi|off has joined #openstack-infra12:12
*** jpena is now known as jpena|lunch12:15
*** yamamoto has joined #openstack-infra12:23
*** jamesmcarthur has joined #openstack-infra12:24
*** jamesmcarthur has quit IRC12:29
*** fdegir has joined #openstack-infra12:31
*** fdegir_ has left #openstack-infra12:32
elodAJaeger frickler : the depends-on test finished and I've commented the log details12:39
*** pcaruana has quit IRC12:40
elodI've tested on vmware-nsx stable/ocata, is it enough or should I do some other test on other project and branch?12:40
*** yboaron has quit IRC12:42
*** markmcd has quit IRC12:43
*** ginopc has quit IRC12:44
*** quiquell is now known as quiquell|brb12:48
*** markmcd has joined #openstack-infra12:48
*** pcaruana has joined #openstack-infra12:50
*** kaiokmo has joined #openstack-infra12:50
*** markvoelker has joined #openstack-infra12:50
*** lucas-hungry is now known as lucasagomes12:51
*** apetrich has quit IRC12:52
*** apetrich has joined #openstack-infra12:55
*** agopi|off has quit IRC12:55
*** priteau has quit IRC13:04
*** priteau has joined #openstack-infra13:04
*** sean-k-mooney has joined #openstack-infra13:04
sean-k-mooneyim not sure if this is the right place to ask but im wondering if we encountered this error http://paste.openstack.org/show/744167/ upstream and how it was corrected in the devstack post playbook13:09
sean-k-mooneyspecifcally with a ubuntu 18.04 guest https://github.com/openstack-dev/devstack/blob/94ca9f6756e7b677b1ee3fd2e32b555447e950dd/roles/fetch-devstack-log-dir/tasks/main.yaml is failing to sync the logs13:09
*** xek has joined #openstack-infra13:11
sean-k-mooneyi had considerd submitting a patch to devstack to disable preserving permissions/ownership but i was wonderign if infra had another solution13:11
*** priteau has quit IRC13:12
sean-k-mooneythis is all in relation to seting up a third party ci. at this point my devstack job is reusign the upstream devstack job and stacking but this final role in the post playbook is failing an causeing the job to fail as a result13:13
*** trown|outtypewww is now known as trown13:13
*** e0ne has quit IRC13:13
*** markvoelker has quit IRC13:20
*** sean-k-mooney has quit IRC13:20
*** sean-k-mooney has joined #openstack-infra13:21
*** quiquell|brb is now known as quiquell13:26
*** agopi|off has joined #openstack-infra13:26
*** agopi|off is now known as agopi13:26
*** kgiusti has joined #openstack-infra13:27
*** mriedem has joined #openstack-infra13:28
*** e0ne has joined #openstack-infra13:29
*** rlandy has joined #openstack-infra13:30
*** jpena|lunch is now known as jpena13:36
openstackgerritMatt Riedemann proposed openstack-infra/project-config master: Fix the ceph grafana dashboard  https://review.openstack.org/63373513:44
*** jcoufal has joined #openstack-infra13:44
*** rh-jelabarre has joined #openstack-infra13:47
*** bhavikdbavishi has quit IRC13:48
*** sthussey has joined #openstack-infra13:54
*** panda|numb is now known as panda13:56
*** psachin has quit IRC13:56
mriedemi noticed that https://bugs.launchpad.net/cinder/+bug/1810526 is predominantly on ovh nodes,13:58
openstackLaunchpad bug 1810526 in Cinder "volume delete fails due to ToozConnectionError in the gate" [Undecided,Confirmed]13:58
mriedemis there some trend we've seen about ovh nodes that might indicate why that could be failing so much, i.e. i/o issues that might cause etcd delays?13:58
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for cinder tooz connection error bug 1810526  https://review.openstack.org/63373813:59
*** yamamoto has quit IRC14:00
*** ykarel is now known as ykarel|away14:01
*** jcoufal has quit IRC14:07
openstackgerritMerged openstack-infra/elastic-recheck master: fix tox python3 overrides  https://review.openstack.org/60561814:09
*** yboaron has joined #openstack-infra14:14
*** efried has quit IRC14:15
*** efried has joined #openstack-infra14:15
*** jcoufal has joined #openstack-infra14:16
*** efried has quit IRC14:19
*** efried has joined #openstack-infra14:19
*** roman_g has joined #openstack-infra14:20
fricklermriedem: seems https://review.openstack.org/626885 tackles the same thing. maybe we should consider switching this on by default instead of only for kuryr14:22
*** yamamoto has joined #openstack-infra14:23
*** bobh has joined #openstack-infra14:23
*** udesale has joined #openstack-infra14:24
*** ykarel|away has quit IRC14:25
mriedemhmm yeah14:26
mriedemthe other thing i was thinking about was just not using etcd as the cinder DLM backend on single host jobs since it's kind of pointless14:26
*** dkehn has joined #openstack-infra14:27
fungiinfra-root: i'm not really here, but i see rackspace fanatical support opened a ticket to let us know the host for afs01.ord.openstack.org has suffered some sort of critical failure so likely want to keep an eye on our afs infrastructure and check out that fileserver when it comes back online14:31
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for cinder tooz connection error bug 1810526  https://review.openstack.org/63373814:34
openstackbug 1810526 in Cinder "volume delete fails due to ToozConnectionError in the gate" [Undecided,Confirmed] https://launchpad.net/bugs/181052614:34
*** smarcet has quit IRC14:34
*** smarcet has joined #openstack-infra14:36
*** dave-mccowan has joined #openstack-infra14:39
*** weshay has quit IRC14:43
*** dave-mccowan has quit IRC14:45
*** pcaruana has quit IRC14:45
*** eharney has joined #openstack-infra14:49
*** sdake has joined #openstack-infra14:49
*** bhavikdbavishi has joined #openstack-infra14:50
*** sdake has quit IRC14:51
*** roman_g has quit IRC14:51
*** e0ne has quit IRC14:51
*** pcaruana has joined #openstack-infra14:53
*** sdake has joined #openstack-infra14:54
*** e0ne has joined #openstack-infra14:54
*** roman_g has joined #openstack-infra14:58
smcginnismriedem: We could switch most jobs to use local filesystem locks. That's actually the default when you deploy system packages, so devstack kind of sets things up in a non-standardish way.15:01
mriedemsmcginnis: yeah i just haven't dug into that yet, i need to re-learn how to determine from devstack if it's going to be a multi-node job15:02
mriedemi don't want to have to set a flag in all zuul jobs (either single or multinode)15:03
*** sshnaidm is now known as sshnaidm|mtg15:04
*** smarcet has quit IRC15:05
*** smarcet has joined #openstack-infra15:08
openstackgerritMerged openstack-infra/project-config master: Add ansible roles for managing HSM software  https://review.openstack.org/63132615:09
mriedemsmcginnis: having said that, i'm not actively working on that one right now if you're bored15:12
mriedemhttp://status.openstack.org/elastic-recheck/#1813198 is the current killer15:12
*** yamamoto has quit IRC15:15
*** yamamoto has joined #openstack-infra15:16
*** yamamoto has quit IRC15:16
*** yamamoto has joined #openstack-infra15:17
mnaserinfra-root: i see a change that merged but showed this - infra-prod-zuul_reconfigure finger://ze02.openstack.org/b0db56e18f104bfa8f08350254d8405a : RETRY_LIMIT in 23s15:18
*** udesale has quit IRC15:18
pabelangermnaser: expected, I blieve the job is missing known_hosts task15:19
clarkbmnaser: that is noise. Should probably remove the job if I cant get votes on the fix.15:19
pabelangerI believe*15:19
mnaserpabelanger, clarkb: ok cool, thanks, just wanted to make sure zuul didn't fail reconfig or something15:20
clarkbpabelanger: the issue is we cant ssh as root. So I have changes up to ssh as a zuul user15:20
pabelangerlooking15:20
AJaegerpabelanger could you review 632699, please?15:20
*** wolverineav has joined #openstack-infra15:21
*** eharney has quit IRC15:21
pabelangerclarkb: left comment on 60492515:23
*** bobh has quit IRC15:23
*** yamamoto has quit IRC15:23
pabelangerAJaeger: Hmm, I for puppet modules we installed dependencies via bindep and build target15:23
pabelangerAJaeger: will lookup and reply15:24
*** eharney has joined #openstack-infra15:24
clarkbpabelanger: that is possible15:25
*** wolverineav has quit IRC15:25
pabelangerclarkb: I also proposed https://review.openstack.org/633631/ to roll back nm on fedora-29 nodes, since online limestone is booting them currently15:25
pabelangerclarkb: yah, https://github.com/ansible-network/windmill-config/blob/master/tests/playbooks/bastion.yaml is how I CD from zuul today15:26
clarkbpabelanger: I thought fedora 29 required nm? or is it rhel8 that makes the switch required?15:27
pabelangerclarkb: not required, I'm still using legacy script for ansible-network, you just need to install them manually. As I say that, let me check if our elements will do that15:28
*** smarcet has quit IRC15:29
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Fall back to legacy networking for fedora-29  https://review.openstack.org/63363115:32
*** sdake has quit IRC15:32
*** kjackal_v2 has quit IRC15:32
pabelangerclarkb: that's the setup we use in ansible-network^, but can understand if don't want to do it.15:32
*** kjackal has joined #openstack-infra15:32
pabelangerI know ianw_pto is on PTO, so unsure when we're get around to looking at the actualy issue15:33
clarkbIt doesnt fix the broken udev and dbus which may still cause jobs to fail15:33
*** sshnaidm|mtg is now known as sshnaidm15:33
pabelangeryah, this is mostly just to get fedora-29 working on all providers again. Right now, jobs depending on it are having high failure rate. Only running in limestone, and there seems to be dns issue going on there too15:35
*** sdake has joined #openstack-infra15:35
clarkbdns problems can be related to dbus if systemd's resolveris in the mix15:36
pabelangerhttp://logs.openstack.org/44/633544/12/check/ansible-role-borgbackup-src-fedora-latest/f77d065/ara-report/result/321da77d-b4cd-4e4d-83c5-1cd1f45e28ad/15:36
pabelangeryah15:36
clarkbmy point is fedora 29 is broken. Our results reflect that. The fix is to fix dbus and nm15:36
pabelangerI won't have much time to do that this week sadly15:37
*** ykarel|away has joined #openstack-infra15:38
*** ykarel|away is now known as ykarel15:39
*** bhavikdbavishi has quit IRC15:43
*** ccamacho has quit IRC15:45
*** ccamacho has joined #openstack-infra15:46
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for evacuate vif plugging timeout bug 1813789  https://review.openstack.org/63376615:49
corvusAJaeger: yes, let's force-merge 785; i think that is the safest thing to do (so we are extra careful not to mess up the openstack tenant)15:49
openstackbug 1813789 in OpenStack Compute (nova) "Evacuate test intermittently fails with network-vif-plugged timeout exception" [Medium,Confirmed] https://launchpad.net/bugs/181378915:49
*** weshay has joined #openstack-infra15:49
*** openstackgerrit has quit IRC15:51
clarkbpabelanger: thinking about it more limestone likely works asit is an ipv6 cloud and the RAs configure the interface automatically15:51
AJaegercorvus: works for me - please go ahead with force merging...15:51
corvusfrickler made a good point on 631032, or really, 2 good points: should we relax the uniqueness restriction on repo names (eg, is it okay now to have kata/foo and openstack/foo).  and how should we name gerrit groups?15:54
corvuswe have some jobs in openstack that assume unique repo names.  i think as long as we maintain repo name uniqueness in the openstack zuul tenant, that should be okay.15:55
pabelangerclarkb: yah, believe so. I've send an email to ML asking for help from the community. For now, I'll have to flip jobs non-voting and hope I can get some spare cycles later this week to look more15:55
*** bhavikdbavishi has joined #openstack-infra15:55
corvus(eg, doc, tarball, all work on repo names).15:55
*** eernst has joined #openstack-infra15:56
corvuswe should avoid setting up a pypi publishing job in another tenant with the same credential.15:56
corvusotherwise starlingx/nova could overwrite openstack/nova15:56
corvusclarkb: ^15:57
clarkbcorvus credential uniqueness probably desireable for other reasons too15:57
clarkbIf we make the rule we must be unique within a tenant/namespace that shoyld be relatively easy to check and we can prefix group names if we want to avoid collisions15:58
*** kjackal has quit IRC16:03
*** kjackal has joined #openstack-infra16:03
ttxfrom a practical perspective, allowing for duplicate names would prevent intentional/unintentional landgrab16:09
ttxWe have lots of things like openstack/governance16:09
*** dkehn has quit IRC16:10
ttxMaybe uniqueness within a Zuul tenant is the right way16:10
ttxat least for now16:11
ttxcorvus: should I just rename my repo to kata-containers/kata-zuul-config and table that question (I kinda need that repo up asap)? Would that work for you?16:13
dmsimardIs Zuul not picking up new jobs right now ? I have two patches up that haven't triggered anything16:13
dmsimardHmm, there are new jobs starting.. I'll try a recheck16:14
corvusttx: i'm happy with /zuul-config, sounds like clarkb is too.  i'm pretty sure we want to be as hands-off as possible with opendev, so i'd leave it.16:14
ttxcorvus: alright! Just +2a at your convenience then16:15
dmsimardOh, I see this message in logs: 2019-01-29 16:12:16,842 DEBUG zuul.Pipeline.openstack.check: <QueueItem 0x7fd9c69599b0 for <Change 0x7fd9beeafa90 openstack/ara-web 633764,1> in check> is a failing item because ['is a non-live item with no items behind']16:15
*** quiquell is now known as quiquell|off16:16
*** jamesmcarthur has joined #openstack-infra16:16
corvusdmsimard: that's likely a red herring16:17
dmsimardokay, false alarm -- file matcher was wrong16:17
dmsimardsorry about the noise16:17
*** jamesmcarthur has quit IRC16:21
*** wolverineav has joined #openstack-infra16:21
redrobotHello infra friends!16:22
redrobotI have a couple of new repos that were created earlier today.  Wondering how I can get myself added to the XXX-core groups on gerrit for them?16:23
*** jamesmcarthur has joined #openstack-infra16:23
redrobothttps://review.openstack.org/#/admin/groups/2005,members16:23
redrobothttps://review.openstack.org/#/admin/groups/2006,members16:23
*** fdegir has quit IRC16:26
clarkbredrobot: I'll take a look in a few minutes16:27
redrobotAwesome, thanks clarkb16:27
*** bobh has joined #openstack-infra16:29
*** wolverineav has quit IRC16:30
clarkbredrobot: because they were added as official openstack-ansible deliverables I've added mnaser as the initial user. Mnaser can add additional group members (and new group members can manage from there too)16:31
*** e0ne has quit IRC16:31
redrobotclarkb, awesome, appreciate it.16:31
*** openstackgerrit has joined #openstack-infra16:32
openstackgerritMerged openstack-infra/project-config master: Add Kata tenant Zuul configuration repository  https://review.openstack.org/63103216:32
*** chandankumar is now known as chkumar|out16:35
clarkbinfra-root I got https://review.openstack.org/#/c/633586/ passing tests yesterday. This change adds support for running multiple vhosts in puppet-zuul so that we can run zuul.opendev.org on zuul.openstack.org16:38
clarkbthe changes there are backward compatilbe and if we like them we'll need a follow on to systemconfig to configure the new vhost stuff in puppet-zuul16:38
*** agopi is now known as agopi|lunch16:41
corvusclarkb: lgtm16:45
clarkbI guess I should recheck to get that last non voting job to go green. I believe it failed due to rubygem download failures16:45
*** sdake has quit IRC16:46
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for evacuate vif plugging timeout bug 1813789  https://review.openstack.org/63376616:47
openstackbug 1813789 in OpenStack Compute (nova) "Evacuate test intermittently fails with network-vif-plugged timeout exception" [Medium,Confirmed] https://launchpad.net/bugs/181378916:47
clarkbmriedem: according to the kuryr emails they found that fsync calls on ovh gra1 take >1s ~1.5% of the time which is far greater than the other clouds. They were working with ovh on the issue though16:50
*** hamzy has quit IRC16:56
corvusclarkb: any other zuul multi-tenant stuff i should be doing/reviewing?16:57
clarkbcorvus: I think that is the current set. I guess I should go ahead and draft the change that depends on ^16:57
clarkbI don't have ssl data yet but we can plug that in fairly easily once ready16:58
*** bhavikdbavishi has quit IRC16:58
corvusclarkb: have you started the ssl process?  i think last one i saw used dns, so it might be a bit slower, so good to get that going in the background16:58
*** noama has quit IRC16:59
clarkbcorvus: I have not16:59
openstackgerritMatthieu Huin proposed openstack-infra/zuul-jobs master: install-nodejs: add support for RPM-based OSes  https://review.openstack.org/63104916:59
clarkbI wasn't sure if we'd have a working lets encrypt setup though it is looking like no16:59
corvusthat seems to be the case17:00
*** hamzy has joined #openstack-infra17:00
*** pcaruana has quit IRC17:01
*** pgaxatte has quit IRC17:03
*** gyee has joined #openstack-infra17:03
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add zuul.opendev.org webserver  https://review.openstack.org/63378317:08
clarkbcorvus: ^ that is the consumption of the puppet-zuul side of things17:08
clarkbI can spin up a new cert after the infra meeting today17:10
clarkbits "only" $8 so may as well keep pushing on that17:11
*** hamzy has quit IRC17:11
clarkbbut then I think its just a matter of merging stuff and checking it works after17:11
clarkbI've been asked to start doing similar for ehterpad, but with an early am flight tomorrow I think I'm going to have to constrain what I start digging into. On the etherpad side of things I think we can simplify a bit and have etherpad.openstack.org redirect to etherpad.opendev.org since their behavior will be the same otherwise (unlike zuul with the multiple tenants)17:14
*** bhavikdbavishi has joined #openstack-infra17:16
*** sdake has joined #openstack-infra17:17
clarkbhrm though we'll still need https certs managed for all of the redirects os maybe not as simple as I first though17:17
corvuswould really be great to have le in place :)17:18
*** sdake has quit IRC17:18
clarkbya finding the balance between moving opendev wforward not blocking on things like that and actually paying that tech debt down currently difficult ;)17:18
*** ccamacho has quit IRC17:18
clarkbI do think the zuul side is worth pushing in particular though17:18
*** yamamoto has joined #openstack-infra17:21
*** rpittau has quit IRC17:21
*** rpittau has joined #openstack-infra17:21
*** anteaya has joined #openstack-infra17:22
*** yboaron has quit IRC17:24
*** yboaron has joined #openstack-infra17:24
*** kopecmartin is now known as kopecmartin|off17:24
*** ramishra has quit IRC17:25
*** yboaron has quit IRC17:26
*** trown is now known as trown|lunch17:26
*** yamamoto has quit IRC17:27
*** yboaron has joined #openstack-infra17:27
*** bnemec has quit IRC17:27
*** anteaya has quit IRC17:28
openstackgerritSorin Sbarnea proposed openstack-infra/git-review master: Allow user to control Java version used by tox  https://review.openstack.org/63066117:30
*** yboaron has quit IRC17:31
*** yboaron has joined #openstack-infra17:31
*** jpich has quit IRC17:32
*** yboaron has quit IRC17:35
*** rpittau has quit IRC17:35
openstackgerritSorin Sbarnea proposed openstack-infra/git-review master: Allow user to control Java version used by tox  https://review.openstack.org/63066117:36
*** ykarel is now known as ykarel|away17:41
*** sdake has joined #openstack-infra17:43
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add zuul.opendev.org webserver  https://review.openstack.org/63378317:45
*** agopi|lunch is now known as agopi17:46
clarkbis there a new release of pep8? seems like a lot of failures happening in openstack integrated gate due to pep8 failures17:46
*** e0ne has joined #openstack-infra17:46
clarkbyup pycodestyle 2.5.0 just released and that is what the jobs that fail are using17:47
clarkbprometheanfire: tonyb ^ fyi that may be something requirements team needs to look at17:47
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient master: Add a resource subcontroller for attachments  https://review.openstack.org/63361117:49
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient master: Display download links for files attached to stories  https://review.openstack.org/63361217:49
*** anteaya has joined #openstack-infra17:53
*** aojea has quit IRC17:58
*** derekh has quit IRC18:01
*** diablo_rojo has joined #openstack-infra18:01
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP Revert "Revert "Add a timeout for the image build""  https://review.openstack.org/63379218:02
prometheanfireclarkb: https://review.openstack.org/633783 ?18:02
clarkbprometheanfire: no pycodestyle 2.5.0 released a few hours ago and has broken pep8 jobs for at least neutron and keystone18:02
prometheanfirek18:02
clarkbprometheanfire: I've pinged them about possibly fixing their errors, but wasn't sure if requirements team had thoughts about maybe pinning?18:03
prometheanfireclarkb: that's controlled per project18:03
clarkbroger18:03
prometheanfirehttps://github.com/openstack/requirements/blob/master/blacklist.txt#L1418:03
clarkbah ok18:03
prometheanfiresince everyone likes to use a diferent versions for linting18:03
clarkbso then neutron and keystone should fix locally18:03
prometheanfirethey can just pin/whatever it18:04
prometheanfireya18:04
*** jamesmcarthur has quit IRC18:06
*** jamesmcarthur has joined #openstack-infra18:08
*** jtomasek has quit IRC18:10
*** sdake has quit IRC18:10
ssbarnea|roverclarkb: prometheanfire as you mentioned the new linter breaking things kind of issue. that is one of the reasons why I love running linters via pre-commit as it always locks them, you never get surprised by a new linter and updating all linter is as simple as running "pre-commit autoupdate".18:15
clarkbssbarnea|rover: the problem in this case is pycodestyle broke on the existing code18:15
*** jpena is now known as jpena|off18:15
diablo_rojofrickler, around?18:16
ssbarnea|roverclarkb: yeah, but it broke because the project was not pinning pycodestyle to specific version, right?18:16
clarkbssbarnea|rover: ya I think they must do <3.018:16
clarkbso its bound but not specific18:16
prometheanfirelikely18:16
ssbarnea|roveryep, that's the easy/fast fix18:17
prometheanfireiirc upper-constraints can only be called once, so combining it with another local linters-requirements list would be needed (or two pip install calls)18:17
*** kjackal has quit IRC18:17
corvuswasn't hacking supposed to pin pep8?  forget i asked.18:18
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient master: Make tags section more consistent  https://review.openstack.org/63361318:18
*** bhavikdbavishi has quit IRC18:20
clarkbhttp://git.openstack.org/cgit/openstack-dev/hacking/tree/requirements.txt apparently not18:20
*** trown|lunch is now known as trown18:21
prometheanfireI'd hope not18:21
clarkbprometheanfire: it did in the past18:21
prometheanfireya, it was annoying (as a packager)18:22
ssbarnea|rovercorvus: clarkb: unrelated to current issue with pycodestyle, i am curious if you would be pro/con starting to run linters via pre-commit in infra related projects.18:22
clarkbssbarnea|rover: I don't think it would help us, as we'd still ahve to enforce it on the server side18:23
clarkbyou can already set up pre commit hooks locally if you want to (and maybe we should document that), but we can't enforce it remotely18:24
ssbarnea|roverclarkb: to be clear: when I say pre-commit i mean the project to the "git-pre-commit hooks". the tool does not depend on being run as a hook, this being fully optional to each user. on CI we use it inside tox -e linters18:24
clarkbssbarnea|rover: git pre commit hooks run when you git commit, How does that fit into tox?18:25
clarkbI guess I don't quite understand18:26
ssbarnea|roverclarkb: sure, everyone that I told about this got confused because we all know how git commit hooks work and when we hear "pre-commit" we assume is a git hook.18:29
*** smarcet has joined #openstack-infra18:30
clarkbwell there is a pre-commit git hook so ya18:30
ssbarnea|rovercheck https://github.com/openstack/tripleo-quickstart/blob/master/tox.ini#L49  which is how you call it as a module (pre-commit cli also works).18:30
clarkb"The pre-commit hook is run first, before you even type in a commit message."18:30
clarkbssbarnea|rover: how is using a different tool better than just running the linters?18:30
clarkbssbarnea|rover: does that buy us something we don't already have?18:30
ssbarnea|roverits config looks like https://github.com/openstack/tripleo-quickstart/blob/master/.pre-commit-config.yaml -- as you can see there "bashare" ansible-lint and other linters defined, All of them pinned to specific revisions.18:31
ssbarnea|roverclarkb: you know hacking, right? pre-commit is like hacking-next-gen.18:31
ssbarnea|roverit does pinning, easy updating versions, optimize installation (reuse linters across projects when version matches).18:32
clarkbok so pre-commit is a generic tool to run linters and similar tools18:32
ssbarnea|roverclarkb: exactly!18:32
clarkband it manages versions very specifically18:32
clarkbwe do already have the ability to pin these tools to specific versions, so we probably need to decide if that is desireable and if it is why we haven't done it with the existing tools18:33
*** electrofelix has quit IRC18:34
*** diablo_rojo has quit IRC18:35
clarkbok I'm going to step out for a bit prior to the meeting. See you all in ~25 minutes18:35
*** sdake has joined #openstack-infra18:37
openstackgerritSorin Sbarnea proposed openstack-infra/git-review master: Switch from testrepository to stestr  https://review.openstack.org/62865518:43
*** hamzy has joined #openstack-infra18:52
*** e0ne has quit IRC18:53
*** yamamoto has joined #openstack-infra19:04
*** ykarel|away has quit IRC19:06
*** whoami-rajat has quit IRC19:07
*** e0ne has joined #openstack-infra19:07
*** dtantsur is now known as dtantsur|afk19:17
openstackgerritJames E. Blair proposed openstack-infra/system-config master: WIP: run a gerrit container  https://review.openstack.org/63040619:20
rm_workanyone aware of an issue with the publish jobs due to Twine being broken?19:39
rm_worklike: http://logs.openstack.org/94/624294/9/gate/test-release-openstack/d804b41/job-output.txt.gz#_2019-01-29_18_38_41_40937819:39
*** kjackal has joined #openstack-infra19:40
*** harlowja has joined #openstack-infra19:44
rm_workhttp://logs.openstack.org/94/624294/9/check/test-release-openstack/891bbd4/ara-report/result/8513865e-c4a4-48ff-ae90-2d47c88d9a3e/19:52
*** raissa has joined #openstack-infra19:53
*** bobh has quit IRC19:54
*** luizbag has quit IRC19:59
clarkbrm_work: looks like ubuntu xenial python3.5 doesn't have an OpenSSL module20:01
clarkbrm_work: I just checked locally and get a similar traceback from import OpenSSL20:01
*** eernst has quit IRC20:01
clarkbrm_work: my hunch is that the fix is to add whatever package adds that module to the bindep.txt for your project20:02
cmurphyclarkb: am i right in remembering that the puppet master no longer has puppet installed on it? should get rid of these silly mgmt_puppet_version things http://git.openstack.org/cgit/openstack-infra/ansible-role-puppet/tree/tasks/main.yaml#n320:02
cmurphythough i'm not sure how that doesn't error out if puppet isn't installed20:03
clarkbrm_work: python3-openssl ?20:03
*** raissa has quit IRC20:03
clarkbcmurphy: we must set mgmt_hieradata ?20:03
*** yamamoto has quit IRC20:05
*** raissa has joined #openstack-infra20:05
*** raissa has quit IRC20:05
rm_workclarkb: hmm but this worked until today20:05
clarkbcmurphy: http://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/group_vars/puppet.yaml20:05
rm_workso i assume twine broke something (though i need to check their releases)20:05
rm_workbut i guess we can add the bindep...20:05
clarkbrm_work: its possible twine was relying on cryptography before?20:05
rm_worki would just assume that this broke much more than us20:06
rm_workbut possibly other projects just happened to have that installed20:06
rm_workclarkb: do i need some magic to only install python3-openssl on py3 tests?20:07
*** raissa has joined #openstack-infra20:07
rm_workor is it safe to just do that generically20:07
*** raissa has quit IRC20:07
*** eernst has joined #openstack-infra20:08
clarkbrm_work: it should be safe to do it generically, it will just add a small amount of time to the job.20:09
*** agopi is now known as agopi|brb20:10
*** agopi|brb has quit IRC20:10
*** eernst has quit IRC20:12
*** fdegir_ has joined #openstack-infra20:14
*** eernst has joined #openstack-infra20:14
openstackgerritClark Boylan proposed openstack-infra/zone-opendev.org master: Verify zuul.opendev.org cert  https://review.openstack.org/63381720:18
openstackgerritClark Boylan proposed openstack-infra/zone-opendev.org master: Cleanup cert verification record  https://review.openstack.org/63381820:18
clarkbinfra-root ^ I think that is the last major piece of that puzzle20:18
clarkbnow to review the dns management docs change20:19
*** eernst has quit IRC20:19
*** eernst has joined #openstack-infra20:21
corvus+320:22
openstackgerritMerged openstack-infra/zone-opendev.org master: Verify zuul.opendev.org cert  https://review.openstack.org/63381720:22
*** eernst has quit IRC20:25
*** eernst has joined #openstack-infra20:27
*** eernst has quit IRC20:31
openstackgerritBrian Rosmaita proposed openstack-infra/irc-meetings master: Update glance meeting info  https://review.openstack.org/63382420:31
*** jamesmcarthur has quit IRC20:32
*** eernst has joined #openstack-infra20:33
*** jamesmcarthur has joined #openstack-infra20:33
*** zaneb has joined #openstack-infra20:36
*** eernst has quit IRC20:37
*** eernst has joined #openstack-infra20:39
*** jtomasek has joined #openstack-infra20:41
tobias-urdinpabelanger: could you check out https://review.openstack.org/#/c/632699/ thanks! :)20:41
*** e0ne has quit IRC20:42
openstackgerritPaul Vinciguerra proposed openstack-infra/zuul master: configloader.py: Not all jobs have an .updated attribute.  https://review.openstack.org/63325920:43
*** eernst has quit IRC20:43
*** sdake has quit IRC20:45
*** eernst has joined #openstack-infra20:45
*** eernst has quit IRC20:49
*** eernst has joined #openstack-infra20:51
*** agopi|brb has joined #openstack-infra20:54
*** agopi|brb is now known as agopi20:54
*** eernst has quit IRC20:54
*** xek has quit IRC20:54
*** eernst has joined #openstack-infra20:55
*** fdegir_ is now known as fdegir20:55
openstackgerritTobias Henkel proposed openstack/gertty master: Add inline comments to change overview  https://review.openstack.org/63382521:02
* corvus is running with that change now ^ :)21:03
tobiashcorvus: this is an attempt to add the inline comments on the change screen in gertty so one can follow discussions on changes without having to switch to the diff screen ^21:03
mriedemdo we have a bug reported for that pep8 error?21:03
mriedemprometheanfire: ^?21:03
clarkbmriedem: I didn't file a bug but notified the keystone and neutron teams (they seemed to be affected)21:04
mriedemtempest is also http://logs.openstack.org/66/633566/1/check/openstack-tox-pep8/b1da858/job-output.txt.gz#_2019-01-29_17_13_12_36770121:04
mriedemwhich is blocking getting the skip in for the other test that is killing the gate21:04
mriedemthe udhcp one21:04
corvustobiash: i think i might suggest a different color, possibly dark cyan, as light cyan gets a little confused with the header for my own messages.21:04
tobiashyeah, noticed that too now21:05
tobiashcorvus: also I have no idea how the inline messages could be indented after the line number21:06
corvustobiash: yeah, i was thinking the same thing...  possibly something in https://docs.python.org/3/library/textwrap.html might help21:07
prometheanfiremriedem: pep8 is in blacklist.txt, it's controlled via devstack initilization iirc21:07
mriedemthis wouldn't be devstack jobs though21:07
mriedemand it's pycodestyle right?21:07
prometheanfirealso in blacklist21:08
prometheanfirewe don't track those21:08
mriedemok so i just need to fix tempest21:08
prometheanfireya21:08
mriedemok21:08
corvustobiash: though right now, urwid is handling the line wrapping, so that could get weird if the terminal got too small.  so textwrap is probably not a good idea.21:08
prometheanfireconstrain it there, if reqs gate rejects the change we are doing something wrong21:08
prometheanfirehttps://github.com/openstack/requirements/blob/master/blacklist.txt21:09
corvustobiash: we'd probably need a new urwid widget, which is a fun weekend activity.  :)21:09
tobiashurwid is totally new to me... ;)21:09
corvustobiash: like maybe adding it as a two-column box, with the line number in the first column, and the message in the second.21:09
prometheanfireit's neat, I've not needed it yet though21:10
corvustobiash: it's powerful and fun, except that if you get *anything* wrong, it tends to tell you by raising an exception.  :)21:10
corvustobiash: i think the color change is the only thing we should do before landing that.  we can improve the formatting later.21:10
openstackgerritTobias Henkel proposed openstack/gertty master: Add inline comments to change overview  https://review.openstack.org/63382521:12
corvustobiash: oh i think i just saw a bug but was too dense to realize it and lost the traceback.21:13
corvusgot it21:13
corvustobiash: http://paste.openstack.org/show/744200/21:14
tobiashthat's triggered by a debugging leftover21:15
tobiashjust a sec21:15
openstackgerritTobias Henkel proposed openstack/gertty master: Add inline comments to change overview  https://review.openstack.org/63382521:15
openstackgerritMerged openstack/gertty master: Do not decode failed POST response  https://review.openstack.org/62607621:16
corvustobiash: same thing when i just review without leaving a comment: http://paste.openstack.org/show/744201/21:17
corvusand that's with the latest ps21:17
openstackgerritMerged openstack/gertty master: Fix crash on python3  https://review.openstack.org/61993821:17
corvusoh hrm, it couldn't be with the latest, sorry21:17
openstackgerritSean McGinnis proposed openstack-infra/zuul-jobs master: Make sure urllib3[secure] is installed for Twine use  https://review.openstack.org/63382921:19
corvustobiash: ok, confirmed that fixes it21:19
tobiash:)21:19
corvustobiash: i added some edge cases there too.  a file comment shows up as "None:" which is not wrong, but maybe could be better.  :)21:20
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP Revert "Revert "Add a timeout for the image build""  https://review.openstack.org/63379221:20
corvusat least it doesn't crash :)21:21
clarkbcorvus: https://review.openstack.org/633818 can be merged at this point (I am adding cert details to hiera now)21:22
corvusclarkb: done21:22
openstackgerritTobias Henkel proposed openstack/gertty master: Add inline comments to change overview  https://review.openstack.org/63382521:22
openstackgerritMerged openstack-infra/zone-opendev.org master: Cleanup cert verification record  https://review.openstack.org/63381821:23
tobiashthat makes the file comment nicer ^21:23
corvus++21:23
tonybprometheanfire: Nope you can have multiple constraint files but if a library is in both it'll cause pip to barf21:24
prometheanfireah21:25
prometheanfiremaybe it was requirements that had that behavior21:25
clarkbhiera/ansible vars all configured https://review.openstack.org/#/c/633783/2 and its depends on should be ready when we are21:27
corvusclarkb: how long until you drop?21:27
clarkbcorvus: its getting close :) though I can probably keep an eyeball on things if you want to hit go buttons21:28
clarkbI was packing while waiting on namecheap to verify :)21:28
*** kgiusti has left #openstack-infra21:28
clarkbcan pack while waiting for puppet to run21:28
corvusbuttons have been hit21:29
clarkbexciting21:29
*** diablo_rojo has joined #openstack-infra21:31
*** gfidente has quit IRC21:32
*** jcoufal has quit IRC21:39
*** hamzy has quit IRC21:42
smcginnisclarkb, others: We appear to have release failures due to twine that this is (hopefully) fixing: https://review.openstack.org/#/c/633829/21:48
smcginnisAny eyes there would be appreciated.21:48
*** eharney has quit IRC21:48
corvussmcginnis: have a link to the failure, and do you know what changed?21:48
clarkbsmcginnis: I think that will require libssl-dev (or whatever it is called on $distro) for the linkaging to happen21:49
smcginniscorvus: This is an example - http://logs.openstack.org/76/767f3ea3efcedb4c8df1dae1a6b40d0b0a230d3d/release/release-openstack-python/9701f8c/ara-report/result/03681c3b-d551-4581-9f7b-96fb4f5df174/21:49
smcginnisclarkb: I believe that's in our default set, IIRC.21:49
smcginniscorvus: Not sure what changed, but twine requires requests which requires urllib3 which only installs all necessary packages with urllib3[security].21:50
corvusthis is running on the executor21:51
corvuslibssl-dev is installed there21:51
smcginnisWould I need to update a bindep somewhere to make sure that has it?21:51
smcginnisOK, good.21:51
johnsomThere were both changes in project-config and urllib3 about the same time, both could be contributing21:51
clarkbif I had to guess urllib3 stopped pulling it by default21:52
clarkbprobably because it requires the header packages to build most places21:52
smcginnisThat area of their setup.cfg changed in Nov I think.21:52
johnsomThey moved how they were handling the "extras_requires"21:52
clarkbthough it like twine hasn't changed super recently21:52
smcginnisCouple layers of indirection with this one I think.21:53
johnsomhttps://github.com/urllib3/urllib3/commit/f80ff34bffb2018528ccd2ccf82e533d321f2af821:53
corvusjohnsom: know the project-config changes that might be relevant?21:53
johnsomcorvus This: https://review.openstack.org/#/c/616673 and https://github.com/openstack-infra/project-config/commit/4f3b36607738be78985982c3dc935836e9a1814d21:54
johnsomSwitched this over to python321:54
johnsomAbout the same time window with the the urllib3 change.21:55
corvusthat merged a long time ago, right?  but it looks like release-openstack-python succeded at 15:00 today21:55
clarkbis it possible that our newer executors don't have python3-openssl installed but the older onces do (or vice versa?)21:55
clarkbmaybe it is executor specific21:56
smcginnisOr it could depend if the project in question has requirements that pull it in.21:56
corvusran on ze0721:56
clarkbsmcginnis: oh ya or that21:56
corvussmcginnis: i don't believe the upload job is affected by the project in any way21:56
smcginnisHmm21:56
corvuslibssl-dev is installed on ze0721:57
johnsomIt wasn't getting as far as libssl-dev though, it is missing the python module21:58
clarkbya it never tries to install the pyopenssl or cryptography stuff. It just assumes it is there21:59
clarkbbut that change would've been back in november? maybe some other dep was pulling it in and they fixed that?22:00
corvushttps://pypi.org/project/requests-toolbelt/#history22:00
*** jtomasek has quit IRC22:01
*** trown is now known as trown|outtypewww22:01
corvusand here's a diff between the last success and first failure: http://paste.openstack.org/show/744210/22:02
corvusthe surprising thing is -- shouldn't we see pyopenssl or crytography or something on the old line?22:03
smcginnisI was wondering the same thing.22:04
clarkbcorvus: ya if it was transitive I think you would22:04
smcginnisNot seeing anyting yet in https://github.com/requests/toolbelt/compare/0.8.0...0.9.022:05
smcginnisBut there's the typical pull request commit message quality.22:05
johnsomIt's this patch: https://github.com/requests/toolbelt/commit/675fab790b0e9fefd6beb0f38d9a5da4b056c59c22:05
*** eernst has quit IRC22:05
johnsomThey added the dependency in tox.ini and dev-requirements.txt but not setup.py22:05
pabelangerI was just about to link that..22:05
smcginnisGood eye(s)22:06
pabelangeryah, I think pyopenssl should be in requirements / setup.py22:06
johnsomhttps://github.com/requests/toolbelt/issues/24122:06
pabelangerhttps://github.com/requests/toolbelt/pull/24222:07
corvusso would urllib3[secure] get what we need, or do we nee to add pyopenssl?22:08
johnsomurllib3[secure] includes pyopenssl and the other dependencies.  Question is, do we want to add this or somehow exclude that version of requests_toolbelt22:09
clarkbcorvus: I think urllib3[secure] pulls in pyopenssl (and others). Probably thye most correct thing is to install only pyopenssl22:09
clarkbjohnsom: oh ya if we just exclude 0.9.0 of requests-toolbelt then theoretically their next release will have that fix pabelangerlinked22:09
smcginnisIt shouldn't hurt anything, but we could also block 0.9.0.22:09
*** roman_g has quit IRC22:09
*** ekcs has joined #openstack-infra22:10
*** eernst has joined #openstack-infra22:10
*** eernst has quit IRC22:11
corvusmaybe that's what we should do?22:11
clarkbya I think that is the best option, block 0.9.022:11
smcginnisProblem with that is we don't install requests-toolbelt, twine does.22:11
clarkbsmcginnis: top level overrides should win22:12
clarkbso if we do pip install requests-toolbelt!=0.9.0 twine then the version we specified in that isntall will win22:12
clarkb(this is not true if you install them in two different pip runs)22:12
smcginnisI can update my patch to do that.22:12
corvussmcginnis: if you respin, can you throw in a link to https://github.com/requests/toolbelt/issues/241 in the commit message?22:13
openstackgerritSean McGinnis proposed openstack-infra/zuul-jobs master: Block installation of requests-toolbelt 0.9.0  https://review.openstack.org/63382922:15
corvusnormally we'd do a lot of testing for a change like that, but i'm pretty sure everyone using that is broken so we can just merge it and test in prod.22:17
smcginnisThe best way to test. :)22:17
*** sdake has joined #openstack-infra22:17
johnsomWe can re-check https://review.openstack.org/#/c/624294/, which is a quick job to test and revert if it doesn't solve it.22:18
corvuswe have a test-release-openstack job?  that's awesome :)22:18
*** rh-jelabarre has quit IRC22:18
pabelangeryah, I think it still uploads to test-pypi22:19
corvusi don't see any secrets for it22:20
corvusif we moved that job to openstack-zuul-jobs, we could use it to live-test changes like this22:20
corvusi assume it's in project-config to share the playbook22:20
corvusi bet we could rework this to do what we want though.22:21
openstackgerritMerged openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant  https://review.openstack.org/63358622:21
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient master: WIP: Add a button for uploading attachments  https://review.openstack.org/63384022:21
openstackgerritMerged openstack-infra/system-config master: Add zuul.opendev.org webserver  https://review.openstack.org/63378322:21
corvuspre and run in untrusted, then post in trusted22:23
clarkbhere we go. The last step if ^ looks good is to set up dns I think22:23
corvusclarkb: let's go ahead and do dns22:23
clarkbok I'll get a change up for that22:24
corvusthere is no existing zuul.opendev.org, so if it fails, it's no big deal22:24
openstackgerritClark Boylan proposed openstack-infra/zone-opendev.org master: Add zuul.opendev.org record  https://review.openstack.org/63384122:26
clarkbthat should do it22:26
openstackgerritMerged openstack-infra/zone-opendev.org master: Add zuul.opendev.org record  https://review.openstack.org/63384122:27
clarkblooking at puppet logs on zuul01 I expect that 23:06UTC ish is when the new stuff will apply22:32
*** gyee has quit IRC22:35
*** slaweq has quit IRC22:36
ekcsnot sure if this is the right place, but anyone got pointers on how to add a new member to project core team in Gerrit?22:45
ekcsThe ptl guide just says: "Update the core team in Gerrit." And I have scoured codesearch.o.o but so far failed to find what to do.22:45
clarkbekcs: in gerrit itself if you are logged in you can browse the groups at https://review.openstack.org/#/admin/groups/ use the little search bar there to find teh group you want, click on it, then add members on the members pane22:46
corvusekcs: this may help: https://review.openstack.org/Documentation/intro-project-owner.html#groups22:46
ekcsahhh of course. had to be the most obvious place =p thanks guys!22:47
clarkbthat link above is navigable from gerrit via the people tab -> list groups22:47
ekcsgot it. thanks a lot!22:55
*** tkajinam has joined #openstack-infra22:55
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: add-build-sshkey: remove previously authorized build-sshkey  https://review.openstack.org/63262023:06
clarkbwell that seems to have made apache have a sad :/23:07
clarkbI'm looking into why now23:07
*** diablo_rojo has quit IRC23:07
smcginnisNot sure if that ^ impacted anything, but I am getting a Not Found (404) error on my zuul.openstack.org page I have up.23:07
*** jamesmcarthur has quit IRC23:08
corvusaw man, we're 0 for 2.23:08
johnsomMe too23:08
clarkbhttps works23:08
clarkbbut http does not23:08
smcginnisFull refresh, Not Found error page.23:08
johnsom+1 https23:08
corvuswe should just redirect http to https (which, i expect, many of us thought was happening already)23:08
clarkbsmcginnis: try https23:08
smcginnisConfirmed switching to https works.23:08
clarkbcorvus: ya I'm confused as to how http worked in the past23:09
smcginniscorvus: I thought it used to work that way.23:09
openstackgerritMerged openstack-infra/zuul-jobs master: Block installation of requests-toolbelt 0.9.0  https://review.openstack.org/63382923:09
clarkbthe good new is its only half broken :)23:09
corvusyeah.  i think we should roll forward23:09
clarkbzuul.opendev.org https vhost also working for me. GIven that ya I think roll forward is good if we can figure out why http stopped working23:11
clarkbhow about I send #status notice http://zuul.openstack.org is not working. https://zuul.openstack.org does work. Please use that while we investigate.23:11
corvus++23:11
clarkb#status notice http://zuul.openstack.org is not working. https://zuul.openstack.org does work. Please use that while we investigate.23:12
openstackstatusclarkb: sending notice23:12
-openstackstatus- NOTICE: http://zuul.openstack.org is not working. https://zuul.openstack.org does work. Please use that while we investigate.23:13
corvusclarkb: we used to always create an http vhost, and optionally create an ssl vhost23:13
clarkbah23:14
clarkbcorvus: what if we add a port 80 to the vhost template that unconditionally redirects to $port23:14
clarkb(and start assuming https?)23:14
corvusclarkb: that wfm, but may not be backwards compatible for other users of the module?23:15
clarkbya it is less backward compatible whereas I tried to be backward compat with the older changes23:15
openstackstatusclarkb: finished sending notice23:15
corvus(mind you, i'm trying to get out of the business of maintaining modules for other users, but i think we decided to do that by switching to ansible)23:15
clarkbwhat we can do is add back in port 80 vhosts23:15
clarkbjust list them without the ssl23:15
corvusclarkb: basically duplicate the data structure that's there to also have a non-ssl version?23:16
clarkbbut I think what we really want is to redirect23:16
clarkbcorvus: ya I think so23:16
corvusclarkb: i think we should add the extra vhost so that we don't have to deal with puppet-zuul more than necessary, then redirect when we switch to ansible.23:16
clarkbwfm23:16
corvusclarkb: you want to write that or do you want me to?23:17
clarkbI'll do it really quick23:17
johnsomYep, that requests_toolbelt fix worked23:18
corvusjohnsom: \o/  need me to re-enqueue anything?23:18
johnsomNo, I am good. Thanks23:18
*** jamesmcarthur has joined #openstack-infra23:19
openstackgerritClark Boylan proposed openstack-infra/system-config master: Add http vhosts for zuul.o.o  https://review.openstack.org/63384523:20
clarkbcorvus: ^ I think that should work?23:20
corvus+323:21
*** jamesmcarthur has quit IRC23:23
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/buildsets route  https://review.openstack.org/63003523:28
clarkbok I'm going to pop out now for early dinner/late lunch. I'll be back to check on the http situation after23:29
*** sdake has quit IRC23:35
*** sdake has joined #openstack-infra23:37
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP add provides/requires support  https://review.openstack.org/63360523:39
smcginnisWould someone be able to reenqueu a failed job from earlier?23:44
smcginnishttp://logs.openstack.org/76/767f3ea3efcedb4c8df1dae1a6b40d0b0a230d3d/release/release-openstack-python/9701f8c/23:44
smcginnisThis was due to the twine>requests-toolbelt>urllib3 openssl issue.23:44
smcginnisI've verified the config change fixed this with a release-test.23:44
*** rcernin has quit IRC23:53
*** sdake has quit IRC23:55
*** sdake has joined #openstack-infra23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!