Friday, 2020-01-24

clarkbinfra-root I do think that https://review.opendev.org/#/c/704085/1 will be necessary after reading gerrit docs00:11
clarkbwe can have gerrit do basic auth via a config flag change00:11
clarkbbut then every script will have to update and that might be easier to swallow with a gerrit upgrade00:12
clarkbI think for now we should have zuul do digest explicitly, then zuul can set its default to basic and we will switch to basic when we update to 2.15 or newer00:12
*** trident has quit IRC00:13
*** trident has joined #openstack-infra00:14
*** dchen has joined #openstack-infra00:22
fungimakes me think a bunch of other scripts/tools we have doing digest auth to the rest api are going to need fixing00:25
fungioh, wait00:25
fungiso gerrit is dropping digest auth in coming releases, or...>00:27
fungiokay, yeah00:27
*** tosky has quit IRC00:28
fungiso we will need to update a bunch of stuff when we upgrade to a gerrit which doesn't support digest auth00:28
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Update sdk to v0.13.0 and install dhall-to-json  https://review.opendev.org/70210300:31
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Setup Prelude and Kubernetes binding  https://review.opendev.org/70210400:31
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Replace existing operator tasks with the new application  https://review.opendev.org/70210600:31
*** diablo_rojo has joined #openstack-infra00:32
openstackgerritMerged zuul/nodepool master: Lower min-ready request priority  https://review.opendev.org/70405800:34
*** goldyfruit_ has quit IRC00:34
*** goldyfruit_ has joined #openstack-infra00:34
*** dchen has quit IRC00:36
*** dchen has joined #openstack-infra00:38
clarkbfungi: yup00:56
fungigovernance tools, election tools, that release liaisons check job ttx just added...01:04
*** zxiiro has quit IRC01:08
*** yamamoto has joined #openstack-infra01:16
*** roman_g has quit IRC01:21
*** goldyfruit_ has quit IRC01:21
*** goldyfruit has joined #openstack-infra01:21
*** andreaf has quit IRC01:37
*** andreaf has joined #openstack-infra01:38
*** zxiiro has joined #openstack-infra01:41
*** rh-jelabarre has joined #openstack-infra01:50
*** diablo_rojo has quit IRC02:03
*** rf0lc0 has quit IRC02:18
*** yamamoto has quit IRC02:19
*** yamamoto has joined #openstack-infra02:25
*** goldyfruit has quit IRC02:25
*** armax has joined #openstack-infra02:40
*** dSrinivas has joined #openstack-infra03:25
*** stevebaker_ has joined #openstack-infra03:35
*** stevebaker has quit IRC03:36
*** yamamoto has quit IRC03:38
*** psachin has joined #openstack-infra03:38
*** goldyfruit has joined #openstack-infra03:38
*** goldyfruit has quit IRC03:43
*** goldyfruit has joined #openstack-infra03:43
*** zxiiro has quit IRC03:49
*** factor has quit IRC03:55
*** factor has joined #openstack-infra03:55
*** rlandy has quit IRC04:05
*** goldyfruit has quit IRC04:05
*** ykarel|away is now known as ykarel04:10
*** goldyfruit has joined #openstack-infra04:11
*** rh-jelabarre has quit IRC04:14
*** lajoskatona has joined #openstack-infra04:15
*** lajoskatona has left #openstack-infra04:15
*** ramishra has joined #openstack-infra04:19
*** goldyfruit has quit IRC04:27
*** goldyfruit has joined #openstack-infra04:27
*** kjackal has joined #openstack-infra04:32
*** goldyfruit has quit IRC04:33
*** goldyfruit has joined #openstack-infra04:33
*** yamamoto has joined #openstack-infra04:36
*** armax has quit IRC04:37
*** goldyfruit has quit IRC04:50
*** goldyfruit has joined #openstack-infra04:50
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Replace existing operator tasks with the new dhall function  https://review.opendev.org/70210604:50
*** udesale has joined #openstack-infra04:55
*** goldyfruit has quit IRC04:59
*** raukadah is now known as chkumar|ruck05:00
*** chkumar|ruck is now known as chkumar|rover05:24
*** evrardjp has quit IRC05:34
*** evrardjp has joined #openstack-infra05:34
*** surpatil has joined #openstack-infra05:40
*** gyee has quit IRC05:47
*** gfidente has joined #openstack-infra05:51
*** kjackal has quit IRC06:04
prometheanfireoffhand, how many containers are we using on rackcdn?  I only ask because enabling js on each of them is annoying :P06:05
*** kjackal has joined #openstack-infra06:29
*** kjackal has quit IRC06:34
*** lmiccini has joined #openstack-infra06:39
*** yamamoto has quit IRC06:40
*** yamamoto has joined #openstack-infra06:43
*** kjackal has joined #openstack-infra06:46
*** kjackal has quit IRC06:51
*** kjackal_v2 has joined #openstack-infra06:51
*** yamamoto has quit IRC06:58
*** yamamoto has joined #openstack-infra07:08
*** yamamoto has quit IRC07:15
*** yamamoto has joined #openstack-infra07:19
*** ykarel is now known as ykarel|lunch07:20
*** yamamoto has quit IRC07:29
*** pgaxatte has joined #openstack-infra07:31
*** kjackal_v2 has quit IRC07:34
*** yamamoto has joined #openstack-infra07:40
*** yamamoto has quit IRC07:41
*** yamamoto has joined #openstack-infra07:45
*** dtantsur|afk is now known as dtantsur07:48
*** ralonsoh has joined #openstack-infra07:56
*** yamamoto has quit IRC07:58
*** jpena|off is now known as jpena08:00
*** yamamoto has joined #openstack-infra08:03
*** elod is now known as elod_off08:09
*** yamamoto has quit IRC08:09
*** yamamoto has joined #openstack-infra08:09
*** yamamoto has quit IRC08:12
*** yamamoto has joined #openstack-infra08:16
*** tkajinam has quit IRC08:20
*** ccamacho has joined #openstack-infra08:20
*** ykarel|lunch is now known as ykarel08:24
*** deke997 has quit IRC08:30
*** tesseract has joined #openstack-infra08:34
*** tosky has joined #openstack-infra08:36
openstackgerritMerged zuul/zuul-jobs master: fetch-tox-output: introduce zuul_use_fetch_output  https://review.opendev.org/68186408:36
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Making opensuse-tumbleweed jobs nv  https://review.opendev.org/70269308:39
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Making opensuse-tumbleweed jobs nv  https://review.opendev.org/70269308:41
openstackgerritAndreas Jaeger proposed openstack/project-config master: Move tap-as-a-service job templates  https://review.opendev.org/70375208:43
*** rpittau|afk is now known as ironic08:51
*** ironic is now known as rpittau08:51
*** evrardjp has quit IRC08:54
*** xek has joined #openstack-infra08:55
*** yamamoto has quit IRC08:55
AJaegerconfig-core, please review https://review.opendev.org/703752 https://review.opendev.org/702693 and https://review.opendev.org/703917 , https://review.opendev.org/703691, https://review.opendev.org/703081 , https://review.opendev.org/#/c/70257408:56
fungiprometheanfire: for logs? thousands08:57
*** yamamoto has joined #openstack-infra08:57
*** yamamoto has quit IRC08:58
AJaegerfungi: What are you doing here at this time? All ok?08:59
fungiAJaeger: woke up, will probably be back asleep soon09:00
AJaegerfungi: ok, get some rest ;)09:02
fungisoon09:03
fungirsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(644) [generator=3.1.2]09:05
fungirsync error: received SIGUSR1 (code 19) at main.c(1447) [receiver=3.1.2]09:05
fungithat happened while or immediately after transferring noarch/tftpboot-installation-openSUSE-Kubic-x86_64-14.452-1.14.noarch.rpm09:05
fungii suspect that's a sign that the rsync took longer than our configured timeout of 30 minutes09:07
AJaegerso, let's hope next run will work... Thanks, fungi !09:07
fungiif we could split the tumbleweed mirror to its own volume we could release updates for it independently of the rest of our opensuse mirroring so that it doesn't block if we hold vos release on update failures09:07
fungibut yeah, it seems there was a fairly large batch of packages to update in tumbleweed on the last pulse09:09
fungiand 30 minutes was insufficient to transfer them all09:09
*** lucas has joined #openstack-infra09:11
*** pcaruana has joined #openstack-infra09:14
*** kjackal has joined #openstack-infra09:22
openstackgerritMerged openstack/project-config master: Move tap-as-a-service job templates  https://review.opendev.org/70375209:25
openstackgerritMerged openstack/project-config master: Add linaro-us cloud to nodepool  https://review.opendev.org/70391709:27
openstackgerritMerged zuul/zuul-jobs master: Making opensuse-tumbleweed jobs nv  https://review.opendev.org/70269309:31
ttxfungi: go back to sleep, stop jinxing me on threads09:32
*** roman_g has joined #openstack-infra09:32
openstackgerritMerged openstack/openstack-zuul-jobs master: Update ansible-lint  https://review.opendev.org/70257409:34
openstackgerritMerged openstack/project-config master: Add charm-interface-keystone-notifications project  https://review.opendev.org/70369109:34
openstackgerritMerged openstack/project-config master: Add dependent charm for watcher  https://review.opendev.org/70308109:35
*** derekh has joined #openstack-infra09:39
*** yamamoto has joined #openstack-infra09:41
*** yamamoto has quit IRC09:45
*** kevko has joined #openstack-infra09:49
kevkohello guys, i'm working on vitrage-dashboard to replace vendorized js script with XStatic versions, so I've already created and uploaded XStatic replacements to PIP .. but now i need to create openstack git repositories ... in this point I'm little bit confused how to deal with it09:50
*** yamamoto has joined #openstack-infra09:55
openstackgerritUzume proposed zuul/zuul master: Change default Gerrit HTTP auth method  https://review.opendev.org/70406810:00
*** yamamoto has quit IRC10:03
AJaegerkevko: docs.openstack.org/infra/manual/creators.html explains repo setup. You might want to talk with Horizon team on #openstack-horizon about XStatic repos as well since they created quite a few10:03
openstackgerritMerged zuul/zuul-jobs master: Return subunit test report as artifact  https://review.opendev.org/70323510:04
*** kjackal has quit IRC10:05
*** yamamoto has joined #openstack-infra10:12
kevkoAJaeger: thanks10:20
openstackgerritAndreas Jaeger proposed openstack/openstack-zuul-jobs master: WIP: Remove legacy-barbican-simple-crypto-dsvm-tempest-nova  https://review.opendev.org/70412810:20
*** witek has joined #openstack-infra10:31
openstackgerritMichal Arbet proposed openstack/project-config master: Add xstatic-** projects for vitrage-dashboard  https://review.opendev.org/70413310:33
*** lmiccini has quit IRC10:38
*** lmiccini has joined #openstack-infra10:40
*** soniya29 has joined #openstack-infra10:42
AJaegerkevko: re-read the docs - you need a governance change and link to it via Needed-By, otherwise we cannot merge it.10:44
kevkoAJaeger: Ah, ok , will process , thanks10:44
AJaegerkevko: did you discuss with amotoki ? You use the same group as horizon, so better get his buyin10:46
kevkoAJaeger: e0one was assigned to story and tasks for this in Storyboard, after discussion with him we took this tasks , me and my colleguage ..10:48
kevkoAJaeger: it is working now , so my colleguage create review for vitrage-dashboard and I add Needed-by to commit message .. correct ?10:49
AJaegerkevko: review for governance repo you mean? Yes, do that, and see my comments on review10:49
AJaegerkevko: and then we need vitrage and horizon PTLs to +1 due to the setup you do with shared ACLs.10:50
kevkoAJaeger: ok, thanks, everything is clear :)10:51
AJaegercool10:54
*** yamamoto has quit IRC10:57
*** gshippey has joined #openstack-infra10:58
*** yamamoto has joined #openstack-infra10:59
*** yamamoto has quit IRC11:01
*** yamamoto has joined #openstack-infra11:02
*** witek has quit IRC11:15
*** Ng has quit IRC11:15
*** Ng has joined #openstack-infra11:16
*** yamamoto has quit IRC11:19
*** witek has joined #openstack-infra11:22
*** yamamoto has joined #openstack-infra11:24
*** rpittau is now known as rpittau|bbl11:28
*** surpatil has quit IRC11:32
*** dSrinivas has quit IRC11:33
openstackgerritMichal Arbet proposed openstack/project-config master: Add xstatic-** projects for vitrage-dashboard  https://review.opendev.org/70413311:38
*** yamamoto has quit IRC11:42
*** yamamoto has joined #openstack-infra11:45
kevkoAJaeger: I have one more question , what I can do if vitrage-dashboard needs 2 additional packages from PIP , ant these packages don't have git repo , and I am not owner :(  ?11:46
kevkoAJaeger: I mean 2 additional xstatic packages11:46
*** iurygregory has joined #openstack-infra11:48
*** yamamoto has quit IRC11:56
amotokikevko: AJaeger: IMHO if the proposed xstatic repos are specific to vitrage dashboard, it would be better to host by the vitrage dashboard team (= vitrage team actually). the number of horizon devs is decreasing, so I don't think it works for the horizon team hosts all xstatic repos including ones not used by horizon itself.12:06
amotokiI know it should be discussed in the governance patch so I just express my feeling now.12:07
kevkoamotoki: ok, understand , one more question > do i need really create openstack git repos ? could I add only requirement for XStatic packages which are already in pip ?12:08
kevkoamotoki: i've uploaded to pip already and working ..12:08
*** ykarel is now known as ykarel|afk12:08
*** kjackal has joined #openstack-infra12:10
amotokikevko: it leads to another question. where do you manage the source of the xstatic PyPI package you uploaded.12:10
amotoki?12:10
kevkoamotoki: my github acc12:11
amotokikevko: if you continue to manage your github repo it would be another option.12:11
kevkoamotoki: yes I will12:11
amotokikevko: the point is that it is important to provide the source and keep it available thoughout the life of vitrage dashboard at least. it might be longer than the period you work on.12:12
amotokikevko: that's the only merit I see in creating a repo under openstack/opendev infra.12:13
*** rf0lc0 has joined #openstack-infra12:14
kevkoamotoki: ok, understand ...12:15
amotokikevko: e0ne might have a different opinion. he is also a vitrage-dashboard core.12:16
amotokihi infra-root?, could you abandon https://review.opendev.org/#/c/398936/ ? It was proposed against refs/meta/config branch somehow, so horizon-core cannot abandon it.12:21
frickleramotoki: done12:23
amotokifrickler: thanks!12:23
*** soniya29 has quit IRC12:29
*** jpena is now known as jpena|lunch12:29
*** soniya29 has joined #openstack-infra12:30
*** yamamoto has joined #openstack-infra12:36
*** eharney has joined #openstack-infra12:41
*** yamamoto has quit IRC12:43
*** udesale_ has joined #openstack-infra12:45
*** rh-jelabarre has joined #openstack-infra12:47
*** udesale has quit IRC12:48
openstackgerritTobias Henkel proposed zuul/zuul master: Handle draft pull requests in canMerge  https://review.opendev.org/70415012:55
*** jtomasek has joined #openstack-infra12:58
*** artom has quit IRC13:01
*** witek has quit IRC13:02
*** kjackal has quit IRC13:03
openstackgerritTobias Henkel proposed zuul/zuul master: Handle draft pull requests in canMerge  https://review.opendev.org/70415013:04
*** sshnaidm|afk has quit IRC13:11
*** iurygregory has quit IRC13:15
*** ykarel|afk is now known as ykarel13:15
*** hashar has joined #openstack-infra13:19
*** ociuhandu has joined #openstack-infra13:25
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Generate TLS certificats for the gearman service  https://review.opendev.org/70271613:25
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add OpenShift SCC and functional test  https://review.opendev.org/70275813:25
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Handle service restart when connections are changed  https://review.opendev.org/70362413:25
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add tenant reconfiguration when main.yaml changed  https://review.opendev.org/70363113:25
*** ociuhandu has quit IRC13:25
*** ociuhandu has joined #openstack-infra13:26
*** rlandy has joined #openstack-infra13:28
*** rf0lc0 is now known as rfolco|doctor13:33
*** ociuhandu has quit IRC13:33
*** jpena|lunch is now known as jpena13:35
*** rpittau|bbl is now known as rpittau13:40
*** jtomasek has quit IRC13:44
*** jamesmcarthur has joined #openstack-infra13:48
*** petevg has quit IRC13:48
*** petevg has joined #openstack-infra13:48
*** kjackal has joined #openstack-infra13:51
*** ykarel is now known as ykarel|afk14:07
*** auristor has quit IRC14:07
openstackgerritAntoine Musso proposed zuul/zuul master: On CI, use a tmpfs for ZUUL_TEST_ROOT  https://review.opendev.org/70416014:10
*** kjackal has quit IRC14:10
openstackgerritsebastian marcet proposed opendev/puppet-openstackid master: Updated Build  https://review.opendev.org/70383414:11
*** sshnaidm has joined #openstack-infra14:13
*** artom has joined #openstack-infra14:13
*** piotrowskim has quit IRC14:13
*** sshnaidm has quit IRC14:17
*** ociuhandu has joined #openstack-infra14:23
openstackgerritAntoine Musso proposed zuul/zuul master: On CI, use a tmpfs for ZUUL_TEST_ROOT  https://review.opendev.org/70416014:23
*** sshnaidm has joined #openstack-infra14:24
*** zxiiro has joined #openstack-infra14:24
*** ociuhandu has quit IRC14:28
*** hashar has quit IRC14:29
fungiinfra-puppet-core: can we get a second pair of eyes on https://review.opendev.org/703834 ?14:29
fungipuppet-openstackid change for an upgrade to the openstackid-dev server14:29
*** ralonsoh is now known as ralonsoh-lunch14:30
*** sshnaidm_ has joined #openstack-infra14:31
*** sshnaidm has quit IRC14:31
*** andrewbonney has joined #openstack-infra14:33
*** ociuhandu has joined #openstack-infra14:33
*** witek has joined #openstack-infra14:33
*** goldyfruit has joined #openstack-infra14:34
fricklerconfig-core: please review this change to add py38 nv jobs https://review.opendev.org/693401 . AJaeger: I'm hoping your previous concern have been resolved by what the TC did14:35
*** panda is now known as panda|bbl14:45
*** soniya29 has quit IRC14:45
*** Tengu has quit IRC14:46
*** lucas is now known as lucasagomes14:46
fricklerfungi: failed lint, otherwise lgtm14:48
*** seongsoocho has quit IRC14:49
fungioh, yep, i missed that there was a new patchset14:49
*** seongsoocho has joined #openstack-infra14:49
fungii'll fix that for him real quick14:49
openstackgerritJeremy Stanley proposed opendev/puppet-openstackid master: Updated Build  https://review.opendev.org/70383414:52
fungifrickler: ^{14:52
*** psachin has quit IRC14:52
fungithanks for spotting that14:52
*** dklyle has quit IRC14:54
*** goldyfruit_ has joined #openstack-infra14:54
*** goldyfruit has quit IRC14:55
*** goldyfruit_ has quit IRC14:55
*** goldyfruit_ has joined #openstack-infra14:55
*** coreycb has quit IRC14:57
*** coreycb has joined #openstack-infra14:57
*** hashar has joined #openstack-infra15:03
*** dklyle has joined #openstack-infra15:04
openstackgerritAntoine Musso proposed zuul/zuul master: On CI, use a tmpfs for ZUUL_TEST_ROOT  https://review.opendev.org/70416015:04
openstackgerritAntoine Musso proposed zuul/zuul master: On CI, use a tmpfs for ZUUL_TEST_ROOT  https://review.opendev.org/70416015:05
*** goldyfruit_ has quit IRC15:07
*** goldyfruit_ has joined #openstack-infra15:07
*** jamesmcarthur has quit IRC15:08
*** yamamoto has joined #openstack-infra15:10
*** yamamoto has quit IRC15:15
*** goldyfruit_ has quit IRC15:16
*** goldyfruit_ has joined #openstack-infra15:17
*** chkumar|rover is now known as raukadah15:19
*** jtomasek has joined #openstack-infra15:20
AJaegerfrickler: thanks - missed that the dependency finally merged15:24
*** zzzeek has quit IRC15:24
*** sshnaidm_ is now known as sshnaidm|off15:25
*** ralonsoh-lunch is now known as ralonsoh15:26
*** Tengu has joined #openstack-infra15:27
*** jtomasek has quit IRC15:28
*** dulek has quit IRC15:28
*** jamesmcarthur has joined #openstack-infra15:28
openstackgerritMerged openstack/openstack-zuul-jobs master: Add non-voting tox py38 jobs to Ussuri templates  https://review.opendev.org/69340115:28
fungiwe've been granted some really nice weather this morning, so i'm going to take advantage of it while i can and go for a long walk. should be back in a couple hours though15:29
AJaegerfungi: enjoy!15:29
*** armax has joined #openstack-infra15:30
*** mnasiadka has quit IRC15:31
*** mnasiadka has joined #openstack-infra15:32
*** zzzeek has joined #openstack-infra15:33
*** auristor has joined #openstack-infra15:33
openstackgerritMerged opendev/puppet-openstackid master: Updated Build  https://review.opendev.org/70383415:44
*** dulek has joined #openstack-infra15:44
*** ociuhandu has quit IRC15:49
*** ociuhandu has joined #openstack-infra15:50
*** ociuhandu has quit IRC15:55
*** panda|bbl is now known as panda16:02
openstackgerritMerged zuul/zuul master: Fix canceling builds in starting phase  https://review.opendev.org/68441416:06
*** aedc_ has joined #openstack-infra16:14
*** gyee has joined #openstack-infra16:14
*** aedc has quit IRC16:15
*** tosky has quit IRC16:16
*** lmiccini has quit IRC16:18
*** udesale_ has quit IRC16:19
*** auristor has quit IRC16:23
*** artom has quit IRC16:26
*** auristor has joined #openstack-infra16:26
*** hashar has quit IRC16:28
*** yamamoto has joined #openstack-infra16:30
*** Lucas_Gray has joined #openstack-infra16:32
*** gshippey has quit IRC16:36
*** gshippey has joined #openstack-infra16:36
openstackgerritPaul Belanger proposed zuul/zuul master: Update component diagram for GitHub / Web events  https://review.opendev.org/70417216:37
openstackgerritPaul Belanger proposed zuul/zuul master: Update component diagram for GitHub / Web events  https://review.opendev.org/70417216:41
*** jackedin has joined #openstack-infra16:41
*** ociuhandu has joined #openstack-infra16:45
*** rfolco|doctor is now known as rfolco16:45
AJaegerconfig-core, a small doc update for zuul-jobs for review, please: https://review.opendev.org/70401716:49
*** Goneri has joined #openstack-infra16:50
*** Lucas_Gray has quit IRC16:55
*** Lucas_Gray has joined #openstack-infra16:59
*** gfidente is now known as gfidente|afk17:00
*** diablo_rojo has joined #openstack-infra17:03
*** dtantsur is now known as dtantsur|afk17:03
*** hashar has joined #openstack-infra17:05
*** hashar has quit IRC17:06
*** ociuhandu has quit IRC17:07
*** hashar has joined #openstack-infra17:07
*** hashar has quit IRC17:08
openstackgerritMerged zuul/zuul-jobs master: Update roles/prepare-workspace-openshift/README.rst  https://review.opendev.org/70401717:08
*** hashar has joined #openstack-infra17:08
*** ociuhandu has joined #openstack-infra17:09
*** lucasagomes has quit IRC17:09
*** hashar has quit IRC17:09
*** jpena is now known as jpena|off17:11
*** ociuhandu has quit IRC17:13
*** artom has joined #openstack-infra17:13
*** ociuhandu has joined #openstack-infra17:14
clarkbfungi: when you get back from your walk can you review https://review.opendev.org/#/c/704085/ so that we don't get caught by the zuul change to use basic auth against gerrit?17:15
fungioh, yep, already back just catching up quietly. will certainly take a look17:16
*** pgaxatte has quit IRC17:17
fungiclarkb: also part of the reason not to do basic auth right now is, i think, that they didn't drop digest auth until after they switched to storing web password hashes instead of the plaintext versions17:18
*** ralonsoh has quit IRC17:18
*** goldyfruit_ has quit IRC17:18
*** kjackal has joined #openstack-infra17:18
fungi(your long-time security bug report)17:18
fungior, i guess they did it at the same time17:19
fungisince digest auth would need the server to have access to the plaintext17:19
clarkbah17:20
fungianyway, we get some limited security of passwords in-flight with digest auth, but it's a reasonable sacrifice to switch to basic auth if it means we get to no longer have a database full of plaintext passwords17:22
*** tesseract has quit IRC17:27
*** ykarel|afk is now known as ykarel|away17:30
openstackgerritMerged zuul/zuul master: test: prevent ResourceWarning in test_client  https://review.opendev.org/70378217:32
*** rpittau is now known as rpittau|afk17:35
*** ociuhandu has quit IRC17:39
openstackgerritMerged opendev/system-config master: Set gerrit digest auth in zuul config  https://review.opendev.org/70408517:39
*** ociuhandu has joined #openstack-infra17:40
*** rlandy is now known as rlandy|brb17:41
openstackgerritPaul Belanger proposed zuul/zuul master: Add estimated time remaining tooltip to UI  https://review.opendev.org/70389217:42
openstackgerritPaul Belanger proposed zuul/zuul master: WIP: debug estimate_time on job start  https://review.opendev.org/70417917:42
*** ociuhandu has quit IRC17:46
openstackgerritMerged zuul/zuul master: Whitelist debug threads used by visual studio code  https://review.opendev.org/70324817:49
*** jamesmcarthur has quit IRC17:52
*** yamamoto has quit IRC17:56
openstackgerritTobias Henkel proposed zuul/zuul master: Use apt mirror infrastructure during zuul-quick-start  https://review.opendev.org/64944817:56
*** yamamoto has joined #openstack-infra17:57
fungiand now i'm needing to go run some lunch errands, but will be back as soon as i can17:57
*** derekh has quit IRC18:01
*** elod_off has quit IRC18:05
*** elod_off has joined #openstack-infra18:06
*** rlandy|brb is now known as rlandy18:09
clarkbre linaro-us the nodepool.yaml config has updated but not the clouds.yaml18:18
openstackgerritPaul Belanger proposed zuul/zuul master: WIP: debug estimate_time on job start  https://review.opendev.org/70417918:20
zbrwhat do you think about these timeouts on zuul: https://zuul.opendev.org/t/zuul/builds?job_name=tox-py35&project=zuul%2Fzuul18:20
zbrshould we override it from 3600 to 4000 ?18:21
clarkblinaro-us template was failing because nodepool_linaro_us_project is used in the templates but nodepool_linaro_us_project_name was the group var. I edited the group var to match the code18:22
clarkbI expect it will be fixed on the next pass around18:22
clarkbzbr: that is a question for the zuul project in #zuul. I believe there has been work to make the tests run more efficiently as well as splitting them up so they don't take so long18:23
zbryeah, https://review.opendev.org/#/c/702473/18:23
zbri have mixed feelings about myself18:23
zbri personally do not like sharding, but....18:24
clarkbya that is why I started with trying to make slow tests run faster. hashar seems to ahve taken some of my investigating and added further improvements18:24
*** jamesmcarthur has joined #openstack-infra18:24
zbr10 days latest, the fix did not ship.18:24
clarkbwe're stilltalking like ~5 minutes but its something18:24
clarkbin any case if extending the timeout is a question for zuul, it is project configurable18:25
*** pkopec has joined #openstack-infra18:25
*** witek has quit IRC18:30
*** yamamoto has quit IRC18:35
openstackgerritTobias Henkel proposed zuul/zuul master: Disable pip cache when managing ansible  https://review.opendev.org/70418418:35
openstackgerritSorin Sbarnea proposed zuul/zuul master: Increase timeout for tox-py27/36 jobs  https://review.opendev.org/70418518:38
openstackgerritTobias Henkel proposed zuul/zuul master: DNM: Test multi-threaded ansible setup without cache  https://review.opendev.org/70418618:39
openstackgerritSorin Sbarnea proposed zuul/zuul master: Increase timeout for tox-py35/37 jobs  https://review.opendev.org/70418518:41
*** psachin has joined #openstack-infra18:47
openstackgerritClark Boylan proposed opendev/system-config master: Update project doc to reflect OpenDev changes  https://review.opendev.org/70348818:49
clarkbttx: ^ thank you for the feedback18:49
clarkbI think linaro-us is working now18:53
clarkbbut hasn't done any work yet18:53
clarkbyoctozepto: ^ if you've got any kolla changes that need arm64 testing now is a good time to test out the new cloud :)18:53
clarkbwe are probably watiing on image uploads right now though18:53
yoctozeptoclarkb: thanks, I believe we are constantly sending jobs on the aarch64 pipeline18:54
clarkbcool I'll just be patient then18:54
*** ociuhandu has joined #openstack-infra18:55
openstackgerritPaul Belanger proposed zuul/zuul master: Add estimated time remaining tooltip to UI  https://review.opendev.org/70389218:56
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: DNM: test arbritary uid support for zuul image  https://review.opendev.org/70298018:57
*** ociuhandu has quit IRC18:59
openstackgerritMerged zuul/zuul master: On CI, use a tmpfs for ZUUL_TEST_ROOT  https://review.opendev.org/70416019:01
clarkbkevinz: hrw: we are seeing connections reset when uploading images to linaro-us19:02
clarkbI'm trying to sort out where that is happening now19:03
clarkbport 5000 appears to be the problem which is keystone I think19:04
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add zuul-operator-promote-image job  https://review.opendev.org/70418719:04
clarkbok I think it must be a firewall problem19:05
clarkbkevinz: hrw: nl03.openstack.org (the nodepool launcher for linaro) can talk to linaro-us but nb03.openstack.org (the image build and uploader for arm images) cannot19:05
*** pkopec has quit IRC19:06
*** pkopec has joined #openstack-infra19:08
*** pkopec has quit IRC19:09
*** pkopec has joined #openstack-infra19:09
*** kjackal has quit IRC19:10
openstackgerritMerged zuul/zuul master: Don't add implicit role if plugin found  https://review.opendev.org/69171519:11
*** ociuhandu has joined #openstack-infra19:11
*** yamamoto has joined #openstack-infra19:13
*** ociuhandu has quit IRC19:16
openstackgerritTobias Henkel proposed zuul/zuul master: Increase timeout for tox-py35/37 jobs  https://review.opendev.org/70418519:19
*** jamesmcarthur has quit IRC19:23
*** psachin has quit IRC19:23
*** yamamoto has quit IRC19:24
*** jackedin has quit IRC19:36
*** pkopec has quit IRC19:40
*** pkopec has joined #openstack-infra19:41
*** Lucas_Gray has quit IRC19:45
openstackgerritMerged zuul/zuul master: Update component diagram for GitHub / Web events  https://review.opendev.org/70417219:48
openstackgerritClark Boylan proposed zuul/zuul master: Be more specific with remaining time  https://review.opendev.org/70419119:55
*** pkopec has quit IRC19:55
*** pkopec has joined #openstack-infra19:56
openstackgerritTristan Cacqueray proposed zuul/zuul master: Dockerfile: add support for arbritary uid  https://review.opendev.org/65024619:56
*** pkopec has quit IRC19:59
*** pkopec has joined #openstack-infra20:01
*** slaweq_ has joined #openstack-infra20:01
*** kjackal has joined #openstack-infra20:06
fungigood news: we are no longer copying wheels from pypi into our wheel cache, so anything we delete from the cache which exists on pypi will not be repopulated20:14
fungii've gone digging in the logs from the past couple nights across the three architectures for which we do wheel building20:15
fungiseems to be operating as hoped20:15
openstackgerritMerged zuul/zuul master: Increase timeout for tox-py35/37 jobs  https://review.opendev.org/70418520:16
fungiwe still have two obvious holes though... no py37 wheels for ubuntu-bionic, and no wheels for centos-8 yet20:16
*** artom has quit IRC20:19
openstackgerritTristan Cacqueray proposed zuul/nodepool master: kubernetes: add release note about service account token issue  https://review.opendev.org/70386920:20
*** goldyfruit_ has joined #openstack-infra20:26
*** hashar has joined #openstack-infra20:30
clarkbcentos8 is in progress though20:41
*** hashar has quit IRC20:41
clarkbI sent email to ianw hrw and kevinz re the firewall issue with nb03 uploading images to the new linaro cloud20:44
clarkbhopefully won't get missed with the weekend and holdiays this way20:44
fungii also haven't found time yet to perform comprehensive testing for reproducible wheel builds across our full data set20:46
openstackgerritClark Boylan proposed zuul/zuul master: Be more specific with remaining time  https://review.opendev.org/70419120:55
*** hashar has joined #openstack-infra20:58
openstackgerritSlawek Kaplonski proposed opendev/irc-meetings master: Move Neutron CI meeting to Wednesday  https://review.opendev.org/70420021:00
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: fetch-subunit-output: introduce zuul_use_fetch_output  https://review.opendev.org/68188221:05
*** slaweq_ has quit IRC21:06
openstackgerritAntoine Musso proposed zuul/zuul master: Add client_id to RPC client  https://review.opendev.org/70420121:07
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: fetch-subunit-output: introduce zuul_use_fetch_output  https://review.opendev.org/68188221:08
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: DNM: test tox-py36 on openshift node  https://review.opendev.org/68204921:08
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: fetch-javascript-tarball: introduce zuul_use_fetch_output  https://review.opendev.org/68160321:15
*** kjackal has quit IRC21:18
openstackgerritClark Boylan proposed zuul/zuul master: Be more specific with remaining time  https://review.opendev.org/70419121:18
smcginnisThat's some life advice there. ^21:19
clarkbha21:19
clarkbfungi: I think it is my turn to pop out and enjoy the outdoors. Though I wouldn't call it nice out right now. Cool and Damp. Could be worse21:20
clarkbsmcginnis is probably under a foot of snow21:22
*** zxiiro has quit IRC21:23
fungiyeah, definitely go be more specific with your remaining time. it's worthwhile21:26
fungii'll keep an eye out for emergencies21:26
*** ociuhandu has joined #openstack-infra21:29
*** ociuhandu has quit IRC21:34
*** artom has joined #openstack-infra21:35
*** pkopec has quit IRC21:41
*** rlandy has quit IRC21:44
*** diablo_rojo has quit IRC21:48
*** Lucas_Gray has joined #openstack-infra21:50
*** rfolco has quit IRC22:00
*** pkopec has joined #openstack-infra22:17
*** jamesmcarthur has joined #openstack-infra22:21
ianwclarkb: hrm, yeah i'm not sure what would cause nb->cloud issues ... can it ping the mirror?22:22
ianw64 bytes from 139.178.85.143: icmp_seq=1 ttl=40 time=111 ms22:23
ianwyes22:23
*** pkopec has quit IRC22:23
ianwi can telnet us.linaro.cloud 500022:23
ianwCould not find versioned identity endpoints when attempting to authenticate.22:24
ianwi wonder if that is the root cause22:24
ianwi think it needs the /v3 on the nodepool config ...22:24
ianwyeah, i added that by hand and now it's uploading22:27
fungiif you can push the fix i'll review asap22:28
ianwhrm, hang on, maybe not22:28
ianwstill seems to report the old url, i wonder if nb doesn't notice the clouds.yaml update22:29
fungiprobably needs restarting22:30
fungii don't think any nodepool services deteck conffile updates22:30
ianwit's not building anything, i'm going to restart it22:30
fungier, detect22:30
*** rcernin has quit IRC22:34
ianwhrm, the builder seems to be in a busy loop doing something, but it's not putting any info in the logs22:35
ianwcannot create temp file for here-document: No space left on device22:36
*** xek has quit IRC22:38
fungiyeah, /mnt seems to be entirely full22:46
fungii guess it needs the service disabled, the server rebooted, and the disk cleaned up22:47
fungii can give that a shot if you want, it's your saturday after all22:47
ianwfungi: yeah, removing it now and see if we can get it going with the /v322:48
ianwit's ok, i'm housebound waiting for someone to come connect my internet cable anyway :)  yes the saga continues22:48
fungiyeesh22:49
*** hashar has quit IRC22:49
*** petevg has quit IRC22:49
ianwi don't know why we have /mnt and /opt22:50
fungisomething to do with the flavor in that cloud maybe?22:51
ianwyeah, it seems to be cloud-init related.  i'm commenting it out, as we have the uuid mount below it22:51
clarkbianw: oh, did nl03 have the /v3?22:52
clarkbianw: I was able to curl without the v3 on nl03 but not from nb0322:52
fungiseems /mnt and /opt are the same device?22:52
clarkbfungi: ya I think config drive magic causes that22:52
fungifstab has a /opt entry22:53
fungibut no /mnt entry22:53
ianwfungi: i just commented out the /mnt entry22:53
fungioh22:53
funginow i see it22:53
ianw(but it's still mounted from prior boot)22:53
fungiyeah, one was by uuid22:53
fungidefaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig22:54
fungithat does indeed look like something cloud-init might have helpfully done for us22:54
ianwclarkb: hrm, no the launcher doesn't have /v3, and neither does the uk22:55
ianwbut it did spew that error about versioned enpoints22:56
*** jamesmcarthur has quit IRC22:57
clarkbjust double checked `curl https://us.linaro.cloud:5000` works from nl03 but not nb0322:58
clarkbfrom nb03 it seems the connection never successfully happens22:58
*** apetrich has quit IRC22:58
ianwyeah, agree, weird.  it seems to connect (like the telnet did) but not talking23:02
ianwsecurity group for linaro-uk seems ok? http://paste.openstack.org/show/788786/23:05
clarkbianw: ya I wonder if it is a firewall in front of the linaro-us api server23:08
clarkbthough I can actually hit it from home so would be a weird set of rules23:09
fungisomething terminating ssl and doing layer 7 inspection?23:10
ianwi jumped on another testing node in uk.linaro.org and it also can't get there23:14
ianwso it's not specific to the control plane project23:14
*** jamesmcarthur has joined #openstack-infra23:14
ianwkevinz: i've put all this into https://storyboard.openstack.org/#!/story/2007195 , task  3846323:16
ianwfungi: are there plans to make it so tasks have individual comments ... or am i using it wrong?23:16
*** bdodd has quit IRC23:17
fungiianw: tasks basically have descriptions for now. you can fill those with commentary but there's no history for them23:19
funginot attribution for the prose you add23:19
fungier, nor attribution23:20
*** jamesmcarthur has quit IRC23:23
*** eharney has quit IRC23:29
clarkbthe fact that ping works implies routing is fine23:29
clarkbwhich really does make me think a firewall is involved, but our security groups look ok so probably outside our control23:30
fungiwell, pathologically there could be something else responding to ping on behalf of that address, but i'd rate that fairly unlikely23:30
fungispecifically layer 7 firewalling yeah? you said you can get a response to some api queries on that port?23:31
openstackgerritClark Boylan proposed zuul/zuul master: Be more specific with remaining time  https://review.opendev.org/70419123:32
clarkbI am not able to no23:33
clarkbianw indicated that may be the case though23:33
ianwno, i don't think i can.  we see the connect happen, but no communications23:33
fungiahh, okay, socket is established though?23:34
ianweven after cleaning /opt(/mnt) on nb03, it /only/ has ~20g available23:34
fungibut yeah, starting to sound more and more like a basic firewall problem23:35
fungior maybe repeated/rapid api call failures tripped some sort of anti-ddos mechanism23:36
ianwshould we perhaps leave nb03 disabled until we get upstream to look at this?23:36
funginodepool does have a well-earned reputation for hammering apis23:36
*** jamesmcarthur has joined #openstack-infra23:40
ianwok, i'm rebooting nb0323:43
ianwi guess it doesn't hurt for it to be stuck uploading, really23:43
ianwbut i i think we'll need kevinz and maybe a packet trace on the us.linaro.org side to make further progress23:43
clarkbdoes the graphite composer have no option for "give me the url for this graph"?23:47

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!