Friday, 2020-02-07

openstackgerritJames E. Blair proposed zuul/zuul master: Extract allow/disallow filter into util function  https://review.opendev.org/70614400:00
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: WIP: add an ensure-tox test job  https://review.opendev.org/70637100:04
*** Tengu has quit IRC00:04
*** diablo_rojo has quit IRC00:05
*** Tengu has joined #openstack-infra00:06
*** diablo_rojo has joined #openstack-infra00:06
*** mattw4 has quit IRC00:08
*** armax has joined #openstack-infra00:10
openstackgerritMerged opendev/system-config master: Add tarballs.<openstack|opendev>.org to static.opendev.org  https://review.opendev.org/70491300:22
openstackgerritMerged zuul/zuul master: Add disallowed-labels tenant option  https://review.opendev.org/70585600:53
*** diablo_rojo has quit IRC01:13
*** zxiiro has quit IRC01:18
*** michael-beaver has quit IRC01:28
*** Lucas_Gray has quit IRC01:32
*** armax has quit IRC01:32
*** dangtrinhnt has joined #openstack-infra01:49
*** gagehugo has quit IRC01:58
*** gyee has quit IRC02:03
*** rosmaita has quit IRC02:16
*** rosmaita has joined #openstack-infra02:17
*** pkopec has quit IRC02:17
*** eernst has joined #openstack-infra02:21
*** rh-jelabarre has quit IRC02:21
*** armax has joined #openstack-infra02:22
*** rfolco has quit IRC02:22
*** Goneri has quit IRC02:26
*** dSrinivas has joined #openstack-infra02:27
*** eernst has quit IRC02:28
*** rfolco has joined #openstack-infra02:36
ianwkevinz: ok, one small fix to the launcher and i think we've got a nb comin gup02:38
*** rosmaita has quit IRC02:41
*** jamesmcarthur has joined #openstack-infra02:44
*** rosmaita has joined #openstack-infra02:58
*** rosmaita has quit IRC02:59
*** rfolco has quit IRC03:02
openstackgerritIan Wienand proposed opendev/system-config master: launch-node.py : make sure new inventory comes last  https://review.opendev.org/70642403:09
*** dangtrinhnt has quit IRC03:12
*** apetrich has quit IRC03:13
*** dangtrinhnt has joined #openstack-infra03:13
*** rlandy has quit IRC03:16
*** dangtrinhnt has quit IRC03:17
*** dangtrinhnt has joined #openstack-infra03:18
*** jamesmcarthur has quit IRC03:31
*** jamesmcarthur has joined #openstack-infra03:32
*** psachin has joined #openstack-infra03:35
*** sgw has quit IRC03:55
*** armax has quit IRC03:56
*** jamesmcarthur has quit IRC03:59
*** jamesmcarthur has joined #openstack-infra04:00
*** jamesmcarthur has quit IRC04:04
*** tetsuro has quit IRC04:13
*** tetsuro has joined #openstack-infra04:13
*** ykarel|away is now known as ykarel04:16
openstackgerritIan Wienand proposed opendev/system-config master: Replace nb03.openstack.org  https://review.opendev.org/70642804:16
ianwkevinz / clarkb: ^ i'm about out of time to babysit that -- but it should be ready.  i've attached a large partition to it.  maybe if you have time, merge it, purge the old ssh key from bridge.o.o and it should ... just work?04:22
*** raukadah is now known as chkumar|rover04:38
*** udesale has joined #openstack-infra04:49
*** jamesmcarthur has joined #openstack-infra05:01
*** jamesmcarthur has quit IRC05:06
*** dangtrinhnt has quit IRC05:13
*** dangtrinhnt has joined #openstack-infra05:14
*** dangtrinhnt has quit IRC05:19
*** gmann has quit IRC05:27
*** dangtrinhnt has joined #openstack-infra05:31
*** evrardjp has quit IRC05:34
*** evrardjp has joined #openstack-infra05:34
*** dangtrinhnt has quit IRC05:35
*** dangtrinhnt has joined #openstack-infra05:38
*** drifterza has quit IRC05:44
*** ociuhandu has joined #openstack-infra05:53
*** ociuhandu has quit IRC05:57
*** owalsh has quit IRC06:01
*** owalsh has joined #openstack-infra06:04
*** dangtrinhnt has quit IRC06:21
*** sgw has joined #openstack-infra06:23
*** lmiccini has joined #openstack-infra06:28
*** Lucas_Gray has joined #openstack-infra06:31
*** jamesmcarthur has joined #openstack-infra06:52
*** jamesmcarthur has quit IRC06:54
*** dkehn has quit IRC07:03
*** ricolin has quit IRC07:11
*** Lucas_Gray has quit IRC07:13
*** Lucas_Gray has joined #openstack-infra07:14
*** dtantsur|afk is now known as dtantsur07:27
*** dpawlik has joined #openstack-infra07:28
*** lpetrut has joined #openstack-infra07:29
*** lpetrut has quit IRC07:30
*** lpetrut has joined #openstack-infra07:31
*** ccamacho has joined #openstack-infra07:35
*** dpawlik has quit IRC07:44
*** dpawlik has joined #openstack-infra07:44
*** pgaxatte has joined #openstack-infra07:49
*** Lucas_Gray has quit IRC07:52
*** Lucas_Gray has joined #openstack-infra07:56
evrardjpis it me or is gerrit in a bad state?07:58
evrardjp(a little bit slower than usual)07:59
*** dychen has quit IRC08:01
*** slaweq__ has joined #openstack-infra08:09
*** rpittau|afk is now known as rpittau08:10
*** tkajinam has quit IRC08:12
*** ykarel is now known as ykarel|lunch08:12
*** andrewbonney has joined #openstack-infra08:16
*** carli has joined #openstack-infra08:17
*** Lucas_Gray has quit IRC08:17
*** slaweq has joined #openstack-infra08:18
*** gfidente has joined #openstack-infra08:18
*** iurygregory has joined #openstack-infra08:19
*** slaweq__ has quit IRC08:19
*** Lucas_Gray has joined #openstack-infra08:22
*** tesseract has joined #openstack-infra08:36
*** ralonsoh has joined #openstack-infra08:41
*** xek has joined #openstack-infra08:44
*** tetsuro has quit IRC08:46
*** tosky has joined #openstack-infra08:48
*** jpena|off is now known as jpena08:52
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Make revoke-sudo more general.  https://review.opendev.org/70626208:56
*** apetrich has joined #openstack-infra09:02
yoctozeptohiyah infra09:08
yoctozeptohttps://bugs.launchpad.net/tempest/+bug/1862240/comments/509:08
openstackLaunchpad bug 1862240 in Designate "Replace Tempest tox deprecated 'all-plugin' env to 'all' env" [Undecided,In progress] - Assigned to Ghanshyam Mann (ghanshyammann)09:08
yoctozeptois it possible to tweak codesearch to search other branches?09:08
*** ykarel|lunch is now known as ykarel09:10
*** lucasagomes has joined #openstack-infra09:10
*** david-lyle has quit IRC09:16
*** david-lyle has joined #openstack-infra09:16
*** pkopec has joined #openstack-infra09:22
*** armax has joined #openstack-infra09:31
openstackgerritThierry Carrez proposed openstack/project-config master: Do not post comments in release-approval pipeline  https://review.opendev.org/70645309:42
*** ociuhandu has joined #openstack-infra09:45
*** derekh has joined #openstack-infra09:46
frickleryoctozepto: IIUC the answer is no, I think you'd have to do local checkouts for that09:51
*** ociuhandu has quit IRC09:51
yoctozeptofrickler: yeah, I was afraid it might be the case; kinda painful this way - looking to modifying codesearch to allow that but need infra assistance with that anyhow09:53
yoctozeptofrickler: this is future-proofing, for now we can let teams know via ml how to fix their projects09:53
yoctozepto(if they read [all] or [qa] :-) )09:54
frickleryoctozepto: I think we don't want to spend a lot of efforts on codesearch anymore since it is planned to be replaced by search within gitea once we get that working properly09:55
yoctozeptowe are essentially forced to pay lots of technical debt atm ;d09:55
yoctozeptofrickler: fine by me, though that "once" has no eta for some time :-)09:56
frickleriirc this is waiting for some upstream work, but I didn't track that closely09:57
*** ociuhandu has joined #openstack-infra10:02
AJaegeryoctozepto: my workflow in those cases: Search in master, fix master - and try backporting ;)10:08
AJaegeryoctozepto: https://opendev.org/explore/code?q=all-plugin10:08
AJaegerthat one works, open the file and check out older versions10:09
yoctozeptoAJaeger: yeah, normally yes; the problem here is they may have this set only in stable branches10:09
yoctozeptoAJaeger: gitea search is lame atm, it ignores '-' and not supports regexp and/or file matchers10:10
yoctozeptoso no - this is no solution here :-)10:10
AJaegeryoctozepto: no solution - just workarounds ;)10:10
yoctozeptoAJaeger: not quite useful but thanks for trying!10:10
AJaegeryoctozepto: https://opendev.org/openstack/project-config/src/branch/master/tools/batch_update_projects.py was done to update all branches of all repos10:11
AJaegeryoctozepto: copy it, and adapt it10:11
AJaegeryoctozepto: So, you can use it to search and just print "Found something on branch X" and not change - or do a full automatic search & replace10:12
AJaegeryoctozepto: still a workaround ;) Another tool10:12
*** slaweq has quit IRC10:13
yoctozeptoAJaeger: oh, nice, the bad thing is it does no cloning and only acts on your server10:15
yoctozeptoyeah, I'm picky today :-)10:15
*** slaweq has joined #openstack-infra10:15
AJaegeryoctozepto: you don't have all of openstack checked out? ;)10:16
AJaegeryoctozepto: want my script?10:16
yoctozeptoAJaeger: well, how to say this... I don't feel like a mirror :-) if you have one, sure10:17
yoctozeptofrickler: https://review.opendev.org/706385 looked sane but that did not help10:21
yoctozeptofrickler: "mist" issues ;-)10:22
AJaegeryoctozepto: http://paste.openstack.org/show/789283/10:32
AJaegeryoctozepto: that's my own tool ;)10:32
yoctozeptoAJaeger: why not in some repo? :-)10:33
yoctozeptoAJaeger: thanks for sharing ;-)10:33
yoctozeptoanyways, new problem with zuul10:33
yoctozeptohttps://review.opendev.org/70605910:33
yoctozeptowhy is it not gating on any k-a jobs now?10:33
yoctozeptothe "touched" logic b0rk again? ;/10:34
yoctozeptoignore me, we quickly found the culprit with mgoddard10:35
yoctozeptobase job ignores \. so .zuul too10:35
yoctozepto;D10:35
openstackgerritJan Kubovy proposed zuul/zuul master: WIP: Store unparsed branch config in Zookeeper  https://review.opendev.org/70571610:40
openstackgerritRiccardo Pittau proposed openstack/diskimage-builder master: Allow python3 to be used in Debian  https://review.opendev.org/70592010:41
*** priteau has joined #openstack-infra10:45
*** slaweq_ has joined #openstack-infra10:51
*** slaweq has quit IRC10:53
*** dSrinivas has quit IRC10:55
*** bauzas has quit IRC10:57
*** bauzas has joined #openstack-infra10:57
*** ahosam has joined #openstack-infra11:03
*** rpittau is now known as rpittau|bbl11:18
yoctozeptoAJaeger: btw - how much do all those repos weigh?11:53
*** pgaxatte has quit IRC11:53
*** udesale_ has joined #openstack-infra12:04
*** udesale has quit IRC12:07
*** gmann has joined #openstack-infra12:08
openstackgerritThierry Carrez proposed openstack/project-config master: Fix ACLs for deliverables no longer under RelMgt  https://review.opendev.org/70647912:08
*** slaweq__ has joined #openstack-infra12:10
*** slaweq_ has quit IRC12:11
*** nicolasbock has joined #openstack-infra12:13
*** rfolco has joined #openstack-infra12:16
openstackgerritThierry Carrez proposed openstack/project-config master: Fix ACLs for deliverables no longer under RelMgt  https://review.opendev.org/70647912:23
*** pgaxatte has joined #openstack-infra12:23
*** ociuhandu has quit IRC12:27
*** ociuhandu has joined #openstack-infra12:28
*** jpena is now known as jpena|lunch12:29
mordredttx: something is a little weird with the label though - other labels have a range12:32
ttxmordred: yeah... somehow it translated +0..+1 into ALLOW12:32
mordredttx: I see it12:33
ttxhttps://opendev.org/openstack/project-config/src/branch/master/gerrit/acls/openstack/releases.config#L412:34
ttxMaybe I should just try to make it -1..+1 even if nobody can vote -112:34
openstackgerritMonty Taylor proposed openstack/project-config master: Add Label definition for PTL-Approved  https://review.opendev.org/70648812:36
mordredttx: maybe somethinng like that ^^12:36
mordredttx: (was looking at governance and the custom labels there)12:36
ttxah, yes12:36
mordredAJaeger: ^^ if you have a sec?12:37
ttxmight fail alphaordering, we'll see12:37
AJaegeryoctozepto: 8.8 GB in total is my working tree12:37
AJaegermordred: I agree with ttx, needs alphaordering12:38
mordredwhat needs alphaordering?12:39
mordredOH12:39
mordredgotit12:39
AJaegerthe sections12:39
AJaegerput it before "[receive]"12:39
openstackgerritMonty Taylor proposed openstack/project-config master: Add Label definition for PTL-Approved  https://review.opendev.org/70648812:39
AJaegermordred: +212:40
mordredttx: let's not land your tiurn-off-comments patch until we see zuul voting correctly12:40
ttxack12:40
yoctozeptoAJaeger: ack, thanks12:40
mordredShrews: got a sec and feel like a +A on that ^^ ?12:41
Shrewsnever12:41
*** Wryhder has joined #openstack-infra12:42
*** ociuhandu has quit IRC12:42
*** Lucas_Gray has quit IRC12:43
*** Wryhder is now known as Lucas_Gray12:43
AJaegerShrews: thanks for doing it anyway ;)12:44
AJaegeryoctozepto: majority is openstack/ - 6 GB12:44
AJaeger(that might include a few venvs)12:44
*** Lucas_Gray has quit IRC12:49
*** dSrinivas has joined #openstack-infra12:50
dSrinivasHi All, After Uploading the nodepool image to devstack. Created an instance with the nodepool image and I am unable to login to the instance with the Jenkins private key.12:52
dSrinivasfor nodepool image is it creating an jenkins user or other user12:53
dSrinivasfor bionic image12:53
yoctozeptoAJaeger: running the checkout for openstack now12:57
yoctozeptoAJaeger: so it requires project-config only to exist12:57
yoctozeptoand then it can handle stuff12:57
mordredShrews: how were the storms when they passed through Durham?12:58
*** ykarel is now known as ykarel|afk12:58
Shrewsmordred: LOTS of rain, a few tornado scares, and very heavy winds.12:59
openstackgerritMerged openstack/project-config master: Add Label definition for PTL-Approved  https://review.opendev.org/70648812:59
Shrewsmordred: gusting in the 40s right now. sounds like a train outside the house. power just flickered, so i expect to lose it soon13:00
*** Lucas_Gray has joined #openstack-infra13:01
mordredShrews: *lovely* - we had some really killer thunderstorms ... and *hail*13:04
mordredin other news though- the nice human from AT&T came and reattached the light to the light tube so I have non-phone internet again \o/13:04
*** rlandy has joined #openstack-infra13:05
*** jtomasek_ has joined #openstack-infra13:05
Shrewsmordred: i suppose that beats airplane wifi either way13:06
*** jtomasek has quit IRC13:06
*** ociuhandu has joined #openstack-infra13:08
*** rh-jelabarre has joined #openstack-infra13:08
*** jtomasek_ has quit IRC13:08
*** jtomasek has joined #openstack-infra13:09
mordredyas13:10
mordredI've also now learned (because corvus asked yesterday) - my new place is 1m above sea level13:11
dSrinivasmordred: Hi13:13
*** slaweq has joined #openstack-infra13:13
*** rpittau|bbl is now known as rpittau13:14
*** slaweq__ has quit IRC13:15
*** hongbin has joined #openstack-infra13:15
openstackgerritMerged zuul/zuul-jobs master: Add event id to emit-job-header  https://review.opendev.org/70622513:17
*** dtantsur is now known as dtantsur|brb13:26
*** Lucas_Gray has quit IRC13:28
*** Lucas_Gray has joined #openstack-infra13:29
*** jamesmcarthur has joined #openstack-infra13:31
openstackgerritRiccardo Pittau proposed openstack/diskimage-builder master: [WIP] Allow python3 to be used in Debian  https://review.opendev.org/70592013:31
ttxmordred: sounds very safe13:33
fricklerany tox experts around that can explain why this passes without executing any tests? I'd expect tox to fail in this situation instead. I only stumbled about it because the runtime of the tests was much lower than for the other versions13:33
fricklerhttps://zuul.opendev.org/t/openstack/build/4483dd93c6374081a57200d418d9a882/log/job-output.txt#67813:33
*** aedc has joined #openstack-infra13:34
ttxmordred: your change fixed the PTL-Approved label. I think we can push https://review.opendev.org/#/c/706453/ now13:34
fungiyoctozepto: frickler: the current codesearch is running https://github.com/hound-search/hound which, last i checked, is hard-coded to only index one branch (master). i'm pretty sure the repo search in gitea also only indexes master branches of projects though13:37
AJaegeryoctozepto: yeah, parses project-config to get the non-retired repos ;)13:37
*** jpena|lunch is now known as jpena13:37
fungiShrews: mordred: so far the water is staying on the other side of our bulkhead where it belongs13:38
fungibut staying up all night to check on it every couple of hours was annoying. i need to set up a depth gauge and some flood alarms13:39
fungiwe only had a few brownouts last night, though they lost power for a few hours in kitty hawk. also i didn't hear of any tornado or waterspout sightings. all in all seems fairly mild compared to what it could be13:39
*** gmann has quit IRC13:40
mordredttx: woot!13:41
*** gmann has joined #openstack-infra13:42
mordredfungi: that's excellent news! and also yes - you should set up a depth gauge and flood alarms13:42
fungittx: one thing you'll probably want to test with 706453 is whether that also prevents zuul from adding the vote, since that's done as part of commenting (and i don't see other examples in there of us using comment: false with gerrit)13:42
mordredbecause staying up all night watching a water level is yuck13:42
mordredfungi: good point13:43
*** jamesmcarthur has quit IRC13:44
*** jamesmcarthur has joined #openstack-infra13:45
*** ykarel|afk is now known as ykarel13:48
*** jamesmcarthur has quit IRC13:50
*** jamesmcarthur has joined #openstack-infra13:52
openstackgerritMerged openstack/project-config master: Do not post comments in release-approval pipeline  https://review.opendev.org/70645313:53
*** hongbin has quit IRC13:54
*** yamamoto has joined #openstack-infra13:58
*** eharney has joined #openstack-infra13:59
*** aedc has quit IRC14:00
openstackgerritTobias Henkel proposed zuul/zuul master: Offload setRefs to multiple processes  https://review.opendev.org/70651814:09
*** lbragstad has quit IRC14:17
*** tesseract-RH has joined #openstack-infra14:19
*** lbragstad has joined #openstack-infra14:19
*** tesseract has quit IRC14:21
fungi#status log requested delisting of ipv6 address for graphite01.opendev.org from the spamhaus sbl/css14:24
openstackstatusfungi: finished logging14:24
fungi#status log requested delisting of ipv4 address for pbx01.opendev.org from the spamhaus pbl14:28
openstackstatusfungi: finished logging14:28
*** piotrowskim has quit IRC14:30
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Fix broken setRefs whith missing objects  https://review.opendev.org/62166714:30
yoctozeptoAJaeger: clean openstack is 4G14:30
*** aedc has joined #openstack-infra14:31
*** aedc has quit IRC14:34
*** aedc has joined #openstack-infra14:34
*** tesseract has joined #openstack-infra14:34
*** Lucas_Gray has quit IRC14:35
*** Lucas_Gray has joined #openstack-infra14:35
*** ociuhandu has quit IRC14:35
AJaegeryoctozepto: so, you've checked it out? ;)14:41
yoctozeptoAJaeger: yeah, thanks14:42
*** aedc has quit IRC14:42
*** dSrinivas has quit IRC14:42
*** tesseract has quit IRC14:43
*** tesseract-RH has quit IRC14:43
*** tesseract has joined #openstack-infra14:44
openstackgerritGhanshyam Mann proposed openstack/devstack-gate master: Use stable constraints in Tempest tox till stable/rocky  https://review.opendev.org/70617814:51
*** Goneri has joined #openstack-infra14:57
*** zxiiro has joined #openstack-infra15:00
*** rpittau is now known as rpittau|brb15:04
*** lpetrut has quit IRC15:08
*** ykarel is now known as ykarel|away15:09
*** pkopec has quit IRC15:09
*** yamamoto has quit IRC15:12
*** Lucas_Gray has quit IRC15:14
*** ociuhandu has joined #openstack-infra15:15
*** yamamoto has joined #openstack-infra15:15
*** roman_g has joined #openstack-infra15:16
openstackgerritJames E. Blair proposed zuul/zuul master: Don't run jobs if only their file matchers are updated  https://review.opendev.org/70639915:16
*** ociuhandu has quit IRC15:16
*** ociuhandu has joined #openstack-infra15:17
*** yamamoto has quit IRC15:20
*** rpittau|brb is now known as rpittau15:20
*** udesale_ has quit IRC15:23
*** udesale_ has joined #openstack-infra15:24
openstackgerritJeremy Stanley proposed opendev/system-config master: Add missing Lets Encrypt certs to ssldomains list  https://review.opendev.org/70654315:31
fungiinfra-root: ^ we need to remember to add https sites to our certcheck. i've already found one stale le cert we weren't testing15:31
*** ociuhandu_ has joined #openstack-infra15:33
mordredfungi: we should maybe add a short blurb about remembering to do that to the letsencrypt doc15:34
fungigood call15:34
*** ociuhandu has quit IRC15:37
*** igordc has joined #openstack-infra15:37
*** jamesmcarthur has quit IRC15:38
*** jamesmcarthur has joined #openstack-infra15:38
*** ociuhandu_ has quit IRC15:40
corvusi bet that the data are structured enough we could write a linter that checks that all the letsencrypt hosts are in the sslcheck list15:42
corvus(parsing hostvars files, etc)15:42
corvusyou know, if someone's bored on a sunday or something15:42
*** ociuhandu has joined #openstack-infra15:42
mordredor killing time between checking flood water levels15:43
*** lmiccini has quit IRC15:46
*** Goneri has quit IRC15:46
*** ociuhandu has quit IRC15:47
fungiwe'll see. i'm trying to grok enough of our new static hosting solution to take a stab at setting up the docs.airshipit.org site after i saw their e-mail lodged in the moderation queue for the past couple days15:49
fungiwhich is how i discovered the missing cert entries15:49
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: test-registry-buildset-registry-k8s-crio: increase wait timeout  https://review.opendev.org/70655115:51
*** yamamoto has joined #openstack-infra15:53
*** david-lyle is now known as dklyle15:53
*** bnemec has quit IRC15:55
*** bnemec has joined #openstack-infra15:56
*** eernst has joined #openstack-infra15:58
*** yamamoto has quit IRC15:58
*** eernst has quit IRC15:58
*** jackedin has joined #openstack-infra15:59
openstackgerritMerged zuul/zuul-jobs master: fetch-output-openshift: initial role  https://review.opendev.org/68204416:01
mordredinfra-root: https://review.opendev.org/#/c/705804 is good to go and I'm around to watch it16:02
clarkbmordred: k, we may need to trigger replication when it is done16:03
AJaegerregarding watching: ianw left a comment before he signed of that https://review.opendev.org/#/c/706428/ - in case somebody is around to merge and watch...16:04
mordredclarkb: good point16:05
clarkbAJaeger: thanks. I can watch that one from ianw if it lands16:05
mordredclarkb: 2nd +2 from me - land whenever16:07
clarkbthanks16:07
*** jamesmcarthur has quit IRC16:10
openstackgerritMonty Taylor proposed opendev/system-config master: Upgrade gitea to 1.11.0-rc  https://review.opendev.org/70580716:11
*** jamesmcarthur has joined #openstack-infra16:12
*** chkumar|rover is now known as raukadah16:14
fungiokay, the yard is inundated and has reached the edge of the slab/patio so i'll probably be disappearing for a while here to keep a closer eye on it, relocate the car, et cetera16:16
*** mattw4 has joined #openstack-infra16:16
*** jamesmcarthur has quit IRC16:17
clarkbfungi: good luck16:17
mordredfungi: fingers crossed16:21
fungithanks!16:21
*** jamesmcarthur has joined #openstack-infra16:22
openstackgerritMerged opendev/system-config master: Add missing Lets Encrypt certs to ssldomains list  https://review.opendev.org/70654316:22
openstackgerritMonty Taylor proposed opendev/system-config master: WIP Upgrade gitea to master  https://review.opendev.org/70580816:23
clarkbI'm launching a mirror in the airship citycloud region now16:26
clarkbwe got quotas sorted out16:27
*** jamesmcarthur has quit IRC16:27
*** pgaxatte has quit IRC16:32
*** slaweq has quit IRC16:37
*** tesseract has quit IRC16:38
openstackgerritMerged opendev/system-config master: Replace nb03.openstack.org  https://review.opendev.org/70642816:39
*** xek has quit IRC16:39
*** carli has quit IRC16:43
*** gyee has joined #openstack-infra16:44
*** udesale_ has quit IRC16:45
*** udesale_ has joined #openstack-infra16:46
openstackgerritMerged zuul/zuul master: Offload setRefs to multiple processes  https://review.opendev.org/70651816:47
*** udesale_ has quit IRC16:48
*** udesale_ has joined #openstack-infra16:48
openstackgerritJames E. Blair proposed zuul/zuul master: Fix test race with paused build  https://review.opendev.org/70656616:49
openstackgerritJames E. Blair proposed zuul/zuul master: WIP: see if we can remove .paused from test fake build  https://review.opendev.org/70656716:49
*** udesale_ has quit IRC16:50
*** udesale_ has joined #openstack-infra16:51
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: WIP: add an ensure-tox test job  https://review.opendev.org/70637116:52
*** lucasagomes has quit IRC16:52
clarkbunattended upgrades is taking a long time on this server. I wonder how stale the cloud image is16:52
clarkbthe good news is we update before we put anything in production :)16:52
*** hongbin has joined #openstack-infra16:55
openstackgerritClark Boylan proposed opendev/zone-opendev.org master: Add airship citycloud mirror node  https://review.opendev.org/70657316:55
openstackgerritClark Boylan proposed opendev/system-config master: Add airship-citycloud mirror  https://review.opendev.org/70657416:55
clarkbroman_g: ^ fyi I think that is the last step before turning the cloud on16:56
clarkbinfra-root ^ those should be ready to review16:56
*** tesseract has joined #openstack-infra16:57
mordredclarkb: lgtm16:57
clarkbtyty16:58
*** dpawlik has quit IRC16:58
*** hongbin has quit IRC16:59
corvusclarkb: ooh, we're already getting cronspam bounces17:02
*** rpittau is now known as rpittau|afk17:02
mordredthat's the mark of success, right?17:02
*** yamamoto has joined #openstack-infra17:04
clarkbsuccessful failure17:04
*** jamesmcarthur has joined #openstack-infra17:07
openstackgerritMerged opendev/zone-opendev.org master: Add airship citycloud mirror node  https://review.opendev.org/70657317:07
*** yamamoto has quit IRC17:08
openstackgerritRiccardo Pittau proposed openstack/diskimage-builder master: Allow python3 to be used in Debian  https://review.opendev.org/70592017:11
smcginnisMaybe a glitch, but wanted to point out we had a release note publish job failure in "Synchornize files to AFS".17:15
smcginnishttps://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_4c0/11c050368e99df52d64837c711f36b90da6ba974/tag/publish-openstack-releasenotes-python3/4c0b8e7/job-output.txt17:15
openstackgerritMerged zuul/nodepool master: add ebs-optimized support for aws provider  https://review.opendev.org/70575517:15
smcginnisrsync: close failed on "/afs/.openstack.org/docs/releasenotes/ironic-ui/fr/_static/fonts/.glyphicons-halflings-regular.woff.Kto7mJ": No such file or directory (2)17:15
clarkbsmcginnis: is it possible there were multiple writers of that file?17:15
clarkbif there are two rsyncs copying the same files one of them does the cleanup and the other gets mad but it should mostly come out correct17:16
smcginnisWe've had similar cases in the past due to that. It's been awhile though.17:17
smcginnisLet me see what else was running through at the time...17:17
smcginnisYeah, looks like that maybe could be it. There were a few others running right around that same time.17:18
smcginnishttps://zuul.opendev.org/t/openstack/builds?job_name=publish-openstack-releasenotes-python317:18
*** jamesmcarthur has quit IRC17:20
*** jamesmcarthur has joined #openstack-infra17:21
*** psachin has quit IRC17:21
*** jamesmcarthur has quit IRC17:27
*** Goneri has joined #openstack-infra17:28
*** jamesmcarthur has joined #openstack-infra17:29
*** dtantsur|brb is now known as dtantsur17:31
*** udesale_ has quit IRC17:32
*** rlandy is now known as rlandy|brb17:32
*** jamesmcarthur has quit IRC17:33
*** jamesmcarthur has joined #openstack-infra17:33
*** evrardjp has quit IRC17:34
*** evrardjp has joined #openstack-infra17:34
*** tesseract has quit IRC17:36
*** jamesmcarthur has quit IRC17:38
*** bdodd has quit IRC17:39
*** yamamoto has joined #openstack-infra17:40
*** bdodd has joined #openstack-infra17:40
*** tosky has quit IRC17:40
*** jamesmcarthur has joined #openstack-infra17:41
* mordred goes to find a sandwich17:42
*** jamesmcarthur has quit IRC17:43
*** jamesmcarthur has joined #openstack-infra17:44
*** yamamoto has quit IRC17:44
*** jamesmcarthur has quit IRC17:48
*** ociuhandu has joined #openstack-infra17:50
*** jamesmcarthur has joined #openstack-infra17:53
*** diablo_rojo has joined #openstack-infra17:55
clarkbnow that I've acepted the ssh hostkey for the new mirror we should actually get ansible to ru nthere17:56
clarkbI always forget that step17:56
*** ociuhandu has quit IRC17:56
*** priteau has quit IRC17:56
fungi#status log increased the /afs/.openstack.org/project/zuul-ci.org quota from 100000kb to 1000000kb17:57
openstackstatusfungi: finished logging17:57
openstackgerritJames E. Blair proposed zuul/nodepool master: WIP: Add ability to specify pod service accounts  https://review.opendev.org/70659617:57
fungi#status log created new project.airship volume mounted at /afs/.openstack.org/project/airshipit.org with two read-only replicas and set quota consistent with project.zuul and project.starlingx volumes17:58
openstackstatusfungi: finished logging17:58
*** jamesmcarthur has quit IRC18:02
*** jpena is now known as jpena|off18:02
*** jamesmcarthur has joined #openstack-infra18:04
*** rlandy|brb is now known as rlandy18:05
openstackgerritMerged opendev/system-config master: Add airship-citycloud mirror  https://review.opendev.org/70657418:05
*** andrewbonney has quit IRC18:05
*** derekh has quit IRC18:05
fungihow do i set an initial acl on an afs path? fs setacl is complaining "You don't have the required access rights on ..."18:11
clarkbfungi: you are using your admin account I assume?18:14
fungihah, yes and no ;)18:15
fungii for got to `aklog`18:15
* fungi sighs at distractedness18:15
fungiokay, acls set on the new volume now18:16
*** igordc has quit IRC18:18
openstackgerritJeremy Stanley proposed openstack/project-config master: Add a job to promote docs.airshipit.org content  https://review.opendev.org/70659818:19
openstackgerritJeremy Stanley proposed opendev/system-config master: Set up LE certs for docs.airshipit.org static site  https://review.opendev.org/70660018:23
openstackgerritJeremy Stanley proposed opendev/system-config master: Add a new docs.airshipit.org vhost on static01  https://review.opendev.org/70660118:23
*** artom has quit IRC18:25
*** artom has joined #openstack-infra18:25
*** ahosam has quit IRC18:27
*** jamesmcarthur has quit IRC18:27
AJaegerfungi: https://review.opendev.org/#/c/706598/ LGTM - want to create an extra file like we did for starlingx in case more jobs show up?18:29
fungiAJaeger: the job is going to publish to /docs within there and the vhost sets that as the docroot18:30
fungistarlingx's is a little backwards in that they called their docs tree "/www"18:30
fungithis is set up so that *if* they also decide to start publishing their main airshipit.org site with the same tooling in the future, they can put that in /www or something18:31
fungiand have it share one volume18:32
AJaegerfungi: No, I mean move from zuul.d/jobs.yaml to zuul.d/airshipit.yaml18:32
fungioh18:32
AJaegerand the job will not work as is, let me comment.18:32
fungithanks, i'm trying to learn the new solution myself, which is why i jumped into this18:32
*** jamesmcarthur has joined #openstack-infra18:33
AJaegermmh, digging deeper...18:33
AJaegerfungi: yeah, they are tricky ;(18:33
AJaegerfungi: left a comment, check how starlingx does it18:35
*** jamesmcarthur has quit IRC18:44
*** dtantsur is now known as dtantsur|afk18:45
*** igordc has joined #openstack-infra18:49
*** aedc has joined #openstack-infra18:55
*** yamamoto has joined #openstack-infra18:56
clarkbLE verification seems to have failed for the new mirror for some reason.18:56
clarkbthe log message isn't incredibly clear18:56
clarkbit almost looks like it was using the web based auth rather than dns18:57
mordredclarkb: hrm.18:58
mordredclarkb: it ran via the normal cron?18:58
clarkbmordred: ya18:59
*** imacdonn has quit IRC19:00
*** yamamoto has quit IRC19:00
*** imacdonn has joined #openstack-infra19:01
clarkbI've double checked the acme challenge records are in dns for the two domains the acme.sh command tried to issue certs for19:02
clarkbthat all looks good. There is also a TXT record in the acme.opendev.org zone file from around when this was happening19:02
mordredclarkb: I don't see anything obviously wrong with your patch that's different than the other things19:02
clarkbok the acme_txt_required entry is empty for the new mirror19:05
clarkblooking in the ansible run all log on bridge19:05
clarkbI think that is the underlying issue19:06
clarkbat this point maybe it was racy with dns updates and things?19:06
clarkband see if next pulse is happier?19:06
mordredclarkb: seems reasonable19:07
clarkbbut ya for it to verify the new certs I'm 99% sure that acme_txt_required needed to have something in it, then that would update adns1's zonefile and that would allow us to verify the cert19:08
clarkband it being empty prevents verification19:08
clarkbfungi: additionally it looks like static01 had a sad LE posisbly due to airshipit.org updates?19:08
clarkb(not sure you are that far along yet though)19:09
*** gfidente is now known as gfidente|afk19:11
fungii am not that far along yet, no19:12
clarkbre new nb03 it failed to pip install nodepool due to ruamel.yaml.clib package install failing for some reason19:13
clarkbFeb  7 18:01:06 nb03 puppet-user[23146]: (/Stage[main]/Nodepool/Exec[install_nodepool]/returns) ERROR: Files/directories not found in /tmp/pip-install-qx54nt1d/ruamel.yaml.clib/pip-egg-info19:13
clarkbis ruamel a new dep?19:13
Shrewsnope. never heard of it19:14
clarkbI think the issue there may be that that package doesn't have arm64 version?19:14
fungiyesterday i saw ianw and corvus discuss using it for regenerating parts of configuration19:14
fungipresumably so that comments and ordering would be retained19:15
clarkbeverything else would be installing the x86-64 wheels19:15
clarkbbut on the arm builder we grab the source sdist19:15
fungiso probably not a runtime dependency, but maybe listed in the deps out of convenience?19:15
clarkbruamel.yaml.clib.egg-info/ is the path not pip-egg-info/ if I look in that sdist19:16
*** igordc has quit IRC19:19
clarkbnot able to reproduce that with `pip install --no-binary :all: --no-cache-dir ruamel.yaml.clib` though19:21
clarkbmordred: ^ any idea what that means in a python package context?19:26
mordredclarkb: nope. I'm guessing it has something to do with PACKAGE.in listing something that then doesn't exist?19:28
mordredbut if you can't reproduce via a pip install, that's confusing19:29
*** yamamoto has joined #openstack-infra19:29
clarkbwell I can reproduce it on that server19:30
clarkbbut not on my laptop19:30
clarkbalso I downgraded to pip==18.0 from 20.0.2 because that is what the old arm builder had and tried again with no luck19:30
corvusis ruamel a transitive dep?19:31
clarkbcorvus: yes19:31
clarkb(I havne't sorted out where from yet, but nodepool doesn't explicitly require it)19:31
clarkbmy hunch is this is something arm64 related19:32
clarkbsince I fail to reproduce at all on x8619:32
clarkbhrm maybe it is looking for arch specific files?19:32
* clarkb looks in the sdist deeper19:32
*** dpawlik has joined #openstack-infra19:33
*** yamamoto has quit IRC19:35
clarkbre the mirror this current ansible run has txt record content for the two mirror names now19:35
clarkband I have a ca.cer so maybe it was just a race19:36
clarkbthat is one serious setup.py19:38
openstackgerritJeremy Stanley proposed openstack/project-config master: Add a job to promote docs.airshipit.org content  https://review.opendev.org/70659819:40
clarkbhttps://mirror.kna1.airship-citycloud.opendev.org/ is up19:41
*** msdisme has joined #openstack-infra19:42
clarkbI've removed my WIP from https://review.opendev.org/#/c/706316/1 if we want to land that19:42
clarkband start using the new cloud region19:42
clarkbupdating setuptools seems to have fixed that ruamel issue19:45
mordredclarkb: joy19:46
clarkbianw: ^ fyi on that. setuptools-20.7.0 is not capable of installing nodepool due to ruamel.yaml.clib but setuptools-45.1.0 is19:46
*** xek has joined #openstack-infra19:47
clarkbI've manually updated setuptools and ruamel.yaml.clib on new nb03. I think puppet might need some convincing (remove nodepool source tree) to try and rerun everything though19:47
clarkbI'm going to pop out for a bit now and find lunch and stuff then will look and seeif ^ is necessary19:47
mordredclarkb, corvus: openshift<=0.8.9 is what pulls in ruamel.yaml fwiw19:47
*** jackedin has quit IRC19:48
*** jtomasek has quit IRC19:54
*** dpawlik has quit IRC19:55
*** jtomasek has joined #openstack-infra19:56
*** msdisme has left #openstack-infra19:59
*** ociuhandu has joined #openstack-infra20:16
*** xek has quit IRC20:19
*** ociuhandu has quit IRC20:21
*** aedc has quit IRC20:30
AJaegerfungi: reviewed your change. I was wrong, both versions work.20:37
AJaegerwhich one do you prefer? I think the first is better...20:37
*** rh-jelabarre has quit IRC20:40
*** dpawlik has joined #openstack-infra20:41
*** yamamoto has joined #openstack-infra20:47
*** lbragstad has quit IRC20:48
*** yamamoto has quit IRC20:52
fungithe first seems to involve a lot of copying around identical zuul secrets20:52
fungirather than being able to reuse the same one20:52
fungier, the second i mean20:52
fungiso you're saying you prefer patchset #1? if so i can switch back20:52
fungii see your comments now, so back to patchset #1. will do that momentarily. thanks!20:54
clarkblooks like we also need to remove python3-yaml from the new nodepool builder21:04
clarkbit is a distutils instlled package so the dib install fails on it21:04
clarkbI think cloud-init pulled it in21:04
clarkbI'm going to remove it now, then remove the git dirsfor dib and nodepool forcing puppet to rerun them21:04
clarkbok I expect next puppet run will do something useful now21:05
fungiyeah, the ubuntu package for cloud-init drags in ubuntu packaged pyyaml21:09
*** rfolco has quit IRC21:12
mordreddo we not have a remove-cloud-init task in our launch-node bootstrapping?21:12
clarkbmordred: we don't but I don't know that that removes its deps. you have to run apt-get autoremove to get parents package right?21:13
clarkb*we do21:13
*** Lucas_Gray has joined #openstack-infra21:14
*** owalsh has quit IRC21:16
*** dpawlik has quit IRC21:18
*** sshnaidm|afk has quit IRC21:18
*** sshnaidm|afk has joined #openstack-infra21:23
*** yamamoto has joined #openstack-infra21:25
openstackgerritJeremy Stanley proposed openstack/project-config master: Add a job to promote docs.airshipit.org content  https://review.opendev.org/70659821:25
clarkbmordred: its possible that if we left it alone for a day that the unattended upgrades cron would clean up the uneeded package for us21:26
mordredclarkb: this is a good point - and yeah, apt-get autoremove would almost certainly be needed21:26
* fungi concurs21:27
openstackgerritMonty Taylor proposed opendev/system-config master: Run apt-get autoremove after rm cloud-init  https://review.opendev.org/70662421:29
mordredclarkb, fungi: ^^ maybe that'll be useful?21:29
mordredfungi: how's the water?21:29
*** derekh has joined #openstack-infra21:29
*** derekh has quit IRC21:30
*** yamamoto has quit IRC21:30
fungireceding for the moment, but it's also (ostensibly) low tide21:32
mordredwell - that's good - although maybe not as good as it could be21:33
*** owalsh has joined #openstack-infra21:34
fungigonna go get pizza and then see if we think it's safe to keep the car here for the evening21:34
*** ahosam has joined #openstack-infra21:39
clarkbmordred: https://review.opendev.org/#/c/706178/5 gmann needs that to support installing tempest from tags on older stable releases21:39
clarkbparticularly for grenade jobs which still use d-g21:39
clarkbif oyu have a moment to look at that21:39
clarkbalso I think https://review.opendev.org/#/c/706316/ is ready with the mirror and quotas being set21:40
mordredclarkb: done21:41
clarkbok we only run set-hostnames on launch so that won't affect running servers21:41
clarkbI've approved the autoremove change21:41
*** ociuhandu has joined #openstack-infra21:44
*** ociuhandu has quit IRC21:47
*** ociuhandu has joined #openstack-infra21:47
openstackgerritMerged openstack/project-config master: Add capacity to airship CI cloud  https://review.opendev.org/70631621:51
clarkbianw: I updated https://storyboard.openstack.org/#!/story/2007195 with nb03 infos21:57
clarkbianw: I think that has gotten as far as a working installation but srevices are not running. I may leave it there for now. As weekend is approaching and I'm not sure what the transition between hosts should look likewitout some thought22:03
*** ociuhandu has quit IRC22:04
*** rlandy has quit IRC22:05
*** ociuhandu has joined #openstack-infra22:05
clarkbroman_g: you should be able to write jobs using that cloud now22:23
*** ahosam has quit IRC22:24
*** ociuhandu has quit IRC22:25
*** iurygregory has quit IRC22:29
*** tosky has joined #openstack-infra22:31
*** rh-jelabarre has joined #openstack-infra22:37
*** yamamoto has joined #openstack-infra22:39
*** yamamoto has quit IRC22:44
*** arif-ali has quit IRC22:47
*** arif-ali has joined #openstack-infra22:48
*** dpawlik has joined #openstack-infra22:51
*** dpawlik has quit IRC22:59
*** armax has quit IRC23:06
*** yamamoto has joined #openstack-infra23:12
*** jtomasek has quit IRC23:14
*** sshnaidm|afk has quit IRC23:16
*** yamamoto has quit IRC23:18
*** rh-jelabarre has quit IRC23:18
*** ralonsoh has quit IRC23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!