Monday, 2019-06-24

*** yamamoto has joined #openstack-infra00:06
*** rcernin has quit IRC00:06
*** yamamoto has quit IRC00:07
*** yamamoto has joined #openstack-infra00:08
*** eharney has quit IRC00:15
*** rcernin has joined #openstack-infra00:18
*** yamamoto has quit IRC00:29
*** ajo has quit IRC00:51
*** mwhahaha has quit IRC00:51
*** mwhahaha has joined #openstack-infra00:51
*** ajo has joined #openstack-infra00:51
*** masayukig has quit IRC00:51
*** mrhillsman has quit IRC00:51
*** niceplace has quit IRC00:52
*** masayukig has joined #openstack-infra00:53
*** zhurong has quit IRC00:53
*** niceplace has joined #openstack-infra00:54
*** gregoryo has joined #openstack-infra00:55
*** rcernin has quit IRC01:12
*** markvoelker has quit IRC01:14
*** markvoelker has joined #openstack-infra01:16
*** rcernin has joined #openstack-infra01:17
*** hwoarang has quit IRC01:18
*** yamamoto has joined #openstack-infra01:21
*** yamamoto has quit IRC01:22
*** yamamoto has joined #openstack-infra01:23
*** hwoarang has joined #openstack-infra01:24
*** yamamoto has quit IRC01:29
*** yamamoto has joined #openstack-infra01:30
*** yamamoto has quit IRC01:35
*** yamamoto has joined #openstack-infra01:35
*** yamamoto has quit IRC01:36
*** zhurong has joined #openstack-infra02:11
*** jhesketh has quit IRC02:12
*** jhesketh has joined #openstack-infra02:13
*** sgw has quit IRC02:33
*** bhavikdbavishi has joined #openstack-infra02:45
*** dklyle has quit IRC03:00
*** psachin has joined #openstack-infra03:15
*** bhavikdbavishi1 has joined #openstack-infra03:34
*** bhavikdbavishi has quit IRC03:35
*** bhavikdbavishi1 is now known as bhavikdbavishi03:35
*** ykarel has joined #openstack-infra03:49
*** whoami-rajat has joined #openstack-infra03:54
*** udesale has joined #openstack-infra04:01
*** ricolin has joined #openstack-infra04:19
*** ykarel has quit IRC04:23
*** ykarel has joined #openstack-infra04:41
*** adrianreza has joined #openstack-infra04:44
*** chandankumar has quit IRC05:06
*** chandankumar has joined #openstack-infra05:07
*** jbadiapa has joined #openstack-infra05:48
*** AJaeger has joined #openstack-infra05:57
*** sparkycollier_ has joined #openstack-infra06:07
*** ildikov_ has joined #openstack-infra06:07
*** evgenyl_ has joined #openstack-infra06:08
*** aprice_ has joined #openstack-infra06:08
*** coreycb_ has joined #openstack-infra06:08
*** tinwood_ has joined #openstack-infra06:10
openstackgerritOpenStack Proposal Bot proposed openstack/project-config master: Normalize projects.yaml  https://review.opendev.org/66701606:10
*** Anticime1 has joined #openstack-infra06:10
*** cmurphy has joined #openstack-infra06:10
*** lathiat has quit IRC06:12
*** gouthamr_ has joined #openstack-infra06:14
*** ddurst- has joined #openstack-infra06:14
*** virendra-sharma has joined #openstack-infra06:14
*** tinwood has quit IRC06:15
*** evgenyl has quit IRC06:15
*** sparkycollier has quit IRC06:15
*** coreycb has quit IRC06:15
*** smcginnis has quit IRC06:15
*** ddurst has quit IRC06:15
*** Anticimex has quit IRC06:15
*** aprice has quit IRC06:15
*** fnordahl has quit IRC06:15
*** ildikov has quit IRC06:15
*** gouthamr has quit IRC06:15
*** cmorpheus has quit IRC06:15
*** sparkycollier_ is now known as sparkycollier06:15
*** aprice_ is now known as aprice06:15
*** coreycb_ is now known as coreycb06:15
*** ildikov_ is now known as ildikov06:15
*** evgenyl_ is now known as evgenyl06:15
*** lathiat has joined #openstack-infra06:16
*** irclogbot_3 has quit IRC06:18
*** dpawlik has joined #openstack-infra06:18
*** panda has quit IRC06:18
*** JpMaxMan has quit IRC06:18
*** irclogbot_0 has joined #openstack-infra06:19
*** ccamacho has joined #openstack-infra06:19
*** JpMaxMan has joined #openstack-infra06:19
*** panda has joined #openstack-infra06:20
*** pcaruana has joined #openstack-infra06:25
*** virendra-sharma has quit IRC06:34
*** virendra-sharma has joined #openstack-infra06:38
*** ramishra has joined #openstack-infra06:44
virendra-sharmaCI watch (http://ciwatch.mmedvede.net/) is not updating, any alternate for it to monitor latest patchset logs?06:46
openstackgerritMerged openstack/project-config master: Normalize projects.yaml  https://review.opendev.org/66701606:48
*** lpetrut has joined #openstack-infra06:49
*** rcernin has quit IRC06:57
*** jtomasek has joined #openstack-infra06:58
*** e0ne has quit IRC06:59
*** pgaxatte has joined #openstack-infra07:00
*** iurygregory has joined #openstack-infra07:00
*** ginopc has joined #openstack-infra07:02
*** kopecmartin has joined #openstack-infra07:03
*** apetrich has joined #openstack-infra07:05
*** markvoelker has quit IRC07:07
*** jaosorior has joined #openstack-infra07:11
*** tesseract has joined #openstack-infra07:11
*** tosky has joined #openstack-infra07:14
*** ccamacho has quit IRC07:31
*** xek has joined #openstack-infra07:33
*** ginopc has quit IRC07:38
*** ccamacho has joined #openstack-infra07:39
*** trident has quit IRC07:40
*** aedc has joined #openstack-infra07:40
*** trident has joined #openstack-infra07:41
*** fnordahl has joined #openstack-infra07:44
*** ykarel is now known as ykarel|lunch07:48
*** sshnaidm|off has quit IRC07:48
*** lucasagomes has joined #openstack-infra07:49
*** sshnaidm has joined #openstack-infra07:49
*** aedc has quit IRC07:53
*** ralonsoh has joined #openstack-infra07:54
*** jpich has joined #openstack-infra08:05
*** jaosorior has quit IRC08:13
*** jaosorior has joined #openstack-infra08:17
*** gregoryo has quit IRC08:21
*** dchen has quit IRC08:23
*** yboaron_ has joined #openstack-infra08:25
*** e0ne has joined #openstack-infra08:27
*** tkajinam has quit IRC08:29
*** Emine has joined #openstack-infra08:32
*** imacdonn has quit IRC08:42
*** imacdonn has joined #openstack-infra08:42
*** factor has quit IRC08:43
*** whoami-rajat has quit IRC08:48
*** aedc has joined #openstack-infra08:48
*** ykarel|lunch is now known as ykarel08:52
*** jpich has quit IRC09:00
*** gfidente has joined #openstack-infra09:01
*** udesale has quit IRC09:02
*** udesale has joined #openstack-infra09:03
*** udesale has quit IRC09:04
*** markvoelker has joined #openstack-infra09:08
*** jpich has joined #openstack-infra09:08
*** udesale has joined #openstack-infra09:08
*** udesale has quit IRC09:13
*** udesale has joined #openstack-infra09:14
*** ricolin has quit IRC09:22
*** _Cyclone_ has quit IRC09:24
*** ociuhandu has joined #openstack-infra09:30
*** gfidente has quit IRC09:31
*** markvoelker has quit IRC09:42
*** roman_g has joined #openstack-infra09:47
openstackgerritMerged openstack/devstack-gate master: Add gate job for stable/stein  https://review.opendev.org/64791209:49
*** udesale has quit IRC09:51
*** udesale has joined #openstack-infra09:52
*** bstinson has quit IRC09:52
*** jpich has quit IRC09:52
*** jpich has joined #openstack-infra09:52
*** bstinson has joined #openstack-infra09:53
*** udesale has quit IRC09:56
*** udesale has joined #openstack-infra09:57
*** jpich has quit IRC09:57
openstackgerritStephen Finucane proposed openstack/pbr master: option to print only the version of a package  https://review.opendev.org/57761409:58
*** jpich has joined #openstack-infra09:58
*** hrww has joined #openstack-infra09:59
hrwwmorning09:59
hrwwcan someone take a look at https://review.opendev.org/#/c/666533/ patch? It adds mirroring of Train packages from UCA09:59
*** jpena|off is now known as jpena10:00
*** udesale has quit IRC10:00
*** udesale has joined #openstack-infra10:00
*** gfidente has joined #openstack-infra10:05
*** aedc has quit IRC10:07
*** ykarel is now known as ykarel|afk10:16
*** mnencia has quit IRC10:25
*** mnencia has joined #openstack-infra10:26
*** yamamoto has joined #openstack-infra10:32
*** rh-jelabarre has quit IRC10:33
*** bhavikdbavishi has quit IRC10:34
*** yamamoto has quit IRC10:36
*** rkukura has quit IRC10:37
*** markvoelker has joined #openstack-infra10:39
*** yboaron_ has quit IRC10:47
*** yboaron_ has joined #openstack-infra10:48
*** aedc has joined #openstack-infra10:51
*** ramishra has quit IRC10:58
*** dciabrin has quit IRC11:04
*** markvoelker has quit IRC11:13
zbrdo we have some presentation templates that we can use for openinfra presentations? if so where I can find them (google did not help)11:14
*** _erlon_ has joined #openstack-infra11:19
*** jpena is now known as jpena|lunch11:20
*** jaosorior_ has joined #openstack-infra11:21
*** dciabrin has joined #openstack-infra11:22
*** jaosorior has quit IRC11:23
*** ykarel|afk is now known as ykarel11:28
*** Lucas_Gray has joined #openstack-infra11:29
hrwwfrickler: thanks11:29
*** kjackal has quit IRC11:38
*** kjackal has joined #openstack-infra11:38
*** udesale has quit IRC11:39
*** rfolco has joined #openstack-infra11:46
*** rlandy has joined #openstack-infra11:55
*** apetrich has quit IRC11:56
*** jcoufal has joined #openstack-infra12:00
*** apetrich has joined #openstack-infra12:03
*** markvoelker has joined #openstack-infra12:04
*** ramishra has joined #openstack-infra12:06
*** ykarel is now known as ykarel|afk12:09
*** hrww has quit IRC12:09
*** virendra-sharma has quit IRC12:12
*** whoami-rajat has joined #openstack-infra12:14
openstackgerritTobias Henkel proposed zuul/zuul master: Fix build leak with paused jobs  https://review.opendev.org/66707212:15
*** ykarel|afk has quit IRC12:17
*** _Cyclone_ has joined #openstack-infra12:17
*** markvoelker has quit IRC12:24
*** rh-jelabarre has joined #openstack-infra12:26
*** jroll has quit IRC12:28
*** jroll has joined #openstack-infra12:29
*** hrww has joined #openstack-infra12:33
*** cyberpear has joined #openstack-infra12:36
*** rh-jelabarre has quit IRC12:37
*** rkukura has joined #openstack-infra12:38
*** rh-jelabarre has joined #openstack-infra12:40
*** AJaeger has quit IRC12:41
*** priteau has joined #openstack-infra12:42
*** ykarel|afk has joined #openstack-infra12:43
*** ykarel|afk is now known as ykarel12:43
*** georgk9 has quit IRC12:44
*** georgk has joined #openstack-infra12:44
*** beekneemech is now known as bnemec12:51
*** AJaeger has joined #openstack-infra12:53
*** zbr is now known as zbr|ruck12:53
*** aaronsheffield has joined #openstack-infra12:55
*** jpena|lunch is now known as jpena13:02
*** kjackal has quit IRC13:07
*** kjackal_v2 has joined #openstack-infra13:07
*** sthussey has joined #openstack-infra13:07
*** ekultails has joined #openstack-infra13:07
*** yamamoto has joined #openstack-infra13:12
*** yamamoto has quit IRC13:12
*** yamamoto has joined #openstack-infra13:13
*** goldyfruit has joined #openstack-infra13:14
*** sreejithp has joined #openstack-infra13:20
*** lseki has joined #openstack-infra13:26
*** smarcet has joined #openstack-infra13:27
*** trident has quit IRC13:29
*** mriedem has joined #openstack-infra13:29
cloudnullanyone around mind giving https://review.opendev.org/#/c/666655/  a nudge13:30
*** trident has joined #openstack-infra13:35
*** eharney has joined #openstack-infra13:37
*** yamamoto has quit IRC13:41
*** yamamoto has joined #openstack-infra13:42
*** yamamoto has quit IRC13:42
*** goldyfruit has quit IRC13:42
*** _erlon_ has quit IRC13:43
openstackgerritFabien Boucher proposed zuul/zuul master: URLTrigger driver time based  https://review.opendev.org/63556713:46
*** jaosorior_ has quit IRC13:48
*** hemna has joined #openstack-infra13:48
*** jaosorior has joined #openstack-infra13:49
*** jaosorior has quit IRC13:49
*** tdasilva has joined #openstack-infra13:49
*** yamamoto has joined #openstack-infra13:52
openstackgerritMerged openstack/project-config master: Set the tripleo-ansible project to use storyboard  https://review.opendev.org/66665513:55
*** udesale has joined #openstack-infra13:58
*** woojay has joined #openstack-infra13:59
*** smcginnis has joined #openstack-infra14:02
*** bhavikdbavishi has joined #openstack-infra14:03
*** bhavikdbavishi1 has joined #openstack-infra14:06
*** lpetrut has quit IRC14:06
*** dave-mccowan has joined #openstack-infra14:06
*** bhavikdbavishi has quit IRC14:07
*** bhavikdbavishi1 is now known as bhavikdbavishi14:07
*** goldyfruit has joined #openstack-infra14:09
*** dave-mccowan has quit IRC14:11
clarkbzbr|ruck: check the different branches of https://opendev.org/opendev/publications they arent templates but have a few categories (http/presentty/etc)14:17
*** ginopc has joined #openstack-infra14:17
zbr|ruckclarkb: thanks! who would have known to look on branches.14:19
corvusit's the first line of the readme14:19
openstackgerritJames E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job  https://review.opendev.org/66502314:22
*** ccamacho has quit IRC14:25
*** dpawlik has quit IRC14:25
*** jaosorior has joined #openstack-infra14:33
openstackgerritSean McGinnis proposed openstack/openstack-zuul-jobs master: Add openstack-tox-functional-py37 job  https://review.opendev.org/66711914:36
*** bogdando has joined #openstack-infra14:37
bogdandohi folks. What is supposed method of installing a package for zuul jobs preparations, after there is no more sudo available as of https://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/tox/run.yaml ?14:37
bogdandoAJaeger: hi, specifically I'm talking ^^ about https://review.opendev.org/#/c/664555/14:39
*** rajinir has joined #openstack-infra14:41
*** goldyfruit has quit IRC14:41
clarkbbogdando: The idea is to get that done prior to sudo removal. bindep14:41
bogdandoclarkb: I see, thanks. Not sure how to apply bindep.txt via zuul ansible?..14:42
clarkber bindep.txt and the tools/test_setup.sh (Or whatever that script is called) shoulf both run prior14:42
bogdandooh well, thanks... I have to look deeper into that14:42
clarkbbogdando: if the file is present in the root of the repo those packages are installed14:42
bogdandoclarkb: then that's perfect enough already! :)14:43
bogdandothanks14:43
*** ykarel is now known as ykarel|afk14:49
*** zhangfei has joined #openstack-infra14:52
hemnawhat is the kernel version in our CI jobs?14:53
*** bogdando has left #openstack-infra14:53
hemna(assuming it's all ubuntu)14:53
*** kopecmartin has quit IRC14:54
clarkbhemna: it varies per test node (we hvae ~3 ubuntus, ~2 fedoras, centos 7, and ~3 suses as well as ~2 debians?)14:54
clarkbhemna: the test should record that as part of its fact gathering though. Look in the zuul info dir in the logs14:54
hemnaclarkb: ok thanks14:56
hemnaok looks like 4.1514:57
clarkbalso we rebuild images daily14:57
hemnaI'm working on a ceph-iscsi job for a new cinder ceph-iscsi volume driver.14:57
clarkbso we'llget minor updates as they show up14:57
smcginnisclarkb: hemna is working on a driver for Ceph iSCSI. That functionality needs a specific kernel version as a minimum. If none of the distros we currently use (our "official" distros) then we wouldn't be able to officially support Ceph iSCSI, though we could set up an image to at least start testing it?14:58
hemnaand unforuntately, the ceph-iscsi tooling needs 4.16 >14:58
*** jamesmcarthur has joined #openstack-infra14:58
clarkbsmcginnis: hemna the tumbleweed image definitely has a 5.x kernel14:58
clarkbso some of our images do have newer kernels. I am guessing that the latest fedora does too14:58
smcginnishemna: I guess that should make your management happy if we can only say it's officially support on SUSE. ;)14:59
clarkbkeep in mind that end users don't typically run fedora or tumbleweed on production servers14:59
hemnalol14:59
hemnayah14:59
smcginnisFedora 30 is 5.1.12.14:59
hemnaubuntu is always behind it seems14:59
clarkbhemna: ubuntu is a balance between latest and greatest and stability15:00
clarkbthey do have a hardware eneablement kernel program on their stable releases but I don't think bionic has received one yet15:00
smcginnisI'm sure if we used a non-LTS release of Ubuntu we could get the version we need.15:00
hemnaguess it would be a risk to update out ubuntu images to use 4.16 or > then15:00
*** jlibosva has joined #openstack-infra15:00
clarkbsmcginnis: we don't do non lts ubuntu releases because they only have 9 months of support so aren't worth the investment15:00
clarkbhemna: you mean build your own kernel? you could do that but again remember that end users aren't really doing that so have to weigh that against the value of the testing15:01
hemnaubuntu has updated kernel packages15:01
clarkbhemna: for bionic? I checked recently and 4.15 seems to be it for now15:01
hemnaI'm running 4.18 in my bionic vm15:02
clarkbah so https://wiki.ubuntu.com/Kernel/Support?action=AttachFile&do=view&target=18.04.x+Ubuntu+Kernel+Support+Schedule.svg probably hasn't been updated yet?15:03
smcginnisDo we have CentOS 7 images?15:03
clarkbsmcginnis: yes15:03
smcginnisCool, looks like that may be an option.15:03
clarkbisn't centos 7 4.4 or something like that?15:04
hemna linux-image-4.18.0-20-generic - Signed kernel image generic15:04
hemnathat's the package I used to install it15:04
*** jaosorior has quit IRC15:05
clarkbhemna: you can update the kernel in the job and reboot15:06
clarkbI think you still have to restart the console log daemon on the test node, but otherwise that should just work as long as the new kernel functions15:06
hemnanot sure what the best approach is.  I have zero CI experience15:07
openstackgerritMatt Riedemann proposed opendev/elastic-recheck master: Add query for tempest race fail bug 1834046  https://review.opendev.org/66712315:07
openstackbug 1834046 in tempest "test_create_second_image_when_first_image_is_being_saved intermittently fails" [Undecided,Triaged] https://launchpad.net/bugs/183404615:07
clarkbthe easiest thing is to use an image with a kernel that is new neough. So if centos or fedora or tumbleweed offer that I would probably start there15:07
*** josephrsandoval has joined #openstack-infra15:09
clarkb(we default to whatever the platform default kernel is on ubuntu to help align with what people will actually use in the wild, but if using a different platform is too much work you can reboot into newer kernels)15:09
*** yamamoto has quit IRC15:09
*** panda has quit IRC15:09
*** yamamoto has joined #openstack-infra15:10
*** bhavikdbavishi has quit IRC15:10
*** factor has joined #openstack-infra15:11
*** panda has joined #openstack-infra15:11
*** jpich has quit IRC15:12
*** yamamoto has quit IRC15:12
*** jpich has joined #openstack-infra15:13
fungiclarkb: don't forget there's also the ubuntu ppa-ish thing ianw_pto set us up to use for the kafs testing, where ubuntu is providing mainline/rc builds even15:15
openstackgerritMatt Riedemann proposed opendev/elastic-recheck master: Add query for iptables_hybrid fw driver resize revert bug 1833902  https://review.opendev.org/66712815:16
openstackbug 1833902 in OpenStack Compute (nova) "Revert resize tests are failing in jobs with iptables_hybrid fw driver" [High,In progress] https://launchpad.net/bugs/1833902 - Assigned to Matt Riedemann (mriedem)15:16
fungialso he was working out a way to restore the console stream after rebooting onto a new kernel during a build15:16
*** yboaron_ has quit IRC15:17
*** priteau has quit IRC15:17
*** jaosorior has joined #openstack-infra15:18
fungii think he determined that a zuul ansible role for that wouldn't work for the system-config testing because it's our nested ansible doing the reboot, so instead he was going to try making a job role to install a systemd unit file which just reestablishes it at boot15:19
*** yamamoto has joined #openstack-infra15:20
*** tinwood_ is now known as tinwood15:20
openstackgerritTobias Henkel proposed zuul/zuul master: Split ansible output into its own sub logger  https://review.opendev.org/66713115:22
*** kopecmartin has joined #openstack-infra15:24
*** priteau has joined #openstack-infra15:25
*** yamamoto has quit IRC15:28
*** kopecmartin is now known as kopecmartin|off15:31
openstackgerritJames E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job  https://review.opendev.org/66502315:32
clarkbfungi: are we ready for https://review.opendev.org/#/c/656195/ and https://review.opendev.org/#/c/663151/15:33
*** tesseract has quit IRC15:33
mattmceuenHey, GM folks.  When someone gets a free moment, could I please get added as the initial team member to the airshipctl-core gerrit group?  I will pull in the appropriate seed crew.  Thanks!15:35
clarkbmattmceuen: done15:35
openstackgerritMerged opendev/elastic-recheck master: Add query for tempest race fail bug 1834046  https://review.opendev.org/66712315:36
openstackbug 1834046 in tempest "test_create_second_image_when_first_image_is_being_saved intermittently fails" [Undecided,Triaged] https://launchpad.net/bugs/183404615:36
*** jaosorior has quit IRC15:37
openstackgerritMerged opendev/elastic-recheck master: Add query for iptables_hybrid fw driver resize revert bug 1833902  https://review.opendev.org/66712815:38
openstackbug 1833902 in OpenStack Compute (nova) "Revert resize tests are failing in jobs with iptables_hybrid fw driver" [High,In progress] https://launchpad.net/bugs/1833902 - Assigned to Matt Riedemann (mriedem)15:38
*** zhangfei has quit IRC15:38
mattmceuenthanks clarkb!15:39
*** hrww is now known as hrw15:40
fungiclarkb: i was going to start in on those once my current meeting finishes, but yes15:43
*** yamamoto has joined #openstack-infra15:43
*** yamamoto has quit IRC15:43
*** yamamoto has joined #openstack-infra15:47
*** jpena is now known as jpena|off15:48
AJaegerfungi: want to merge https://review.opendev.org/663119 as well?15:50
fungiAJaeger: i'll take a look when my meeting's over15:50
*** yamamoto has quit IRC15:52
hrwcan I get second eye on https://review.opendev.org/#/c/666533/ patch so UCA/Train will get mirrored so we could move Kolla to use them?15:53
*** jamesmcarthur has quit IRC15:56
*** josephrsandoval has quit IRC15:56
*** ddurst- is now known as ddurst15:57
*** ginopc has quit IRC16:01
clarkbfungi: corvus ^ you think UCA/train is small enough I can jus tapprove that or should we grab the lock and publish it like some of our smallers volumes?16:02
*** hrww has joined #openstack-infra16:02
zbr|ruckfungi: any idea about when it will be safe to remove zuul-cloner? re your' https://github.com/ansible/molecule/pull/212416:04
zbr|ruckoops, wrong pase, that was correct: https://review.opendev.org/#/c/663151/16:04
*** Lucas_Gray has quit IRC16:05
openstackgerritJames E. Blair proposed zuul/zuul master: Allow config projects to override allowed-projects  https://review.opendev.org/66673316:05
clarkbzbr|ruck: the announced plan is to do it today and fungi said he would approve after morning meeting(s)16:05
*** priteau has quit IRC16:05
*** hrw has quit IRC16:06
*** lucasagomes has quit IRC16:06
*** hrww is now known as hrw16:07
fungiyeah, meeting just concluded so i'm catching up on the last hour's irc/ml backlog to make sure nothing new is on fire before i go ahead with it16:08
*** pgaxatte has quit IRC16:09
corvusclarkb: i've forgotten how large a release needs to be in order to require special treatment.  maybe <5g would be safe?16:09
clarkblooking at UCA I think it is quite small. The volume is ~6GB total based on vos example. And there are currently 8 releases in there16:09
clarkbabout 800MB per reelase?16:09
corvusi bet that can run in the normal timeout16:10
clarkbcorvus: cool I'll go ahead and approve it as is then16:10
clarkbs/example/examine/16:10
*** e0ne has quit IRC16:12
rpiosoIt appears the sushy repo doesn't offer permalinks. Are they available in that repo?16:13
corvusevrardjp, hogepodge: when https://review.opendev.org/666733 is merged, we can add loci post jobs back to other projects (we'll just have to do it in the project-config repo instead of the projects themselves)16:14
clarkbrpioso: if you open a file there is a permalink link at the top of the file16:14
clarkbnext to blame raw history16:15
rpiosoclarkb: I don't see that -- https://github.com/openstack/sushy/blob/master/sushy/main.py16:15
* rpioso confirms he has his eyeglasses on :-)16:16
hrwclarkb: thanks16:16
*** iurygregory has quit IRC16:17
clarkbrpioso: we don't manage github16:17
corvusrpioso: try this page on opendev: https://opendev.org/openstack/sushy/src/branch/master/sushy/main.py16:17
corvusthe permalink button there produces https://opendev.org/openstack/sushy/src/commit/640c567bae6c11fb4a8f92ec591066ad400e80cb/sushy/main.py16:17
rpiosoclarkb, corvus: Ooops and ugh! My browser took me to the wrong site. My apologies.16:18
*** icarusfactor has joined #openstack-infra16:23
*** factor has quit IRC16:23
*** jlibosva has quit IRC16:26
*** dpawlik has joined #openstack-infra16:27
*** jpich has quit IRC16:29
openstackgerritMerged ttygroup/gertty master: Add prev/next patchset keys to diff view  https://review.opendev.org/66604916:29
*** diablo_rojo has joined #openstack-infra16:30
*** trident has quit IRC16:30
fungislaweq: mriedem: dansmith: should we enqueue/promote 667035,4 to get it merged sooner?16:31
fungisounds like it's blocking neutron?16:31
*** mattw4 has joined #openstack-infra16:32
openstackgerritJames E. Blair proposed zuul/zuul master: Allow config projects to override allowed-projects  https://review.opendev.org/66673316:32
mriedemi didn't want to be the one to ask but probably16:32
*** trident has joined #openstack-infra16:32
fungijust catching up on developments on that thread, happy to shuffle it to the front16:32
mriedemi was just surprised to get a node in under 6 hours16:33
fungiit's at the top of the gate now, so should hopefully be in within a couple hours16:35
fungilogan-: not sure if you saw our connectivity issues with limestone over the weekend, but are we safe to readd it do you think?16:36
*** bhavikdbavishi has joined #openstack-infra16:38
openstackgerritJames E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job  https://review.opendev.org/66502316:41
*** igordc has joined #openstack-infra16:42
fungilogan-: looking at http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=64934&rra_id=all we're still losing connectivity to the mirror server there for long periods, so i'm guessing it's not safe to reenable yet16:43
logan-fungi: I did not, catching up on backscroll16:43
*** whoami-rajat has quit IRC16:44
logan-yeah that doesn't look great :/16:44
openstackgerritJames E. Blair proposed zuul/nodepool master: WIP: remove devstack plugin functional test jobs  https://review.opendev.org/66715616:45
fungilogan-: no worries, mostly just wanted to be sure you're aware strange things are afoot16:47
*** dklyle has joined #openstack-infra16:49
*** mattw4 has quit IRC16:50
logan-fungi: thank you. I will let you know once I track down the cause16:52
fungithanks for taking a look!16:52
*** mattw4 has joined #openstack-infra16:52
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add install-devstack role  https://review.opendev.org/66715716:53
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add install-devstack role  https://review.opendev.org/66715716:55
openstackgerritJames E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job  https://review.opendev.org/66502316:56
openstackgerritJames E. Blair proposed zuul/nodepool master: WIP: remove devstack plugin functional test jobs  https://review.opendev.org/66715616:56
*** Emine has quit IRC16:57
openstackgerritMerged opendev/system-config master: UCA: mirror Train packages  https://review.opendev.org/66653316:57
fungiokay, i'm caught up on discussions, ml moderation requests, wiki patrol... approving 663151 and 656195 nowish16:58
*** smarcet has quit IRC16:58
fungipreparing to send e-mail to the announcement thread about that on openstack-discuss once they merge16:59
*** icarusfactor has quit IRC17:00
*** whoami-rajat has joined #openstack-infra17:02
*** Goneri has quit IRC17:02
*** weifan has joined #openstack-infra17:06
*** e0ne has joined #openstack-infra17:07
*** udesale has quit IRC17:07
*** e0ne has quit IRC17:08
*** josephrsandoval has joined #openstack-infra17:09
openstackgerritMerged opendev/base-jobs master: Remove fetch-zuul-cloner  https://review.opendev.org/66315117:10
fungithere's one17:10
openstackgerritMerged zuul/zuul-jobs master: Use password lookup for run-buildset-registry role  https://review.opendev.org/66311917:11
clarkbThe first should take effect immediately but the second requires a puppet run iirc17:11
fungiahh, that file gets installed on the executors?17:12
fungi(zuul/site-variables.yaml)17:12
clarkbya the site vars are installed I think17:12
clarkband ansible reads them when it runs17:12
clarkbso don't have to restart anything17:13
fungiwell, i'll still send the e-mail as soon as it merges, and we'll see it take effect within 30-60 minutes i guess17:13
clarkbsounds good17:13
*** gfidente has quit IRC17:14
openstackgerritMerged openstack/project-config master: Make bindep fallback only happen in openstack  https://review.opendev.org/65619517:15
fungiand there it is17:16
fungiclarkb: mordred: i am somewhat confused by the commit message on that one17:17
fungiwhat does "only happen in openstack" mean in this context?17:17
*** psachin has quit IRC17:17
clarkbfungi: the depends on change is the explanation17:17
fungiaha, thanks17:17
clarkbfungi: basiclaly before we set that var globally but now we'll only set it in the openstack legacy bsae jobs17:18
*** michael-beaver has joined #openstack-infra17:18
fungii see, so it really should have been "don't use bindep fallback unconditionally"17:18
*** panda has quit IRC17:18
fungiwe're not even doing it for all of openstack once that takes effect17:18
clarkbcorrect17:19
clarkband if jobs need the fallback they can either explicitly set the var themselves or add a bindep.txt17:19
fungiright17:19
fungiokay, so just a misleading commit message. still have the effect i remembered17:19
fungiokay, followup sent mentioning these are now merged17:20
*** panda has joined #openstack-infra17:20
*** ociuhandu_ has joined #openstack-infra17:21
*** ralonsoh has quit IRC17:23
openstackgerritMerged openstack/pbr master: option to print only the version of a package  https://review.opendev.org/57761417:24
*** ociuhandu has quit IRC17:25
*** smarcet has joined #openstack-infra17:25
*** ociuhandu_ has quit IRC17:27
*** e0ne has joined #openstack-infra17:27
fungigoing ahead and abandoning my test changes now17:29
*** goldyfruit has joined #openstack-infra17:30
clarkbskimming the zuul status I am not seeing anything terribly broken yet17:30
clarkbtox jobs are running and not doing the z-c shim17:30
fungiodds are we won't know until folks start complaining ;)17:31
clarkbgranted they weren't running z-c17:31
clarkbthe z-c thing has two pieces, there is the installation and the usage of the script17:32
clarkbI think only the usage of the script is where we'll see problems17:32
clarkbpuppet should run on ze* shortly.17:33
clarkb~5 minutes17:33
fungiyeah, and there's stuff like https://opendev.org/openstack/neutron-dynamic-routing/src/branch/master/playbooks/neutron-dynamic-routing-dsvm-functional/run.yaml#L2017:35
fungiis that only run in the context of jobs inheriting from a legacy abstract definition? who knows?17:35
fungiit's using devstack-gate at least, so... probably?17:35
fungiand similar "legacy" copy pasta17:36
clarkbya I think d-g is likely to only work with legacy jobs17:36
fungiwhere it gets fuzzier is places like https://opendev.org/openstack/puppet-openstack-integration/src/branch/master/functions#L4417:37
fungior https://opendev.org/openstack/tripleo-ci/src/branch/master/playbooks/tripleo-buildimage/run.yaml#L2917:38
clarkbpuppet is running on ze01 now17:40
clarkbJun 24 17:40:00 ze01 puppet-user[5723]: (/Stage[main]/Zuul/File[/etc/zuul/site-variables.yaml]/content) content changed '{md5}568e923c479a41bc31971fe9e3a5233f' to '{md5}283d968eab36e6dbdd95f1dded92c955'17:40
clarkbwe should start seeing jobs run without that var set now17:40
fungiexcellent17:41
fungido we still use this script? https://opendev.org/opendev/system-config/src/branch/master/tools/install_modules_acceptance.sh#L7217:42
fungiahh, nevermind, it fails safe on line 89 if there's no zuul-cloner present17:43
clarkbfungi: http://logs.openstack.org/69/665269/6/check/infra-puppet-apply-4-ubuntu-xenial/f27435b/job-output.txt.gz#_2019-06-14_03_14_15_351993 I think we do17:44
clarkbthat job doesn't parent to legacy base17:44
clarkbI think our cross project testing for puppet modules will be broken but as you say the tests will fail to testing upstream17:45
clarkbwe can reparent to legacy-base if we want that behavior back17:45
clarkbor add the z-c shim install specifically17:45
fungihere's another one which may not inherit from a legacy abstract, and instead manually injects zuul_legacy_vars into the task environment: https://opendev.org/openstack/magnum/src/branch/master/playbooks/magnum-buildimages-base.yaml#L1717:45
*** yamamoto has joined #openstack-infra17:45
clarkbI see the bindep fallback lookup report skipping as condition was not met now17:47
clarkbso that change is working at least in doing what we expected (jobs may not be happy with it017:47
clarkbfungi: I rechecked https://review.opendev.org/#/c/665269/6 to sanity check system-config and z-c shim removal17:49
*** eharney_ has joined #openstack-infra17:51
*** josephrsandoval has quit IRC17:51
*** yamamoto has quit IRC17:51
*** eharney has quit IRC17:53
*** goldyfruit has quit IRC17:54
*** josephrsandoval has joined #openstack-infra17:59
clarkbironic runs bandit non voting (and it fails with legit errors)18:00
clarkbhaven't seen anything break due to our changes yet18:01
*** e0ne has quit IRC18:02
fungiif ironic wants to run bandit, they can always start out with it only failing on critical risks via the -lll option18:03
fungiand then ratchet it down over time18:04
clarkbhttp://logs.openstack.org/71/665171/2/check/openstack-tox-pep8/db2ab5b/job-output.txt.gz#_2019-06-24_18_01_56_259504 I wonder if that is a z-c shim break18:05
*** e0ne has joined #openstack-infra18:06
corvusclarkb: ps1 failed with the same error18:07
clarkbI don't see deckhand in previous runs of that job against that project18:07
corvus(and that was 1 week ago)18:07
corvushttp://logs.openstack.org/71/665171/1/check/openstack-tox-pep8/2570506/job-output.txt.gz#_2019-06-17_15_55_18_06396318:07
corvusso i think that's clear18:07
clarkbya I don't think it was our change18:07
clarkbseems that job updates requirements.txt to point at a new version of deckhand and that is failing18:08
clarkbI've rechecked https://review.opendev.org/#/c/486388/5 to sanity check d-g works with the z-c shim still (as it should run on legacy base)18:10
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add install-devstack role  https://review.opendev.org/66715718:12
clarkbfungi: http://logs.openstack.org/69/665269/6/check/infra-puppet-apply-4-ubuntu-xenial/8f15865/job-output.txt.gz#_2019-06-24_18_10_30_055999 seems the guard wasn't enough?18:13
clarkbI'll work on a legacy-base reparenting change18:13
*** weifan has quit IRC18:14
openstackgerritClark Boylan proposed openstack/openstack-zuul-jobs master: Switch infra puppet apply jobs to legacy base  https://review.opendev.org/66717318:16
clarkbthat should do it18:16
*** hrw has quit IRC18:20
fungithanks!18:26
clarkbcorvus: ^ have a second to be second review on that one?18:27
clarkbLooks like overall things are not on fire18:27
clarkbI'm going to get a bike ride in and find some lunch18:27
*** dciabrin_ has joined #openstack-infra18:29
*** ykarel|afk has quit IRC18:30
*** dciabrin has quit IRC18:31
*** josephrsandoval has quit IRC18:31
*** dpawlik has quit IRC18:39
*** dpawlik has joined #openstack-infra18:40
*** tosky has quit IRC18:41
fungianybody know if we might have changed something recently about how the legacy git domain redirects were working on files.o.o?18:46
fungiahh, i think it's fine, looks like the error was courtesy of a typo18:46
*** mattmceuen has left #openstack-infra18:47
*** dklyle has quit IRC18:49
*** david-lyle has joined #openstack-infra18:49
corvusclarkb: +318:53
*** eharney_ is now known as eharney18:56
*** e0ne has quit IRC19:00
*** e0ne has joined #openstack-infra19:03
openstackgerritMerged openstack/openstack-zuul-jobs master: Switch infra puppet apply jobs to legacy base  https://review.opendev.org/66717319:04
*** dciabrin_ has quit IRC19:06
*** dciabrin_ has joined #openstack-infra19:07
*** tdasilva_ has joined #openstack-infra19:08
openstackgerritMatt Riedemann proposed opendev/elastic-recheck master: Add query for neutron agent tempest-slow* bug 1834079  https://review.opendev.org/66719019:11
openstackbug 1834079 in neutron ""Bridge br-ex for physical network public does not exist. Agent terminated!" in tempest-slow job(s) on stable branches" [Undecided,New] https://launchpad.net/bugs/183407919:11
*** tdasilva has quit IRC19:11
*** diablo_rojo has quit IRC19:12
*** diablo_rojo has joined #openstack-infra19:13
*** e0ne has quit IRC19:16
*** bhavikdbavishi has quit IRC19:18
*** e0ne has joined #openstack-infra19:19
*** jtomasek has quit IRC19:22
*** xek has quit IRC19:23
*** whoami-rajat has quit IRC19:34
*** e0ne has quit IRC19:36
*** jcoufal has quit IRC19:37
*** sreejithp has quit IRC19:38
openstackgerritMerged opendev/elastic-recheck master: Add query for neutron agent tempest-slow* bug 1834079  https://review.opendev.org/66719019:41
openstackbug 1834079 in neutron ""Bridge br-ex for physical network public does not exist. Agent terminated!" in tempest-slow job(s) on stable branches" [Undecided,New] https://launchpad.net/bugs/183407919:41
*** markmcclain has quit IRC19:49
*** e0ne has joined #openstack-infra19:49
*** armstrong has joined #openstack-infra19:56
*** bnemec has quit IRC20:00
*** smarcet has quit IRC20:06
*** david-lyle is now known as dklyle20:06
*** smarcet has joined #openstack-infra20:06
*** bnemec has joined #openstack-infra20:08
openstackgerritJames E. Blair proposed opendev/glean master: Add .zuul.yaml  https://review.opendev.org/66721120:09
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Move Zuul config in-repo  https://review.opendev.org/66721220:10
openstackgerritJames E. Blair proposed openstack/project-config master: Remove nodepool functional job attachments  https://review.opendev.org/66721320:10
*** _erlon_ has joined #openstack-infra20:12
openstackgerritJames E. Blair proposed zuul/nodepool master: WIP: remove devstack plugin functional test jobs  https://review.opendev.org/66715620:26
openstackgerritJames E. Blair proposed zuul/nodepool master: Add env var customization to openstack func job  https://review.opendev.org/66721720:26
*** smarcet has quit IRC20:29
*** smarcet has joined #openstack-infra20:33
*** e0ne has quit IRC20:34
*** ociuhandu has joined #openstack-infra20:35
*** smarcet has quit IRC20:40
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Move existing Zuul project config to projects.yaml  https://review.opendev.org/66722020:41
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Replace nodepool func jobs  https://review.opendev.org/66722120:41
*** pcaruana has quit IRC20:44
openstackgerritJames E. Blair proposed opendev/glean master: Replace nodepool func jobs  https://review.opendev.org/66722520:45
openstackgerritJames E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs  https://review.opendev.org/66715620:46
*** e0ne has joined #openstack-infra20:46
*** e0ne has quit IRC20:47
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Move Zuul config in-repo  https://review.opendev.org/66721220:48
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Move existing Zuul project config to projects.yaml  https://review.opendev.org/66722020:48
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Replace nodepool func jobs  https://review.opendev.org/66722120:48
openstackgerritJames E. Blair proposed openstack/diskimage-builder master: Replace nodepool func jobs  https://review.opendev.org/66722120:49
openstackgerritJames E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs  https://review.opendev.org/66715620:51
corvusokay, that's quite the dependency graph, but i think it's complete... we'll see what zuul thinks :)20:51
corvussee topic:nodepool-func for the whole thing20:51
*** rfolco has quit IRC20:57
openstackgerritJames E. Blair proposed openstack/project-config master: Add zuul-operator project  https://review.opendev.org/66722820:59
openstackgerritJames E. Blair proposed zuul/nodepool master: Add env var customization to openstack func job  https://review.opendev.org/66721721:05
openstackgerritJames E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs  https://review.opendev.org/66715621:05
clarkbsystem-config canary change passes testing now after the z-c shim fix21:08
*** gouthamr_ is now known as gouthamr21:16
*** markmcclain has joined #openstack-infra21:17
*** ianychoi_ has joined #openstack-infra21:19
*** eharney has quit IRC21:20
*** ianychoi has quit IRC21:22
*** michael-beaver has quit IRC21:25
*** ianychoi_ has quit IRC21:30
*** ianychoi_ has joined #openstack-infra21:31
cloudnullhey all, I'm trying to add things to storyboard for tripleo-ansible and I keep getting errors like so21:31
cloudnull> 500: POST /api/v1/tasks: This session is in 'inactive' state, due to the SQL transaction being rolled back; no further SQL can be emitted within this transaction.21:31
cloudnullhowever, the story was added just the tasks seem messed up21:32
cloudnullhttps://storyboard.openstack.org/#!/project/openstack/tripleo-ansible21:32
cloudnullis there something I need to be doing differently ?21:32
*** Emine has joined #openstack-infra21:34
cloudnullanother issue I've been running into is when I try and create a workboard21:34
cloudnull> 500: POST /api/v1/worklists: 'NoneType' object has no attribute 'name'21:34
cloudnull> 500: POST /api/v1/worklists:21:34
cloudnull> 500: POST /api/v1/worklists: 'NoneType' object has no attribute 'name'21:34
*** kjackal_v2 has quit IRC21:35
clarkbfungi: diablo_rojo ^ ?21:35
cloudnullhttps://i.snag.gy/m3rzhc.jpg ^ screenshot21:36
*** factor has joined #openstack-infra21:37
*** rajinir has quit IRC21:38
cloudnullhttps://i.snag.gy/xkSv1R.jpg <- here's a screen grab of when I try and add a story21:38
*** dpawlik has quit IRC21:44
clarkbthe d-g canary is happy post z-c shim changes21:46
fungicloudnull: is it only when you try to add multiple tasks at the creation of the story? or does it happen even with just one initial task being added?21:54
*** mriedem has quit IRC21:55
*** panda has quit IRC21:58
cloudnullfungi let me give just making a story a go, then adding tasks after22:00
cloudnullthe workboard issue happens no matter what I try22:01
*** icarusfactor has joined #openstack-infra22:01
*** factor has quit IRC22:02
cloudnullfungi if i add a story with just the one task it works22:02
cloudnullI can then add additional tasks from within the story22:02
fungibut if you create with two or more it fails? consistently?22:02
*** panda has joined #openstack-infra22:03
cloudnullyes22:03
fungiwe merged a recent fix for task creation sql lock contention so not sure if this is another case of the same or if we just changed the behavior22:03
* cloudnull needs to go eat, but will return 22:04
fungiyeah, https://review.opendev.org/650614 in theory solved something along those lines22:04
*** tobiash has quit IRC22:10
fungicloudnull: for the worklist problem are you trying to add those incomplete tasks maybe?22:13
fungii'll try digging in the api server's logs here shortly22:14
*** ociuhandu has quit IRC22:30
*** ociuhandu_ has joined #openstack-infra22:30
openstackgerritJames E. Blair proposed zuul/zuul master: Strengthen the caution about allowed-projects  https://review.opendev.org/66724122:31
*** ociuhandu_ has quit IRC22:31
corvusfungi, clarkb: has glean bitrotted?  http://logs.openstack.org/11/667211/1/check/tox-py27/75bcd3b/job-output.txt.gz#_2019-06-24_20_18_17_71841322:33
clarkbcorvus: we have to cap sphinx to fix that iirc22:34
* clarkb finds release notes22:34
clarkb<2.022:35
corvusis that recent?22:37
clarkbpypi says march 28, 201922:37
corvushuh... there are pending changes to glean which ran jobs on may 722:38
clarkbhrm maybe they didn't fully remove python2 support with 2.0?22:38
corvusi rechecked https://review.opendev.org/652238 to see if it shows up there now22:39
fungiwhat version of sphinx did they use? openstack upper-constraints.txt may have not updated and if we're using that...22:39
fungithen we wouldn't have seen it break until they updated the constraints22:40
corvusopenstack-python-jobs-no-constraints22:40
corvusis a template that glean uses, so i'm *assuming* based on those words that constraints isn't involved22:40
clarkbreading sphinx git history 2.0 seems to have the setup.py check and it released march 2822:41
clarkbmaybe that job ran under python3 previously then that changed?22:42
corvushard to say.  it's the tox-py27 job, but logs aged out, so i can't say what really happened before22:42
*** Emine has quit IRC22:43
clarkbhttps://opendev.org/openstack/requirements/src/branch/master/global-requirements.txt#L459-L460 is how opentack addressed this22:45
corvusokay... i think i'll wait for the rechecks to both return red, and if so, i'll shrug my shoulders and do that.  if one of them is green, then something weirder is going on and warrants more investigation22:46
clarkb++22:46
fungiyeah, that is all very strange and seemingly inconsistent22:47
*** rcernin has joined #openstack-infra22:51
fungicloudnull: the traceback i found in the api server log definitely passes straight through the function 650614 decorated, so it's likely that change was ultimately ineffective at what it was trying to fix22:53
clarkbTheJulia: did you want to ack ]https://review.opendev.org/#/c/665321/4/gerrit/projects.yaml ?22:53
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Expand documentation of test-setup role  https://review.opendev.org/66724422:54
openstackgerritJames E. Blair proposed zuul/nodepool master: Switch functional testing to a devstack consumer job  https://review.opendev.org/66502322:55
openstackgerritJames E. Blair proposed zuul/nodepool master: Add env var customization to openstack func job  https://review.opendev.org/66721722:55
openstackgerritJames E. Blair proposed zuul/nodepool master: Remove devstack plugin functional test jobs  https://review.opendev.org/66715622:55
*** tkajinam has joined #openstack-infra22:55
openstackgerritMerged openstack/project-config master: Add ``octavia-diskimage-retrofit`` charm  https://review.opendev.org/66656823:02
*** ianw_pto is now known as ianw23:08
*** yamamoto has joined #openstack-infra23:10
*** aaronsheffield has quit IRC23:14
ianwfungi: hi, just checking back on that rax ipv6 ticket, did you confirm the migration of the iad mirror host?23:17
clarkbianw: your kafs stack (starting with ntpd fix for bionic) is bubbling up my todo list. I +2'd it late last week but didn't approve as I wasn't around. Are you around again and wanting to approve those?23:19
*** sgw has joined #openstack-infra23:20
openstackgerritMerged openstack/project-config master: Retire openstack-ansible unused roles  https://review.opendev.org/65042223:20
openstackgerritMerged openstack/project-config master: Retire independent ansible role  https://review.opendev.org/66523723:20
openstackgerritJeremy Stanley proposed opendev/storyboard master: Guard for nonexistent team in worklist creation  https://review.opendev.org/66724823:21
clarkbhttp://mirror.iad.rax.opendev.org/ubuntu-cloud-archive/dists/bionic-updates/train/ exists now23:21
ianwclarkb: hey, yep i am around.  i'm hoping the stack doesn't actually do anything at this point :)  it should all just be framework ... should ...23:21
fungicloudnull: i hope 667248 there is the solution to your worklist problem. i think it's a very recent regression with the team ownership and permissions features which just landed23:22
*** dchen has joined #openstack-infra23:22
*** _erlon_ has quit IRC23:22
fungiianw: yeah, it's been slipping from my to do list to update the ticket. migration went fine, did not fix v6 connectivity. symptoms are basically identical to the earlier ticket we have open for... like... a year23:22
clarkbianw: https://review.opendev.org/#/c/665269/ is the one I'm thinking of (it is parent to the kafs changes) and it will make prod changes23:23
ianwclarkb: yep ... although i think all the bionic hosts are actually have systemd-timesyncd anyway23:23
*** slaweq has quit IRC23:24
fungiianw: if you run tcpdump you'll see that ipv6 packets leave the mirror server and arrive at their destination just fine, and then the responses to those are never received by the mirror server23:24
clarkbianw: oh huh23:24
clarkbfungi: ianw that sounds like a router/firewall issue then?23:24
clarkbianw: I've approved it, which should drag a handful of the others in too (but ya I think they are noops)23:25
ianwfungi: ahh, yeah, so like that exact same issue we saw before23:25
fungiianw: the extra weirdness there is that if you sniff for all ipv6 traffic destined to the mirror server, you'll see what looks like the inbound half of some constant https connection to akamai we aren't initiating... i think there may be an address collision23:25
cloudnullthanks fungi23:25
fungicloudnull: i've reported the tracebacks from both problems in #storyboard if you're interested (channel logs are on eavesdrop as you would expect)23:26
fungithe task deadlock db transaction issue is going to take some extra noodling23:26
ianwfungi: oh, you mean we're getting traffic that should be for someone else??  that sounds less like a annoying connectivity problem and more like a pretty serious leak23:26
* cloudnull joining the storyboard channel 23:26
fungiianw: i concur23:26
cloudnullas always, much appreciated23:26
*** yamamoto has quit IRC23:28
fungicloudnull: your feedback is extremely valuable--thanks for bringing these issues to our attention!23:30
*** lseki has quit IRC23:40
*** rlandy has quit IRC23:41
openstackgerritJeremy Stanley proposed opendev/storyboard master: Correct team iterator lists in worklist creation  https://review.opendev.org/66724823:44
*** smarcet has joined #openstack-infra23:47
logan-fungi: neutron had issues on the network nodes. i restarted everything there and ran a bunch of internal jobs through that cloud today, things seem to work normally again. cacti looks stable now too.23:48
fungilogan-: i noticed it had smoothed out a while ago. thanks for looking into it!!!23:52
clarkbfungi: do we need to get a change in to use that cloud again?23:52
openstackgerritJeremy Stanley proposed openstack/project-config master: Revert "Revert "Revert "Disable provider limestone"""  https://review.opendev.org/66725023:53
funginot any more ;)23:54
clarkb+2'd :)23:54
fungiwell, we still need to get it in, but we have a change at least!23:54
sgwmordred: Hi there, question for you about PBR and if it can populate the RPM version variable somehow?  I know that the PBR_VERSION can be set to the RPM %{version}, I want to know if it can go the other way?23:56
clarkbsgw: I don't believe so23:56
*** sthussey has quit IRC23:57
clarkbit can print an rpm compatible version but there isn't a way to have that be the version iirc23:57
clarkbso you can do PBR_VERSION=$(python setup.py rpm_version) python setup.py sdist23:58
clarkbsomething like that23:58
logan-fungi: no problem.. wish i had time to dig into the cause. maybe next time if it happens again :P23:58
*** rcernin has quit IRC23:59
*** rcernin has joined #openstack-infra23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!