Monday, 2018-10-08

*** jamesmcarthur has quit IRC00:05
*** jamesmcarthur has joined #openstack-infra00:14
*** jamesmcarthur has quit IRC00:23
*** longkb has joined #openstack-infra00:38
*** rlandy has quit IRC00:58
*** jamesmcarthur has joined #openstack-infra00:58
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: WIP: Test using install-kubernetes role  https://review.openstack.org/60580301:00
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: WIP: Test using install-kubernetes role  https://review.openstack.org/60580301:08
*** ansmith has quit IRC01:14
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: WIP: Test using install-kubernetes role  https://review.openstack.org/60580301:14
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: WIP: Test using install-kubernetes role  https://review.openstack.org/60580301:27
*** yamamoto has quit IRC01:29
*** yamamoto has joined #openstack-infra01:29
*** jiapei has joined #openstack-infra01:39
prometheanfireianw: mind looking at the gentoo caching review?01:44
prometheanfiresince you seem to be around :P01:44
*** hongbin has joined #openstack-infra01:48
prometheanfireI should be able to spin up an infra image on my home cluster to do some debugging right?01:59
*** yamamoto has quit IRC02:01
*** yamamoto has joined #openstack-infra02:02
clarkbyes they use glean to set up ssh keys fir root user02:02
prometheanfirecool02:04
mnaserclarkb: correct re: neutron ports starting their life as DOWN02:04
mnaserthere's no way of knowing if they are about to get bound until nova-compute does it02:04
mnasermy suggestion is if port is DOWN and created_at > 30 minutes it's probably bad02:05
*** yamamoto has quit IRC02:06
ianwok02:07
ianwpromethanfire: the devuser element is also useful, see tools/build_image.sh02:07
prometheanfireianw: yep, but I know glean works since that's what I use now02:08
prometheanfireI've used devuser as well, but would rather not have to build the image, takes a while02:08
ianwok, yeah the other thing for testing is you can cut down the repo cloning, again the yaml file and var is described in that script02:11
*** dave-mccowan has quit IRC02:30
*** Shrews has quit IRC02:32
*** Shrews has joined #openstack-infra02:32
*** yamamoto has joined #openstack-infra02:43
*** yamamoto has quit IRC02:57
*** yamamoto has joined #openstack-infra02:58
*** yamamoto has quit IRC03:02
*** bobh has joined #openstack-infra03:35
*** bobh has quit IRC03:40
*** yamamoto has joined #openstack-infra03:46
*** kaisers has quit IRC03:56
*** kaisers has joined #openstack-infra04:05
*** kaisers has quit IRC04:12
*** kaisers has joined #openstack-infra04:19
*** coolsvap has joined #openstack-infra04:25
*** annp has joined #openstack-infra04:25
AJaegerianw: welcome back - is https://review.openstack.org/608069 what you had in mind regarding translation jobs? Or anything elese missing?04:26
*** yamamoto has quit IRC04:29
ianwAJaeger: thanks!  i'm slowly working down my review list :)04:31
*** hongbin has quit IRC04:32
openstackgerritMerged openstack-infra/project-config master: Use prepare-zanata-client role  https://review.openstack.org/60806904:37
*** dayou has quit IRC04:39
*** ramishra has joined #openstack-infra04:39
*** dayou has joined #openstack-infra04:39
ianwcmurphy: so has https://review.openstack.org/#/c/606021/ turned out to be harder than it initially seemed?04:49
*** janki has joined #openstack-infra04:54
AJaegergreat, ianw04:58
*** yamamoto has joined #openstack-infra04:58
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Allow debootstrap to cleanup without a kernel  https://review.openstack.org/60469205:02
openstackgerritMerged openstack-infra/system-config master: Add note about mounting one AFS volume in another  https://review.openstack.org/60827205:04
*** apetrich has joined #openstack-infra05:06
*** jtomasek has joined #openstack-infra05:17
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds  https://review.openstack.org/60426805:41
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: simplify python3.6 selection on gentoo  https://review.openstack.org/60857605:43
*** yamamoto has quit IRC05:45
openstackgerritMerged openstack/diskimage-builder master: fix a typo  https://review.openstack.org/60819105:46
prometheanfireianw: better?05:46
*** quiquell|off is now known as quiquell05:46
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: simplify overlay logic for Gentoo  https://review.openstack.org/60857705:46
quiquellfungi: Hello what happend ?05:49
quiquellfungi: Did our patch break everything or something ?05:49
*** felipemonteiro has quit IRC05:49
*** snapiri has quit IRC06:12
*** e0ne has joined #openstack-infra06:21
*** mrsoul has joined #openstack-infra06:31
*** quiquell is now known as quiquell|brb06:37
*** aojea has joined #openstack-infra06:39
*** aojeagarcia has joined #openstack-infra06:43
*** yamamoto has joined #openstack-infra06:44
*** aojea has quit IRC06:47
*** florianf has joined #openstack-infra06:50
*** kopecmartin|off is now known as kopecmartin|ruck07:02
*** ginopc has joined #openstack-infra07:02
*** quiquell|brb is now known as quiquell07:09
ianwprometheanfire: well the -1 was mostly because it seemed to be failing in the gate07:10
prometheanfireianw: got a +1?07:10
ianwyeah but the -extras tests are non-voting07:12
*** slaweq_ has joined #openstack-infra07:12
*** tosky has joined #openstack-infra07:13
*** jtomasek has quit IRC07:16
*** jtomasek has joined #openstack-infra07:17
*** electrofelix has joined #openstack-infra07:20
*** kjackal has joined #openstack-infra07:21
*** rcernin has quit IRC07:21
prometheanfireah, ya07:24
*** chandankumar has joined #openstack-infra07:40
*** roman_g has joined #openstack-infra07:42
cmurphyianw: yes but not because of the change, the puppet-pip tests are completely broken on centos :(07:43
cmurphyhaving a ton of trouble tracking it down because i can't reproduce it07:43
*** jpena|off is now known as jpena07:44
*** xinliang has joined #openstack-infra07:52
xinliangJeffrey4l: So you suggest arm vm use more vcpus as x86 ubuntu?07:53
*** yamamoto has quit IRC07:55
Jeffrey4lxinliang, yep07:55
xinliangJeffrey4l: thanks, will try see if it can07:56
Jeffrey4lnp :D07:56
*** owalsh_ is now known as owalsh08:03
*** amoralej|off is now known as amoralej08:04
*** olivierb has joined #openstack-infra08:06
*** jpich has joined #openstack-infra08:07
*** finucannot is now known as stephenfin08:15
*** snapiri has joined #openstack-infra08:19
*** shardy has joined #openstack-infra08:39
ianwcmurphy: ok, i'll try to have a look my time tomorrow, see if anything stands out08:42
*** aidin has joined #openstack-infra08:42
cmurphyianw: that would be awesome, thanks :D08:42
xinliangianw: One question,  I see that arm node use flavor 'os.large' https://github.com/openstack-infra/project-config/blob/master/nodepool/nl03.openstack.org.yaml#L38108:43
xinliangbut actually, it "use os.large.infra"08:43
ianwxinliang: hrm ... i feel like we changed that at some point?  i wonder if nodepool has something cached08:44
xinliangianw:  We want to use flavor with 8 vcpus, os.large does has 8 vcpus, but os.large.infra only has 408:45
xinliangUsing 8 vcpus to avoid job timeout.08:46
xinliangianw: Could you help to change to flavor with 8 vcpus?08:46
ianwis it possible that it changed in the cloud but kept the same id?  clutching at straws ...08:47
xinliangianw: You mean changing os.large.infro flavor  make it has 8 vcpus?08:47
ianwi can't quite remember why we had os.large.infra ...08:48
xinliangianw:  will try to change it in the cloud. thanks08:49
ianwis this london?08:49
xinliangyes08:50
xinlianglondon cloud08:50
ianwyeah, due to issues there i can't even list the flavors08:50
ianwExpecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400) (Request-ID: req-45c14132-ebef-4e9e-86fc-4c194272fe43)08:50
ianwthat's a new one08:50
*** chandankumar has quit IRC08:51
*** aidin has quit IRC08:52
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add install and deploy openshift roles.  https://review.openstack.org/60861008:52
cmurphythat's a keystone v3 error, need to set OS_PROJECT_DOMAIN_NAME08:53
*** yamamoto has joined #openstack-infra08:53
ianwcmurphy: hrm, the config file has  "project_domain_name"08:54
ianwhrm, though maybe it doesn't have the identity_api_version ... but why wouldn't it pickup v3 automatically?08:55
cmurphyianw: i think os-client-config defaults to v208:56
ianwhrm, yeah with v3 i can see them :)08:56
*** jrist has joined #openstack-infra08:57
xinliangianw: I've changed flovor os.large.infra. Make it has 8 vcpus.08:57
ianwxinliang: ok, yeah the listing i'm seeing has no os.large.infa flavor08:58
xinliangIt is not public, only accessed by os-jobs and os-control project08:59
ianwxinliang: yeah, ok i see it, sorry wrong project08:59
ianwxinliang: you know what, i bet i know what it is ... it's the prefix matching "os.large.infra" and "os.large" ... i bet shade is partial matching and finds the .infra one first09:00
ianwwe could either specify it by ID, or just remove the os.large.infra flavor09:00
*** jrist has quit IRC09:02
xinliangianw: but maybe there is a bug, specifying os.large actually use os.large.infra09:03
ianwxinliang: have to check with mordred, but pretty sure it's a feature :)  additionally it matches the right amount of ram09:08
xinliangianw: got it09:08
*** jrist has joined #openstack-infra09:13
openstackgerritIan Wienand proposed openstack-infra/system-config master: Specify identity version for Linaro London  https://review.openstack.org/60861509:16
*** sshnaidm is now known as sshnaidm|lnch09:40
*** bhavikdbavishi has joined #openstack-infra09:47
*** dtantsur|afk is now known as dtantsur09:49
*** yamamoto has quit IRC09:49
*** bhavikdbavishi has quit IRC09:52
*** xinliang has quit IRC09:52
*** sshnaidm|lnch is now known as sshnaidm09:59
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066710:06
*** jpena is now known as jpena|off10:07
*** longkb has quit IRC10:08
*** jtomasek has quit IRC10:09
*** janki has quit IRC10:12
*** janki has joined #openstack-infra10:13
*** quiquell is now known as quiquell|brb10:20
*** quiquell|brb is now known as quiquell10:21
*** ssbarnea_ has quit IRC10:24
*** jaosorior has joined #openstack-infra10:26
*** jtomasek has joined #openstack-infra10:28
*** gema has joined #openstack-infra10:33
*** rossella_s has joined #openstack-infra10:33
*** rcernin has joined #openstack-infra10:35
*** dpawlik has joined #openstack-infra10:37
*** colonwq has joined #openstack-infra10:44
*** slaweq_ is now known as slaweq10:44
*** slaweq is now known as dpawlik_10:45
*** dpawlik_ is now known as slaweq10:45
*** dave-mccowan has joined #openstack-infra10:47
*** yamamoto has joined #openstack-infra10:48
*** aidin has joined #openstack-infra10:48
*** quiquell is now known as quiquell|brb10:53
*** rossella_s has quit IRC10:55
*** rossella_s has joined #openstack-infra10:55
*** kjackal has quit IRC10:59
*** yamamoto has quit IRC11:00
*** yamamoto has joined #openstack-infra11:00
*** rossella_s has quit IRC11:04
*** rossella_s has joined #openstack-infra11:05
*** yamamoto has quit IRC11:07
*** psachin has joined #openstack-infra11:12
*** aidin has quit IRC11:16
*** rcernin has quit IRC11:17
*** rossella_s has quit IRC11:19
*** rossella_s has joined #openstack-infra11:19
*** aidin has joined #openstack-infra11:21
*** kjackal has joined #openstack-infra11:26
*** jamesdenton has quit IRC11:30
*** rossella_s has quit IRC11:30
*** rossella_s has joined #openstack-infra11:31
*** rpittau has joined #openstack-infra11:35
*** aidin has quit IRC11:37
*** rossella_s has quit IRC11:39
*** rossella_s has joined #openstack-infra11:39
*** yamamoto has joined #openstack-infra11:46
*** jrist has quit IRC11:49
*** jrist has joined #openstack-infra11:50
*** rossella_s has quit IRC11:50
*** jrist has quit IRC11:54
*** rossella_s has joined #openstack-infra11:54
*** kjackal has quit IRC11:55
*** jrist has joined #openstack-infra11:57
*** yamamoto has quit IRC11:59
*** yamamoto has joined #openstack-infra11:59
*** yamamoto has quit IRC12:01
*** yamamoto has joined #openstack-infra12:01
*** kjackal has joined #openstack-infra12:03
*** rlandy has joined #openstack-infra12:04
*** e0ne has quit IRC12:04
*** yamamoto has quit IRC12:06
*** yamamoto has joined #openstack-infra12:08
*** rh-jelabarre has joined #openstack-infra12:08
*** rossella_s has quit IRC12:09
*** rossella_s has joined #openstack-infra12:12
*** yamamoto has quit IRC12:12
*** e0ne has joined #openstack-infra12:13
*** boden has joined #openstack-infra12:13
*** trown|outtypewww is now known as trown12:18
*** weshay_pto is now known as weshay12:19
*** jamesdenton has joined #openstack-infra12:20
*** quiquell|brb is now known as quiquell12:20
*** armax has quit IRC12:27
fungiheads up, i'm on the road for the first half of my day but should be around by ~1700z or so12:38
*** roman_g has quit IRC12:39
*** fried_rice is now known as efried12:49
*** yamamoto has joined #openstack-infra12:51
*** ansmith has joined #openstack-infra12:51
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add install and deploy openshift roles.  https://review.openstack.org/60861012:52
*** lbragstad has joined #openstack-infra12:54
*** ijw has joined #openstack-infra12:56
*** ijw has quit IRC12:56
*** kgiusti has joined #openstack-infra12:57
*** ijw has joined #openstack-infra12:57
*** ijw has quit IRC12:59
*** ijw_ has joined #openstack-infra13:00
*** psachin has quit IRC13:01
*** eharney has joined #openstack-infra13:02
*** aidin has joined #openstack-infra13:05
*** roman_g has joined #openstack-infra13:05
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066713:05
*** e0ne has quit IRC13:05
*** rossella_s has quit IRC13:07
*** rossella_s has joined #openstack-infra13:07
*** boden has quit IRC13:08
*** boden has joined #openstack-infra13:13
*** mriedem has joined #openstack-infra13:15
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul-jobs master: Add role to install kubernetes  https://review.openstack.org/60582313:21
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Ignore removed provider in _cleanupLeakedInstances  https://review.openstack.org/60867013:22
*** lbragstad has quit IRC13:24
*** bobh has joined #openstack-infra13:27
*** rossella_s has quit IRC13:28
*** rossella_s has joined #openstack-infra13:29
*** lbragstad has joined #openstack-infra13:32
*** agopi is now known as agopi|brb13:36
*** aidin has quit IRC13:37
*** zul has quit IRC13:38
*** jaosorior has quit IRC13:39
*** agopi|brb has quit IRC13:41
*** rossella_s has quit IRC13:42
*** rossella_s has joined #openstack-infra13:43
*** janki has quit IRC13:51
*** janki has joined #openstack-infra13:52
*** armax has joined #openstack-infra13:53
*** kukacz has quit IRC13:54
*** kukacz has joined #openstack-infra13:54
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: Add install and deploy openshift roles.  https://review.openstack.org/60861013:56
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066713:56
openstackgerritDoug Hellmann proposed openstack-infra/yaml2ical master: add monthly recurrence options  https://review.openstack.org/60868014:01
openstackgerritDoug Hellmann proposed openstack-infra/irc-meetings master: add tc meetings  https://review.openstack.org/60868214:02
*** armax has quit IRC14:06
*** munimeha1 has joined #openstack-infra14:07
*** agopi|brb has joined #openstack-infra14:08
*** agopi|brb is now known as agopi14:09
roman_gHello team. I'm trying to play with null nodes set (nodes: []), but my job does not get picked up by Zuul. Could someone take a look and advice, please? Thank you! https://review.openstack.org/#/c/608376/14:09
roman_gZuul says "SUCCESS", whereas I have commented out all jobs, but added my new one.14:10
roman_gI'm probably missing something important here.14:10
*** SteelyDan is now known as dansmith14:13
*** rossella_s has quit IRC14:16
Shrewsroman_g: i think the job will not run because of the 'files' list you have there. you aren't changing any of those files in that change14:16
*** dabukalam has joined #openstack-infra14:17
Shrewsroman_g: so maybe make a test change that depends on that to see if it works?14:17
*** e0ne has joined #openstack-infra14:17
roman_gShrews: shame on me, you are right.14:17
roman_gThank you!14:17
*** olivierb has quit IRC14:21
*** beekneemech is now known as bnemec14:22
*** eharney has quit IRC14:24
*** rossella_s has joined #openstack-infra14:25
*** roman_g has quit IRC14:28
*** lpetrut has joined #openstack-infra14:32
*** rossella_s has quit IRC14:36
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Remove tox-cover job  https://review.openstack.org/60868814:38
*** rossella_s has joined #openstack-infra14:39
*** quiquell is now known as quiquell|off14:44
*** rossella_s has quit IRC14:44
*** rossella_s has joined #openstack-infra14:46
*** Emine has joined #openstack-infra14:49
AJaegerdhellmann, clarkb: dhellmann has a list of reviews for python3-first job move into projects at http://paste.openstack.org/show/731626/ that are failing consistently and we we no progress on. We discussed briefly to ask some infra-root to merge those automatically since they are a 1:1 copy of the jobs they have today. What do you think?14:49
*** gcb_ has joined #openstack-infra14:53
*** e0ne has quit IRC14:54
*** lpetrut has quit IRC14:57
*** ijw_ has quit IRC14:58
*** ijw has joined #openstack-infra14:58
clarkbAJaeger: should be fine, but we should note on the submit comment that we did this due to failing tests that prevented us from merging normally14:59
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: WIP: Second try to fix single cloud config reload  https://review.openstack.org/60869514:59
openstackgerritDoug Hellmann proposed openstack-infra/yaml2ical master: add monthly recurrence options  https://review.openstack.org/60868015:00
dhellmannclarkb , AJaeger : I'll take the plan to the mailing list, too, for the record15:01
AJaegerdhellmann: good idea - thanks!15:01
dhellmannclarkb : I don't know how the process for bypassing the tests works. Is that something you do via gerrit? Do I need to do something other than provide that list of reviews?15:01
clarkbdhellmann: a gerrit admin sets +2 verified +2 code review abd +1 approval then clicks the "submit" button in gerrit which merges the change15:02
clarkba list would be great15:02
dhellmannok, the pastebin has the list of URLs (there are ~17) and I'll include the list in the email, t oo15:02
*** mriedem has quit IRC15:03
clarkbperfect15:03
*** dklyle has joined #openstack-infra15:04
*** mriedem has joined #openstack-infra15:04
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fix on heuristic to get Summit Pages  https://review.openstack.org/60834115:04
dhellmannclarkb , AJaeger : http://lists.openstack.org/pipermail/openstack-dev/2018-October/135527.html15:07
dhellmannclarkb, AJaeger : maybe if we haven't heard any objections by Thursday we can do it then?15:08
dhellmannI'm out for 2 weeks starting friday afternoon...15:08
clarkbI'll be around thursday and can fo it then15:09
*** e0ne has joined #openstack-infra15:11
dalvarezhi im trying to get openstack-health to work and when im running the frontend with gulp dev, im getting the following error15:11
dalvarez    at Object.<anonymous> (/root/openstack-health/node_modules/karma/node_modules/socket.io/lib/manager.js:16:13)15:11
AJaegerdhellmann: Thursday works for me...15:11
dalvarez    at Module._compile (module.js:653:30)15:11
dalvarezany clues somebody? the API seems to work, I populated the db with some data15:12
dhellmannclarkb , AJaeger : great, thanks15:12
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add tox-py36 job  https://review.openstack.org/60870015:14
*** lifeless has quit IRC15:16
*** janki has quit IRC15:20
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add tox-py36 job  https://review.openstack.org/60870015:21
*** Emine has quit IRC15:21
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenShift resource provider  https://review.openstack.org/57066715:21
openstackgerritMerged openstack-infra/openstackid-resources master: Fix on heuristic to get Summit Pages  https://review.openstack.org/60834115:22
*** panda is now known as panda|off15:23
*** coolsvap has quit IRC15:24
*** agopi is now known as agopi|lunch15:32
openstackgerritDoug Hellmann proposed openstack-infra/python-storyboardclient master: add script for tagging stories  https://review.openstack.org/60870715:34
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Create ansible-galaxy-import role  https://review.openstack.org/60847215:34
*** rossella_s has quit IRC15:35
*** rossella_s has joined #openstack-infra15:38
*** e0ne has quit IRC15:40
openstackgerritJames E. Blair proposed openstack-infra/puppet-zuul master: Revert "Revert "Add rewrite rules for HTML5 url""  https://review.openstack.org/60870915:42
*** hrubi has joined #openstack-infra15:42
*** e0ne has joined #openstack-infra15:42
corvusclarkb, pabelanger: https://review.openstack.org/608709 is needed for the react patch, which i'd like to merge.  can you +2 that?  (i'll handle the +3)15:43
pabelangerdone15:46
*** jamesmcarthur has quit IRC15:47
*** jamesmcarthur has joined #openstack-infra15:47
*** e0ne has quit IRC15:49
*** rossella_s has quit IRC15:51
*** sshnaidm is now known as sshnaidm|afk15:51
*** rossella_s has joined #openstack-infra15:51
fungii'm back now too, got home a little earlier than expected and procrastinating on errands for now15:52
*** ssbarnea|bkp2 has joined #openstack-infra15:53
AJaegerclarkb, corvus, fungi, pabelanger, we have patches from ssbarnea on silencing rsync for our repos - want to take those - or -1 since they make debugging harder? changes are https://review.openstack.org/603420 https://review.openstack.org/598599 https://review.openstack.org/60342615:54
*** rossella_s has quit IRC15:56
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: remove job settings for barbican repositories  https://review.openstack.org/59966315:59
*** rossella_s has joined #openstack-infra16:01
*** ginopc has quit IRC16:02
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: remove job settings for Telemetry repositories  https://review.openstack.org/59867116:02
AJaegerdhellmann: want to WIP those two above again? I just rebased ^16:03
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Add query for volume extend timeout bug 1796708  https://review.openstack.org/60872016:03
openstackbug 1796708 in Cinder "VolumesExtendTest.test_volume_extend_when_volume_has_snapshot intermittently fails with "Extend volume failed.: VolumeNotDeactivated: Volume volume-5514a6ad-abbb-46b3-a464-d73cc67e55af was not deactivated in time."" [Undecided,New] https://launchpad.net/bugs/179670816:03
*** gcb_ has quit IRC16:03
dhellmannAJaeger : I'm heading on vacation for 2 weeks starting next week. Maybe it would be easier if you WIP them, so you can just remove the WIP when they're ready?16:04
AJaegerdhellmann: ok, let me do that...16:05
dhellmannAJaeger : thanks16:05
corvusAJaeger: i don't want to change those rsyncs.  not if anyone expects to be able to debug job failures.16:06
*** kjackal has quit IRC16:07
openstackgerritMerged openstack-infra/nodepool master: Remove tox-cover job  https://review.openstack.org/60868816:07
AJaegercorvus: want to hand out some -1s or -2s then? Me continues to ignore the reviews in this case...16:07
*** jrist has quit IRC16:07
*** kjackal has joined #openstack-infra16:08
corvusAJaeger: -1 603420.  I'm ambivalent about the others.16:10
*** agopi|lunch is now known as agopi16:11
*** jrist has joined #openstack-infra16:12
*** jrist has quit IRC16:17
*** aojeagarcia has quit IRC16:19
*** dtantsur is now known as dtantsur|afk16:20
*** panda|off has quit IRC16:22
*** panda has joined #openstack-infra16:23
*** efried has quit IRC16:30
*** rossella_s has quit IRC16:31
*** jpich has quit IRC16:31
*** rossella_s has joined #openstack-infra16:33
*** efried has joined #openstack-infra16:37
*** aidin has joined #openstack-infra16:38
openstackgerritMerged openstack-infra/system-config master: modules: mirror: opensuse: Ensure that old distros are wiped from mirrors  https://review.openstack.org/60070116:38
openstackgerritMerged openstack-infra/system-config master: Run puppet apply test serially on centos  https://review.openstack.org/60027016:38
*** ramishra has quit IRC16:40
*** aidin has quit IRC16:42
*** kaiokmo has quit IRC16:43
*** eharney has joined #openstack-infra16:43
*** rossella_s has quit IRC16:47
*** gyee has joined #openstack-infra16:48
*** sshnaidm|afk is now known as sshnaidm16:48
*** rossella_s has joined #openstack-infra16:48
openstackgerritDoug Hellmann proposed openstack-infra/storyboard master: accept tag values in the body as well as URL  https://review.openstack.org/60872716:50
*** diablo_rojo_phon has joined #openstack-infra16:52
openstackgerritDoug Hellmann proposed openstack-infra/python-storyboardclient master: add script for tagging stories  https://review.openstack.org/60870716:52
openstackgerritMerged openstack-infra/system-config master: Add digit support to etherpad servers  https://review.openstack.org/59101716:52
*** kaiokmo has joined #openstack-infra16:53
clarkbpackethost mirror is unhappy according to e-r16:54
clarkbanyone able to take a look at that? (I'm attending a conference an dtrying to pay attention:) )16:54
fungilooking now16:55
fungiping to mirror02.us-west-1.packethost.openstack.org is coming back with icmp-unreach16:56
fungino route to host16:57
*** rossella_s has quit IRC16:57
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fixed CFP endpoints  https://review.openstack.org/60872816:58
fungiserver show says status is SHUTOFF16:58
*** rossella_s has joined #openstack-infra16:58
fungiissuing a server start via the api now16:59
clarkbthanks for checking16:59
fungii can ping it16:59
fungiit's booted and reachable via ssh now17:00
clarkbweb server is there too17:00
fungilast entry in syslog prior to the reboot was the usual udp connection entry spam for snmp polling at 13:56:23z today17:01
fungiso it's probably been offline for ~3 hours17:02
*** trown is now known as trown|lunch17:02
pabelangerspeaking of packethost, do we have any idea of the baremetal we are using there for the cloud? I don't think I've seen info about that17:03
fungii have no idea what they deployed17:03
fungiwe don't manage it directly17:03
openstackgerritMerged openstack-infra/zuul-jobs master: Add test workspace setup role  https://review.openstack.org/60834217:03
clarkbpabelanger: its one of the large intel flavors in packethost, but john was talking about trying with the new epyc amd flavor because that gives us more cores (and cpu cores are the biggest constraint currently)17:04
clarkbbut I don't remember the specific details17:04
fungi#status log started mirror02.us-west-1.packethost.openstack.org via openstackclient after ~3 hours in SHUTOFF state17:04
openstackstatusfungi: finished logging17:04
fungihuh, so syslog shows a puppet run was underway, completing right before something issued a shutdown17:05
pabelangerclarkb: cool, just started looking at packethost over the return flight. Good info17:05
fungi~1 minute after17:06
fungiOct  8 13:44:37 mirror02 puppet-user[8891]: Finished catalog run in 0.43 seconds17:06
fungiOct  8 13:45:37 mirror02 systemd[1]: Stopping User Manager for UID 0...17:06
fungiOct  8 13:45:37 mirror02 systemd[8783]: Reached target Shutdown.17:06
fungiahh, no, ignore that17:07
fungistill getting used to systemd17:07
fungiit's just reporting closing out the ssh session for the puppet run i think17:07
fungiOct  8 13:45:37 mirror02 systemd[1]: Removed slice User Slice of root.17:08
fungithat was the end of it17:08
fungisystem didn't go away for at least another 10 minutes17:08
funginothing in syslog to explain it, so not a graceful shutdown of any sort17:08
clarkbthe cloud may have done it then. In the past (both packethots and limestone) we saw that when the hypervisor ran out of disk which crashed the VM(s)17:09
fungithanks, sounds likely enough17:10
fungiconcluding my rca for now, at least until studarus is on hand to look at whatever bits he has access to17:11
fungii don't think we can tell much more from this end17:11
*** jamesmcarthur has quit IRC17:11
clarkbI wonder if nova can support a flavor that isn't thin provisioned disk17:11
*** apetrich has quit IRC17:11
clarkbthen we can use that for our mirrors17:11
clarkblogan-: mnaser ^ may know off hte top of their heads17:12
fungior maybe we could cp /dev/zero into /filler and then rm it after the enospc17:12
logan-^ yep you could thick provision it yourself17:13
fungiassuming no backend deduping zero-blocks17:13
*** jamesmcarthur has joined #openstack-infra17:14
*** harlowja has joined #openstack-infra17:15
clarkbfungi: maybe don't write right up to the edge to prevent it crashing again17:17
clarkbbut may be worth trying on on the root device of our mirrors17:17
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: docker-compose quickstart example  https://review.openstack.org/60834417:22
*** e0ne has joined #openstack-infra17:24
*** rossella_s has quit IRC17:25
kmallocclarkb, fungi, mordred: Since ubuntu one is using OpenID 2.0, I'm exploring the recent releases of Ipsilon to see if it can act as an identity broker for us instead of needing custom code.17:25
kmallocOpenID 2.0 precludes use of Dex and Keycloak.17:25
kmalloc(without a bunch more code)17:25
kmallocIpsilon might do what we need.17:25
*** rossella_s has joined #openstack-infra17:27
*** e0ne has quit IRC17:29
*** ssbarnea_ has joined #openstack-infra17:29
*** ssbarnea|bkp2 has quit IRC17:30
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fixed CFP endpoints  https://review.openstack.org/60872817:30
*** sshnaidm is now known as sshnaidm|afk17:32
openstackgerritMerged openstack-infra/openstackid-resources master: Fixed CFP endpoints  https://review.openstack.org/60872817:34
*** rossella_s has quit IRC17:35
*** mlavalle has joined #openstack-infra17:36
openstackgerritMerged openstack-infra/project-config master: Test workspace setup role changes  https://review.openstack.org/60834317:36
openstackgerritClark Boylan proposed openstack-infra/zuul-jobs master: Do no merge testing with base-test  https://review.openstack.org/60873217:36
mlavallehi, we have a feature branch in Neutron: feature/graphql. I just rebased it locally on master. How do I push it to Neutron repo?17:36
clarkbif ^ works then I think the setup retry steps are safe to merge17:36
clarkbmlavalle: you need to merge master into the branch then push the merge to the feature branch if you wnat to pick up latest changes on master on the feature branch17:37
clarkbmlavalle: if you rebase on master instead and try to push that it is not fast forwardable and everyone downstream has to do force updates to their git repos17:37
*** rossella_s has joined #openstack-infra17:38
clarkbmlavalle: swift has great info on how they've used feature branches at https://wiki.openstack.org/wiki/Swift/feature_branches17:38
mlavalleclarkb: cool. Thanks. I'll take a look :-)17:38
*** shardy has quit IRC17:40
AJaegerdhellmann: want to remove your WIP from https://review.openstack.org/#/c/597865/ ? I added mine...17:42
dhellmannAJaeger : done17:42
AJaegerthanks, dhellmann17:42
*** jamesmcarthur has quit IRC17:47
*** apetrich has joined #openstack-infra17:47
*** rossella_s has quit IRC17:52
fungikmalloc: corvus set up an ipsilon poc a while back and may have some suggestions, though i thought we had ruled out ipsilon because it's dead upstream?17:53
*** e0ne has joined #openstack-infra17:54
*** jamesmcarthur has joined #openstack-infra17:55
kmallocfungi: it looks like it's Fedora Account Services code.17:55
fungiyeah, but someone familiar with the situation suggested they're moving off it and have stopped developing it or something17:55
kmallocfungi: it might be somewhat dead, but if it works for the time being we should be good until keystone can be brought up to par. this is a stop-gap solution17:56
kmallocimo.17:56
fungigot it17:56
kmalloci can totally write custom code to do this as well17:56
*** jamesmcarthur has quit IRC17:56
kmallocbut if something already does it, i'm happy to use that and focus on keystone things17:56
fungisure!17:56
kmalloc(and getting keystone up to where it needs to be) :)17:56
*** rossella_s has joined #openstack-infra17:56
corvuskmalloc, fungi: https://review.openstack.org/344368 https://review.openstack.org/34437218:00
kmalloccorvus: i thought I rememebered looking at those18:02
kmallocnot sure how well ipsilon acts as a broker.18:02
kmallocbut i'll poke at it here shortly18:02
*** rossella_s has quit IRC18:03
corvuskmalloc: sounds good18:03
kmallocworst case, I'll roll a simple flask app that does the brokering for us.18:03
kmallocbut I'm trying really hard to not need to write custom code for stop-gap solution (short term) if I can avoid it18:04
kmallocit's never a small amount of code for this stuff :P18:04
*** electrofelix has quit IRC18:05
*** rossella_s has joined #openstack-infra18:05
openstackgerritMerged openstack-infra/elastic-recheck master: Add query for volume extend timeout bug 1796708  https://review.openstack.org/60872018:09
openstackbug 1796708 in Cinder "VolumesExtendTest.test_volume_extend_when_volume_has_snapshot intermittently fails with "Extend volume failed.: VolumeNotDeactivated: Volume volume-5514a6ad-abbb-46b3-a464-d73cc67e55af was not deactivated in time."" [Undecided,New] https://launchpad.net/bugs/179670818:09
*** trown|lunch is now known as trown18:11
*** Swami has joined #openstack-infra18:13
*** rossella_s has quit IRC18:16
*** rossella_s has joined #openstack-infra18:18
*** jamesmcarthur has joined #openstack-infra18:20
*** imacdonn has quit IRC18:23
*** imacdonn has joined #openstack-infra18:23
*** jamesmcarthur has quit IRC18:25
*** panda has quit IRC18:26
*** panda has joined #openstack-infra18:28
*** rossella_s has quit IRC18:38
*** rossella_s has joined #openstack-infra18:39
openstackgerritMerged openstack-infra/zuul master: Add coreutils to bindep on platform apk  https://review.openstack.org/60631418:42
*** kopecmartin|ruck is now known as kopecmartin|off18:45
*** slagle has joined #openstack-infra18:52
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Fix pbr setup hook  https://review.openstack.org/60875318:53
*** rockyg has joined #openstack-infra18:55
*** rosmaita has joined #openstack-infra18:58
*** jamesmcarthur has joined #openstack-infra19:00
*** rossella_s has quit IRC19:01
*** eharney has quit IRC19:02
*** eernst_ has joined #openstack-infra19:03
*** rossella_s has joined #openstack-infra19:03
*** diablo_rojo has joined #openstack-infra19:04
*** eharney has joined #openstack-infra19:04
*** sambetts|afk has quit IRC19:07
openstackgerritMerged openstack-infra/zuul master: Revert "Revert "web: rewrite interface in react""  https://review.openstack.org/60747919:08
mnaserclarkb: nope, disk allocation is not controlled by the user19:13
mnaserinfra-root: this may be late but is packethost stuff resolved?19:15
mnaserhttp://logs.openstack.org/27/608427/5/check/openstack-ansible-deploy-aio_basekit-centos-7/f4a953c/job-output.txt.gz#_2018-10-08_16_17_02_86520919:15
mnaseras of 2 hours ago19:15
fungimnaser: yeah, should have been working again as of 1700z19:15
mnaserfungi: wonderful.  thank you19:15
fungiyw!19:16
*** rossella_s has quit IRC19:18
*** mlavalle has quit IRC19:19
*** rossella_s has joined #openstack-infra19:20
clarkbinfra-root and config-core https://review.openstack.org/#/c/608732/ passes testing. http://logs.openstack.org/32/608732/1/check/tox-py27/d5db2f3/ara-report/file/dfc173c7-ef2c-4865-bbcb-bc6db35dcdd9/#line-9 shows the tests used the retries: 3 code (though I don't think it actually had to retry in this case)19:21
clarkbhttps://review.openstack.org/#/c/608303/ is the change that is testing19:21
clarkbI am in a bad spot to merge a chnage to all tests today and tomorrow but if we can get that in it would be helpful for jobs reseting19:22
*** kjackal has quit IRC19:30
*** kjackal has joined #openstack-infra19:31
*** sthussey has joined #openstack-infra19:36
*** slagle has quit IRC19:40
*** diablo_rojo has quit IRC19:42
*** jamesmcarthur has quit IRC19:49
*** jamesmcarthur has joined #openstack-infra19:51
*** jamesmcarthur has quit IRC19:52
*** jamesmcarthur has joined #openstack-infra19:56
*** rossella_s has quit IRC19:57
*** jamesmca_ has joined #openstack-infra19:58
*** jamesmca_ has quit IRC19:59
*** rossella_s has joined #openstack-infra19:59
*** jamesmcarthur has quit IRC20:00
*** rockyg has quit IRC20:05
*** jamesmcarthur has joined #openstack-infra20:10
*** jtomasek has quit IRC20:13
*** e0ne has quit IRC20:19
*** slagle has joined #openstack-infra20:20
*** e0ne has joined #openstack-infra20:20
*** dayou has quit IRC20:24
*** dayou has joined #openstack-infra20:24
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: docker-compose quickstart example  https://review.openstack.org/60834420:28
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Support mandatory proxy in docker-compose example  https://review.openstack.org/60876420:30
*** kjackal has quit IRC20:31
*** eharney has quit IRC20:32
*** lbragstad has quit IRC20:37
*** slagle has quit IRC20:37
*** lbragstad has joined #openstack-infra20:37
*** kgiusti has left #openstack-infra20:38
openstackgerritMerged openstack-infra/puppet-zuul master: Revert "Revert "Add rewrite rules for HTML5 url""  https://review.openstack.org/60870920:40
clarkbcorvus: ^ fyi20:40
*** rosmaita has left #openstack-infra20:41
corvusthanks; once that applies, the dashboard should break and then be fixed with a zuul-web restart20:42
*** ansmith has quit IRC20:47
*** e0ne has quit IRC20:48
*** pcaruana has quit IRC20:49
*** jamesmcarthur has quit IRC20:54
*** rossella_s has quit IRC20:55
*** rossella_s has joined #openstack-infra20:56
*** trown is now known as trown|outtypewww20:59
*** jamesmcarthur has joined #openstack-infra21:00
*** bobh has quit IRC21:01
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: docker-compose quickstart example  https://review.openstack.org/60834421:08
openstackgerritMerged openstack-infra/zuul master: Fix pbr setup hook  https://review.openstack.org/60875321:16
fungicorvus: looks like the zuul web dashboard is broken now. were you planning to restart zuul-web or want me to?21:19
fungithe revert of the revert of the react rewrite seems to be installed on zuul.o.o too21:21
fungizuul==3.2.1.dev98  # git sha 4e915e121:21
*** jamesmcarthur has quit IRC21:21
fungiper `pbr freeze`21:21
corvusfungi: ah yes i'll restart it now21:22
fungistanding by to help as needed21:22
corvuswelp something still seems to be wrong21:23
tobiashcorvus: what about the apache rules?21:24
tobiashwasn't there a change for this too?21:25
corvustobiash: yes, it merged and should be in place21:25
fungitobiash: https://review.openstack.org/60870921:25
fungiit is in place, i checked the vhost config on the server21:25
fungi[Mon Oct 08 21:22:40.854056 2018] [proxy:error] [pid 27318:tid 139688266409728] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:9000 (*) failed21:26
corvusfungi: i think it's connecting now21:26
corvushttp://zuul.openstack.org/api/info is working21:26
fungiahh, yeah that was probably in the middle of the restart21:27
fungihttps://zuul.openstack.org/api/status seems to work too21:27
tobiashcorvus: I get the html for every request21:27
tobiash(even if I request the js)21:28
tobiashmaybe there is a bug in the apache rules?21:28
tobiashcurl http://zuul.openstack.org/static/main.60fff23a4aff71c7d918.js21:28
tobiashthat should return the static js but returns the index.html21:29
fungiRewriteRule ^/$ /status [L]21:29
fungiRewriteRule ^/status$ http://127.0.0.1:9000/api/tenant/openstack/status [P,L]21:29
tobiashcorvus: fungi: https://review.openstack.org/#/c/608709/1/templates/zuulv3.vhost.erb21:30
fungiso the first is rewriting to the second yeah?21:30
tobiashI think it's the html5 rewrite rule, that needs to be scoped to *.html21:31
fungiyeah, that's the change i'm looking at21:31
tobiashin zuul-web it is scoped to *.html...21:31
*** bobh has joined #openstack-infra21:33
fungi[08/Oct/2018:21:35:35 +0000] "GET /runtime~main.03196f08e5fd472eae3d.js HTTP/1.1" 200 1424 "http://zuul.openstack.org/status.html"21:35
corvusi commented out the html5 rewrite rules21:36
fungibut yeah, seems to just serve the status page21:36
corvusoh ha21:36
fungiwe don't have the rewrite logging turned on, i guess21:37
corvusthe js content publication hasn't happened yet21:37
fungioh, and those files dne, so it's rewriting to the fallback?21:37
corvusi expected it to happen faster.  we're still on the old js21:37
corvus(so merging the rewrite rules change was premature)21:37
corvusand it failed: http://logs.openstack.org/c7/c7148815378b9f26751ec0729a96894e7cd6b4c3/post/publish-openstack-javascript-content/05c6626/ara-report/result/72fa45d7-c12e-4af5-b520-5e1e73e4853b/21:38
tobiashso next revert?21:39
fungii guess that implies the src/git.openstack.org/openstack-infra/zuul/web directory was nonexistent21:39
corvusso we should fast-merge a revert of the puppet change.  we will continue to run on the old js until that job is fixed or the react change is reverted.21:39
corvusthen we can decide whether we want to revert react again, or roll forward with a fix of the js build job21:39
openstackgerritJames E. Blair proposed openstack-infra/puppet-zuul master: Revert "Revert "Revert "Add rewrite rules for HTML5 url"""  https://review.openstack.org/60877821:40
fungithanks, you're faster than me!21:41
corvusfungi: you want to approve that, i'll gate-enqueue it21:41
fungiapproved21:41
fungithanks!21:41
*** bobh has quit IRC21:42
*** agopi has quit IRC21:42
corvusit's enqueued21:42
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: docker-compose quickstart example  https://review.openstack.org/60834421:44
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Revert "Revert "Revert "web: rewrite interface in react"""  https://review.openstack.org/60877921:45
tobiashcorvus: fungi the cwd is "src/git.openstack.org/openstack-infra/zuul/web/build21:47
tobiashThat's why the destdir doesn't exist21:47
*** bobh has joined #openstack-infra21:47
fungiahh, so we actually want ../ instead?21:48
*** rh-jelabarre has quit IRC21:48
tobiashI think so21:48
fungior to set cwd to the workdir21:48
fungiso that the relative path to src/git.openstack.org/openstack-infra/zuul/web is valid21:49
tobiashNo, the cwd is correct because the sourvrs are there21:49
tobiashSo the target file.must be changed21:49
*** slaweq has quit IRC22:01
*** eernst_ has quit IRC22:04
*** agopi has joined #openstack-infra22:05
*** eernst has joined #openstack-infra22:06
*** boden has quit IRC22:10
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Update query for bug 1763181  https://review.openstack.org/60878222:11
openstackbug 1763181 in OpenStack Compute (nova) "test_parallel_evacuate_with_server_group intermittently fails" [Medium,In progress] https://launchpad.net/bugs/1763181 - Assigned to Matthew Booth (mbooth-9)22:11
*** eernst has quit IRC22:11
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: docker-compose quickstart example  https://review.openstack.org/60834422:11
clarkbtobiash: fungi: where are the paths wrgon?22:12
fungiclarkb: http://logs.openstack.org/c7/c7148815378b9f26751ec0729a96894e7cd6b4c3/post/publish-openstack-javascript-content/05c6626/ara-report/result/72fa45d7-c12e-4af5-b520-5e1e73e4853b/22:16
clarkbah in publishing the js22:16
openstackgerritMerged openstack-infra/puppet-zuul master: Revert "Revert "Revert "Add rewrite rules for HTML5 url"""  https://review.openstack.org/60877822:16
*** bobh has quit IRC22:19
*** ijw has quit IRC22:19
*** bobh has joined #openstack-infra22:24
*** bobh has quit IRC22:29
openstackgerritMerged openstack-infra/elastic-recheck master: Update query for bug 1763181  https://review.openstack.org/60878222:34
openstackbug 1763181 in OpenStack Compute (nova) "test_parallel_evacuate_with_server_group intermittently fails" [Medium,In progress] https://launchpad.net/bugs/1763181 - Assigned to Matthew Booth (mbooth-9)22:34
rpiosoA gertty newb question ... I'm running gertty in an lxterminal on Fedora. I can't seem to copy clipboard content into a review comment. Is that possible, and, if so, how?22:35
*** ansmith has joined #openstack-infra22:35
*** dave-mccowan has quit IRC22:49
*** munimeha1 has quit IRC22:50
*** rcernin has joined #openstack-infra22:50
*** dave-mccowan has joined #openstack-infra22:50
fungirpioso: if like me you're used to using middle-click paste from the x11 buffer, you might want to run gertty with handle-mouse: false22:52
fungii'm not a fan of pointer integration in console utilities22:52
fungi(for reasons such as the above)22:52
rpiosofungi: I'll give that a try. Thank you!22:55
*** mriedem has quit IRC22:57
*** tosky has quit IRC23:03
rpiosofungi: Puuuuurfect!23:04
clarkbfungi: is packethost mirror unhappy again?23:04
*** pbourke has quit IRC23:08
*** pbourke has joined #openstack-infra23:08
fungiSHUTOFF23:08
fungii've done an openstack server start on it again23:08
clarkbwe might want to disable in nodepool until we can wrote those zeros?23:09
fungi#status log started mirror02.us-west-1.packethost.openstack.org via openstackclient after ~4 hours in SHUTOFF state23:11
openstackstatusfungi: finished logging23:11
*** rlandy is now known as rlandy|bbl23:11
fungii'm preallocating the rest of the rootfs on it now23:12
fungiwill keep an eye on it for a bit23:12
clarkbok thanks23:12
*** lbragstad has quit IRC23:16
fungicp: error writing '/filler': No space left on device23:17
fungiso that's done23:17
fungii had it trailed with a ;rm /filler23:17
fungiso df is showing / still only 35% used23:17
fungi#status log preallocated remaining space on mirror02.us-west-1.packethost.openstack.org rootfs by writing /dev/zero to a file and then removing it23:19
openstackstatusfungi: finished logging23:19
clarkbany idea hoe full it got before failing?23:23
*** gema has quit IRC23:26
*** quiquell|off has quit IRC23:27
funginope, but that's a great point. i'll rerun it with a df just before the rm23:30
*** bradm has joined #openstack-infra23:31
dmsimardReading gerritbot all over https://github.com/golang/go/wiki/GerritBot hurts my brain :P23:35
dmsimard(because it's something completely different than our gerritbot)23:35
fungiclarkb: 100% used23:37
*** Swami has quit IRC23:37
clarkbgreat23:38
*** tpsilva has quit IRC23:41
*** gyee has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!