Wednesday, 2018-05-30

ianwmtreinish: ahh, there is a file there00:00
fungiianw: could it have already been dead at that point?00:00
fungitimestamp on the crash file?00:00
ianwwith a base64 encoded coredump00:00
fungiyeah, there's a tool to extract the coredump from that00:00
ianw-rw-r----- 1 root root 5.0M May 27 06:25 _usr_sbin_mosquitto.0.crash00:00
*** VW has quit IRC00:00
fungiright, so you were looking at too recent of a log00:01
fungibut the timing there does look like log rotation again00:01
fungijust a couple days prior00:01
ianwoh, it lines up.  the restart thought it was pid 8816 (from old pid file i guess) and the coredump thingy is saying it was for pid 881600:02
fungioh, sorry, i was looking at your start-stop-daemon output and mistaking it for something from syslog during log rotation00:03
fungiyeah, seems pretty likely to be the same problem00:03
fungiand i agree with clarkb, we've actually previously discussed changing the logrotate config to use a different rotation mechanism00:03
fungias a workaround00:03
ianw# apport-retrace -g ./_usr_sbin_mosquitto.0.crash00:04
ianwERROR: report file does not contain one of the required fields: CoreDump DistroRelease Package ExecutablePath00:04
ianwboo00:04
openstackgerritMichael Johnson proposed openstack-infra/zuul-jobs master: Collect the coverage report for npm test jobs  https://review.openstack.org/57026000:04
fungibut wanted to collect some good debugging info first, since obviously having a daemon which crashes on sighup could be similarly problematic for, e.g., configuration reloads00:04
*** geekinutah has quit IRC00:04
*** edwarnicke has quit IRC00:04
*** jamespage has quit IRC00:04
*** zul has quit IRC00:04
fungiianw: you want apport-unpack00:04
*** auggy_ has joined #openstack-infra00:05
*** electrical__ has joined #openstack-infra00:05
*** mgkwill_ has joined #openstack-infra00:05
*** ameade_ has joined #openstack-infra00:05
*** lamt__ has joined #openstack-infra00:05
fungie.g., `apport-unpack _usr_sbin_mosquitto.0.crash foo`00:05
*** patrickeast has quit IRC00:05
*** auggy has quit IRC00:05
*** mgkwill has quit IRC00:05
*** auggy_ is now known as auggy00:05
*** mgkwill_ is now known as mgkwill00:05
mtreinishfungi: to be fair a full restart is probably safe enough for mosquitto because mqtt handles the blips pretty well00:05
*** ildikov_ has joined #openstack-infra00:05
*** patrickeast has joined #openstack-infra00:05
fungiianw: followed by `gdb /usr/sbin/mosquitto foo/CoreDump`00:05
*** csatari_ has joined #openstack-infra00:05
mtreinishwell with the exception of the code I wrote which doesn't have any real error ahndling :p00:05
*** csatari has quit IRC00:05
*** ildikov has quit IRC00:05
*** ameade has quit IRC00:05
*** csatari_ is now known as csatari00:05
*** electrical_ has quit IRC00:05
*** odyssey4me has quit IRC00:05
*** ameade_ is now known as ameade00:05
*** ildikov_ is now known as ildikov00:05
*** electrical__ is now known as electrical_00:05
*** nicolasbock has quit IRC00:06
*** MasterofJOKers has quit IRC00:06
*** mordred has quit IRC00:06
ianwi hacked in a Package: field anyway00:06
ianwhttp://paste.openstack.org/show/722314/00:06
*** pahuang has quit IRC00:06
fungiianw: and before you ask, i'm just pulling that from my command history of what i managed to figure out last time after lots of reading00:06
*** HenryG has quit IRC00:06
*** zigo has quit IRC00:06
*** dhellmann has quit IRC00:06
*** Roamer` has quit IRC00:06
*** afazekas has quit IRC00:06
*** lamt has quit IRC00:06
*** melwitt has quit IRC00:06
*** lamt__ is now known as lamt00:06
*** zigo_ has joined #openstack-infra00:06
mtreinishianw: yeah that looks the same as what we hit last time00:07
*** HenryG has joined #openstack-infra00:07
*** r-daneel has quit IRC00:07
fungiagreed00:07
ianwfungi: so what do you do when the tool to extract the crash dump crashes? http://paste.openstack.org/show/722315/00:07
*** Roamer` has joined #openstack-infra00:08
fungihah!00:08
mtreinishlol00:08
fungiyeah, the old dump is at /home/fungi/_usr_sbin_mosquitto.0.crash if you want to compare metadata00:09
ianwbut ok, if it's similar bt i won't keep digging00:09
*** MasterofJOKers has joined #openstack-infra00:10
*** odyssey4me has joined #openstack-infra00:11
*** melwitt has joined #openstack-infra00:11
fungiand apport-unpack still seems to create a CoreDump file from the new crash file successfully even if it raises an exception00:11
*** dhellmann has joined #openstack-infra00:11
ianwok, yeah, bt looks the same in yours too00:11
*** afazekas has joined #openstack-infra00:11
*** ChanServ has quit IRC00:12
*** camunoz has quit IRC00:12
clarkbreading openstack_inventory.py I can probably just test it out of band of ansible00:13
*** sfeole` is now known as sfeole00:13
clarkbopenstacksdk is installed on puppet master00:13
clarkbheh ansibles dynamic inventory docs are all wrong becuse of this change too00:14
fungidocumentation for a free software project out of sync with reality? hey, at least it _has_ documentation! ;)00:15
*** mordred has joined #openstack-infra00:15
clarkboh sure I'm not complaining as much as revelling in how far this thread goes as I pull it :)00:15
*** geekinutah has joined #openstack-infra00:15
fungiyou can just blame mordred, he won't mind00:16
*** zul has joined #openstack-infra00:16
*** jamespage has joined #openstack-infra00:16
*** edwarnicke has joined #openstack-infra00:16
clarkblooks like I can run openstack_inventory --list --cloud somecloudname as a regular user and it won't try to update global caches00:18
clarkbhowever that potential global state is my biggest worry right now00:18
*** yolanda_ has quit IRC00:19
*** yolanda_ has joined #openstack-infra00:20
clarkbprobably best to test that in the morning so I don't make ansible puppet less happy00:20
*** rwsu has quit IRC00:20
clarkbI'd test it locally except I want ot make sure that puppetmaster will work with it :/00:21
*** vabada has quit IRC00:21
fungicompletely understandable00:21
*** jamesmcarthur has joined #openstack-infra00:22
*** pahuang has joined #openstack-infra00:23
*** jamesmcarthur has quit IRC00:27
*** caphrim007_ has quit IRC00:30
*** rlandy|rover has quit IRC00:33
*** ChanServ has joined #openstack-infra00:33
*** barjavel.freenode.net sets mode: +o ChanServ00:33
*** smcginnis[m] has quit IRC00:37
clarkbI'll pick it up in the morning I think the safe short term option is the pin on the repo, then we can test the newer version of the script and update puppetry to use that if/when it works00:37
*** lbragstad[m] has quit IRC00:37
*** kbaikov[m] has quit IRC00:37
*** masayukig1 has quit IRC00:37
*** aspiers[m] has quit IRC00:37
*** warlord77[m] has quit IRC00:38
*** lamt is now known as Guest4135800:39
*** melwitt is now known as Guest4815300:39
*** vabada has joined #openstack-infra00:39
*** nicolasbock has joined #openstack-infra00:40
*** dpawlik has joined #openstack-infra00:40
fungiwfm00:41
*** VW has joined #openstack-infra00:41
*** VW has quit IRC00:44
*** dhinesh_ has joined #openstack-infra00:45
*** dpawlik has quit IRC00:45
*** rwsu has joined #openstack-infra00:45
*** jesslampe has joined #openstack-infra00:50
*** annp has quit IRC00:53
*** annp has joined #openstack-infra00:54
*** jesslampe has quit IRC00:54
*** masayukig1 has joined #openstack-infra00:59
*** harlowja has quit IRC01:00
*** iyamahat has quit IRC01:06
*** dingyichen has joined #openstack-infra01:12
*** gongysh has joined #openstack-infra01:17
*** gyankum has joined #openstack-infra01:29
*** jamespage has quit IRC01:30
*** mordred has quit IRC01:31
*** gyee has quit IRC01:31
*** sree has joined #openstack-infra01:34
*** kien-ha has joined #openstack-infra01:37
radezianw: ping, hey Ian, I saw there was a review merged to fix https://review.openstack.org/#/c/569121/01:38
*** hongbin has joined #openstack-infra01:38
*** sree has quit IRC01:38
radezbut I appear to still be hitting it, is there a chance it hasn't propegated in some way or should I not be hitting it?01:38
*** warlord77[m] has joined #openstack-infra01:39
*** kbaikov[m] has joined #openstack-infra01:39
*** aspiers[m] has joined #openstack-infra01:39
*** lbragstad[m] has joined #openstack-infra01:39
*** smcginnis[m] has joined #openstack-infra01:39
ianwradez: was it rebased?01:39
radezdifferent project, I'm in openstack-requirements01:40
radezshould I have to rebase it there too>01:40
ianwoh, hang on, it's hitting hte cmd2 stuff right?01:40
radezianw: yup01:41
*** gongysh has quit IRC01:41
fungithe hopeful initial fix https://review.openstack.org/570811 merged at 23:39 utc01:41
fungiprobably better to discuss in #openstack-requirements01:41
ianwyeah, i think the earlier recheck probably just missed it, the last one done just now hopefully passes01:42
fungistill need the equivalent fix merged to cliff and then a new cliff release made before it'll be fixed in unconstrained jobs01:42
fungithe failing jobs in the reqs repo don't run with constraints applied (intentionally)01:42
fungithough it'll just be the py27 unit tests and tox-validate failing on the recheck, presumably01:43
*** jogo has quit IRC01:43
*** roman_g has quit IRC01:44
*** mordred has joined #openstack-infra01:44
*** jamespage has joined #openstack-infra01:44
*** felipemonteiro has joined #openstack-infra01:44
radezoh so I should have to rebase in the reqs repo?01:45
fungino, but the couple of jobs i mentioned will continue to fail for changes to reqs until a new cliff release happens01:45
prometheanfirewell, cliff passed01:46
fungimight want to drum up support in #openstack-oslo if nobody has already01:47
*** armaan has quit IRC01:48
*** armaan has joined #openstack-infra01:49
*** roman_g has joined #openstack-infra01:58
*** aeng has quit IRC02:06
*** dhinesh_ has quit IRC02:06
*** Guest41358 is now known as lamt02:06
*** aeng has joined #openstack-infra02:07
*** pahuang has quit IRC02:11
*** rajinir has joined #openstack-infra02:17
*** yamahata has quit IRC02:20
*** caphrim007 has joined #openstack-infra02:21
*** iyamahat has joined #openstack-infra02:21
*** iyamahat has quit IRC02:22
*** pahuang has joined #openstack-infra02:24
*** caphrim007 has quit IRC02:26
*** iyamahat has joined #openstack-infra02:26
*** iyamahat has quit IRC02:27
*** iyamahat has joined #openstack-infra02:28
*** iyamahat has quit IRC02:35
*** caphrim007 has joined #openstack-infra02:36
*** dklyle has joined #openstack-infra02:40
*** jogo has joined #openstack-infra02:41
*** psachin has joined #openstack-infra02:41
*** zhangfei has joined #openstack-infra02:43
*** caphrim007 has quit IRC02:44
*** caphrim007 has joined #openstack-infra02:45
*** markvoelker_ has quit IRC02:45
*** VW has joined #openstack-infra02:46
*** ykarel|away has joined #openstack-infra02:46
*** dbecker has quit IRC02:47
*** namnh has joined #openstack-infra02:48
*** dchen has joined #openstack-infra02:49
*** rlandy has joined #openstack-infra02:51
*** xinliang has quit IRC02:52
*** dklyle has quit IRC02:54
*** rosmaita has quit IRC02:57
*** xinliang has joined #openstack-infra02:58
*** dbecker has joined #openstack-infra03:01
*** masayuki- is now known as masayukig_03:08
*** masayukig is now known as masayukig__03:11
*** masayukig_ is now known as masayukig03:12
*** janki has joined #openstack-infra03:12
*** masayukig__ is now known as masayukig_03:13
*** auristor has quit IRC03:21
*** auristor has joined #openstack-infra03:27
*** harlowja has joined #openstack-infra03:29
*** felipemonteiro has quit IRC03:32
*** pahuang has quit IRC03:34
*** markvoelker has joined #openstack-infra03:35
*** jesslampe has joined #openstack-infra03:35
*** markvoelker has quit IRC03:39
*** lpetrut has joined #openstack-infra03:46
*** pahuang has joined #openstack-infra03:47
*** felipemonteiro has joined #openstack-infra03:49
*** hongbin has quit IRC03:53
*** nicolasbock has quit IRC03:55
*** udesale has joined #openstack-infra03:58
*** ykarel|away is now known as ykarel03:59
*** calebb has quit IRC04:05
*** rlandy has quit IRC04:05
*** calebb has joined #openstack-infra04:05
*** ykarel is now known as ykarel|afk04:07
*** links has joined #openstack-infra04:11
*** annp has quit IRC04:13
*** harlowja has quit IRC04:14
*** lpetrut has quit IRC04:19
*** markvoelker has joined #openstack-infra04:29
*** sree has joined #openstack-infra04:31
*** dpawlik has joined #openstack-infra04:42
*** pgadiya has joined #openstack-infra04:42
*** pgadiya has quit IRC04:44
*** psachin has quit IRC04:46
*** dpawlik has quit IRC04:47
*** katyafervent_awa has quit IRC04:51
*** rakhmerov has quit IRC04:51
*** igormarnat has quit IRC04:51
*** dteselkin has quit IRC04:51
*** asilenkov has quit IRC04:51
*** VW has quit IRC04:52
*** xarses has joined #openstack-infra04:55
*** xarses has quit IRC04:56
*** xarses has joined #openstack-infra04:56
*** xarses_ has quit IRC04:58
*** e0ne has joined #openstack-infra05:00
*** pahuang has quit IRC05:00
*** markvoelker has quit IRC05:03
*** pahuang has joined #openstack-infra05:04
*** xarses has quit IRC05:04
*** felipemonteiro has quit IRC05:05
*** dingyichen has quit IRC05:05
*** xarses has joined #openstack-infra05:05
*** xarses has quit IRC05:05
*** dingyichen has joined #openstack-infra05:05
*** xarses has joined #openstack-infra05:05
openstackgerritsebastian marcet proposed openstack-infra/openstackid master: Fixed initial DB migrations/seeds  https://review.openstack.org/57109205:08
*** bhujay has joined #openstack-infra05:10
*** eandersson has joined #openstack-infra05:10
*** snapiri- has quit IRC05:11
*** snapiri has joined #openstack-infra05:11
*** dhajare_ has joined #openstack-infra05:11
*** rakhmerov has joined #openstack-infra05:16
openstackgerritMerged openstack-infra/openstackid master: Fixed initial DB migrations/seeds  https://review.openstack.org/57109205:18
*** ykarel|afk is now known as ykarel05:18
*** lifeless_ has quit IRC05:19
*** lifeless has joined #openstack-infra05:19
*** katyafervent_awa has joined #openstack-infra05:20
*** dteselkin has joined #openstack-infra05:21
*** asilenkov has joined #openstack-infra05:24
*** igormarnat has joined #openstack-infra05:25
*** kien-ha has quit IRC05:27
*** felipemonteiro has joined #openstack-infra05:27
*** iranzo has joined #openstack-infra05:41
*** bhujay has quit IRC05:41
*** jesslampe has quit IRC05:42
*** dhajare_ has quit IRC05:42
*** eandersson has quit IRC05:48
*** eandersson has joined #openstack-infra05:49
*** felipemonteiro has quit IRC05:50
*** felipemonteiro has joined #openstack-infra05:53
*** bhujay has joined #openstack-infra05:54
*** dhajare_ has joined #openstack-infra05:55
*** igormarnat has quit IRC05:59
*** asilenkov has quit IRC05:59
*** katyafervent_awa has quit IRC05:59
*** hashar has joined #openstack-infra05:59
*** rakhmerov has quit IRC05:59
*** dteselkin has quit IRC06:00
*** cshastri has joined #openstack-infra06:02
openstackgerritMerged openstack-infra/yaml2ical master: Fix output used by http://eavesdrop.openstack.org  https://review.openstack.org/56859606:03
*** dhajare_ has quit IRC06:06
*** e0ne has quit IRC06:07
*** armaan has quit IRC06:09
*** armaan has joined #openstack-infra06:09
*** jesslampe has joined #openstack-infra06:13
*** jesslampe has quit IRC06:15
*** katyafervent_awa has joined #openstack-infra06:18
*** rakhmerov has joined #openstack-infra06:19
*** dteselkin has joined #openstack-infra06:20
*** markvoelker has joined #openstack-infra06:20
*** alexchadin has joined #openstack-infra06:20
*** asilenkov has joined #openstack-infra06:21
*** e0ne has joined #openstack-infra06:22
*** igormarnat has joined #openstack-infra06:22
*** lpetrut has joined #openstack-infra06:22
*** lifeless has quit IRC06:23
*** e0ne has quit IRC06:25
*** lifeless has joined #openstack-infra06:25
*** jtomasek has joined #openstack-infra06:25
*** rajinir has quit IRC06:27
*** xinliang3 has quit IRC06:27
*** xinliang3 has joined #openstack-infra06:28
*** jtomasek has quit IRC06:28
*** pcaruana has joined #openstack-infra06:33
*** dpawlik has joined #openstack-infra06:41
*** sree has quit IRC06:44
*** dpawlik has quit IRC06:45
*** annp has joined #openstack-infra06:52
*** sree has joined #openstack-infra06:52
*** markvoelker has quit IRC06:54
*** apple01 has joined #openstack-infra06:54
*** cshastri has quit IRC06:57
*** ssbarnea_ has joined #openstack-infra06:57
*** tosky has joined #openstack-infra07:06
*** gongysh has joined #openstack-infra07:06
*** ccamacho has joined #openstack-infra07:11
*** jaosorior has joined #openstack-infra07:15
*** apple01 has quit IRC07:17
*** sshnaidm is now known as sshnaidm_pto07:20
*** adriant has quit IRC07:20
*** tesseract has joined #openstack-infra07:20
*** adriant has joined #openstack-infra07:20
*** apetrich has quit IRC07:21
*** cshastri has joined #openstack-infra07:22
*** apetrich has joined #openstack-infra07:23
*** florianf has joined #openstack-infra07:23
*** jtomasek has joined #openstack-infra07:25
*** rcernin has quit IRC07:27
*** amoralej|off is now known as amoralej07:27
*** alexchadin has quit IRC07:28
*** ociuhandu has joined #openstack-infra07:29
*** jtomasek has quit IRC07:31
*** alexchadin has joined #openstack-infra07:34
*** dpawlik has joined #openstack-infra07:34
*** ykarel is now known as ykarel|lunch07:36
*** aojea has joined #openstack-infra07:37
*** bhujay has quit IRC07:37
*** dpawlik has quit IRC07:39
*** gongysh has quit IRC07:41
*** dpawlik has joined #openstack-infra07:41
*** markvoelker has joined #openstack-infra07:45
*** dpawlik has quit IRC07:45
*** dpawlik_ has joined #openstack-infra07:45
*** jpena|off is now known as jpena07:45
*** ociuhandu has quit IRC07:45
*** apple01 has joined #openstack-infra07:46
*** ykarel|lunch has quit IRC07:47
*** armaan has quit IRC07:48
*** armaan has joined #openstack-infra07:49
*** markvoelker has quit IRC07:49
*** apple01 has quit IRC07:51
*** zhangfei has quit IRC07:53
*** jpich has joined #openstack-infra07:53
*** gongysh has joined #openstack-infra07:53
*** bhujay has joined #openstack-infra07:59
*** armaan has quit IRC08:04
*** armaan has joined #openstack-infra08:04
*** zhangfei has joined #openstack-infra08:05
*** hrubi has joined #openstack-infra08:07
*** hrubi_ has quit IRC08:07
*** bhujay has quit IRC08:12
*** felipemonteiro has quit IRC08:15
*** rossella_s has quit IRC08:15
*** rossella_s has joined #openstack-infra08:16
*** rossella_s has quit IRC08:22
*** xarses_ has joined #openstack-infra08:22
*** vivsoni has quit IRC08:25
*** xarses has quit IRC08:25
*** rossella_s has joined #openstack-infra08:25
hwoaranghello infra. Anyone willing to look at https://review.openstack.org/#/c/570569/ which hopefully enables opensuse leap 15 images on nodepool? thanks!08:28
*** zhangfei has quit IRC08:29
*** markvoelker has joined #openstack-infra08:38
*** rossella_s has quit IRC08:39
zigo_ianw: The fix of yesterday really fixed the repo, that worked. However, I still have an issue with puppet-openstack that believes it can't enable redis-server service, when it works on my local laptop. Could you get a VM stopped again, so I can investigate?08:40
*** gfidente has joined #openstack-infra08:41
*** gfidente has joined #openstack-infra08:41
*** zhangfei has joined #openstack-infra08:41
*** rossella_s has joined #openstack-infra08:42
*** lifeless_ has joined #openstack-infra08:44
*** lifeless has quit IRC08:44
*** xarses has joined #openstack-infra08:45
*** rossella_s has quit IRC08:47
*** xarses_ has quit IRC08:48
*** ykarel|lunch has joined #openstack-infra08:48
*** vivsoni has joined #openstack-infra08:50
*** alexchadin has quit IRC08:51
*** rossella_s has joined #openstack-infra08:52
*** dingyichen has quit IRC08:54
*** dchen has quit IRC08:55
*** apple01 has joined #openstack-infra08:56
*** bhujay has joined #openstack-infra08:56
*** ykarel|lunch is now known as ykarel08:57
*** alexchadin has joined #openstack-infra08:58
*** apple01 has quit IRC09:01
*** salv-orlando has joined #openstack-infra09:02
*** gongysh has quit IRC09:04
*** armaan has quit IRC09:05
*** armaan has joined #openstack-infra09:05
*** ykarel is now known as ykarel|away09:06
*** dpawlik_ has quit IRC09:09
*** dpawlik has joined #openstack-infra09:09
*** armaan has quit IRC09:09
*** alexchadin has quit IRC09:11
*** ykarel|away has quit IRC09:12
*** markvoelker has quit IRC09:12
*** armaan has joined #openstack-infra09:13
*** pcichy has joined #openstack-infra09:13
*** dpawlik has quit IRC09:13
*** dpawlik has joined #openstack-infra09:18
*** gcb has joined #openstack-infra09:19
*** yamamoto has quit IRC09:21
ianwzigo_: ahh, let me see09:22
*** hashar has quit IRC09:23
*** hashar has joined #openstack-infra09:23
zigo_ianw: I probably have found the issue.09:23
zigo_ianw: What I'm experiencing is maybe due to https://bugs.debian.org/85053409:24
openstackDebian bug 850534 in redis-server "redis-server.service warns Unknown lvalue 'RunTimeDirectory' in section 'Service'" [Normal,Fixed]09:24
*** jtomasek has joined #openstack-infra09:24
zigo_ie: RunTimeDirectory -> RuntimeDirectory09:24
ianwzigo_: do you need arm64 or just a debian node?09:25
*** jtomasek has quit IRC09:25
zigo_ianw: Hang on until I do one last test with a modified redis-server package first.09:25
zigo_No need to bother just yet, I'll ping you (or someone else) if I can't fix it with that modified package.09:25
ianwalright, lmn09:26
zigo_Thanks. :)09:26
*** dhajare_ has joined #openstack-infra09:26
*** zoli is now known as zoli|lunch09:31
*** udesale_ has joined #openstack-infra09:31
*** udesale_ has quit IRC09:32
*** udesale_ has joined #openstack-infra09:32
*** udesale_ has quit IRC09:32
*** udesale_ has joined #openstack-infra09:33
*** udesale has quit IRC09:33
*** dtantsur|afk is now known as dtantsur09:38
*** hashar is now known as hasharAway09:50
*** yamamoto has joined #openstack-infra09:50
stephenfincorvus: I think this is ready to go now, whenever you're about https://review.openstack.org/#/c/559348/609:50
zigo_ianw: My patch to the redis-server.service didn't help, so I do need to debug on the VM.09:55
zigo_ianw: The patch is this one: https://review.openstack.org/#/c/564328 and the job is puppet-openstack-integration-4-scenario001-tempest-debian-stable (for the puppet-nova repo)09:56
*** zigo_ is now known as zigo09:56
*** sree has quit IRC09:57
*** rossella_s has quit IRC10:00
*** rossella_s has joined #openstack-infra10:00
*** annp has quit IRC10:01
*** udesale__ has joined #openstack-infra10:02
*** udesale__ has quit IRC10:02
ianw| openstack | git.openstack.org/openstack/puppet-nova | puppet-openstack-integration-4-scenario001-tempest-debian-stable |     .*     |   1   | ianw: 564328 |10:03
ianwok, we should hold one10:03
*** udesale has joined #openstack-infra10:03
zigoianw: Did you do a recheck? :)10:05
*** udesale_ has quit IRC10:05
zigoAh yeah, thanks.10:05
zigorebased ...10:05
ianwto get it to cancel inflight10:05
*** rossella_s has quit IRC10:06
*** apple01 has joined #openstack-infra10:06
*** namnh has quit IRC10:08
*** rossella_s has joined #openstack-infra10:09
*** jaosorior has quit IRC10:14
*** apple01 has quit IRC10:14
*** psachin has joined #openstack-infra10:20
*** sambetts|afk is now known as sambetts10:23
*** salv-orlando has quit IRC10:26
dtantsurhi folks! is it possible to make https://docs.openstack.org/project-install-guide/baremetal/draft/ a redirect to https://docs.openstack.org/ironic/latest/install/ please?10:30
dtantsurthe former is no longer updated, but people still find it somehow..10:30
*** jaosorior has joined #openstack-infra10:31
*** lifeless_ has quit IRC10:32
*** markvoelker has joined #openstack-infra10:35
*** rossella_s has quit IRC10:36
*** rossella_s has joined #openstack-infra10:38
*** gyankum has quit IRC10:41
*** ssbarnea_ has quit IRC10:42
zigoianw: The job should be stopped and VM ready. My ssh key: http://thomas.goirand.fr/ssh-pubkey.txt10:48
*** nicolasbock has joined #openstack-infra10:50
ianwzigo: root@104.239.240.3710:52
zigoianw: Cheers. I'll do my investigations after lunch.10:53
*** zoli|lunch is now known as zoli11:00
*** adreznec has quit IRC11:00
*** lifeless has joined #openstack-infra11:01
*** adreznec has joined #openstack-infra11:01
*** apple01 has joined #openstack-infra11:04
*** markvoelker has quit IRC11:06
*** apple01 has quit IRC11:08
*** armaan has quit IRC11:16
*** armaan has joined #openstack-infra11:17
*** ociuhandu has joined #openstack-infra11:19
*** ssbarnea_ has joined #openstack-infra11:19
*** apple01 has joined #openstack-infra11:20
*** dave-mccowan has joined #openstack-infra11:21
*** zhangfei has quit IRC11:23
*** derekh has joined #openstack-infra11:25
*** dave-mccowan has quit IRC11:26
*** dpawlik has quit IRC11:27
*** dave-mcc_ has joined #openstack-infra11:27
*** ykarel has joined #openstack-infra11:28
*** udesale_ has joined #openstack-infra11:28
*** yamamoto has quit IRC11:29
*** udesale has quit IRC11:31
*** ldnunes has joined #openstack-infra11:32
*** udesale_ has quit IRC11:33
*** armaan has quit IRC11:33
*** armaan_ has joined #openstack-infra11:33
*** jpena is now known as jpena|lunch11:33
*** tpsilva has joined #openstack-infra11:37
*** lifeless_ has joined #openstack-infra11:37
*** ykarel has quit IRC11:37
*** lifeless has quit IRC11:38
*** ociuhandu has quit IRC11:38
*** amoralej is now known as amoralej|lunch11:38
hwoarangianw: is there something else that needs to be done for https://review.openstack.org/#/c/566924/ ? perhaps manual testing before W+1?11:40
*** dpawlik has joined #openstack-infra11:41
*** apple01 has quit IRC11:42
ianwhwoarang: hmm, i don't think so, maybe jhesketh was thinking to run it manually?11:43
*** alexchadin has joined #openstack-infra11:44
ianwis it going to double disk usage?11:44
hwoarangyep... do you want to omit 42.3 since it should go away soon?11:44
jheskethI was jet lagged and not going to be around much longer when I +2'd that so I couldn't babysit it (not that it necessarily needs it)11:45
jheskethI'm not around much longer tonight either, so probably not the best person to +w11:45
hwoarangor we can get it as is and will clean it up when we remove 42.3 images11:46
ianwianw@mirror-update01:/afs/openstack.org/mirror/opensuse$ fs quota11:46
ianw86% of quota used.11:46
ianwso it doesn't have room for essentially doubling itself11:46
ianwhwoarang: i also have to head off (you got the two .au people to review :) but i can sync it in the morning, and we can remove 42.3 asap?11:48
hwoarangianw: current code is broken becuase it syncs all distros instead of the ones we want11:48
hwoarangso disk usage will actually go down11:49
*** apple01 has joined #openstack-infra11:49
*** armaan_ has quit IRC11:49
*** rosmaita has joined #openstack-infra11:49
hwoarangno worries. feel free to merge it tomorrow and i will be around to keep an eye on it during EU working hours11:49
ianwhwoarang: alright, yeah, so babysitting still seems like a good idea :)  i will take it as a todo item to do it in my morning11:50
hwoaranggreat thanks!11:50
*** neiloy has joined #openstack-infra11:53
*** gyankum has joined #openstack-infra11:54
*** rlandy has joined #openstack-infra11:56
*** mikal has joined #openstack-infra11:57
*** mikal_ has quit IRC11:59
*** rlandy is now known as rlandy|rover12:03
*** yamamoto has joined #openstack-infra12:07
*** kgiusti has joined #openstack-infra12:08
*** panda|off is now known as panda12:10
*** Tahvok has quit IRC12:15
*** sree_ has joined #openstack-infra12:19
*** sree_ is now known as Guest7572812:20
*** Guest75728 has quit IRC12:24
*** jpena|lunch is now known as jpena12:28
*** udesale has joined #openstack-infra12:29
*** Odd_Bloke has joined #openstack-infra12:31
*** trown|outtypewww is now known as trown12:31
*** armaan has joined #openstack-infra12:32
*** alexchadin has quit IRC12:33
*** uberjay has quit IRC12:34
*** uberjay has joined #openstack-infra12:35
*** e0ne has joined #openstack-infra12:36
openstackgerritBrian Rosmaita proposed openstack-infra/zuul-jobs master: Handle -/_ ambiguity in package names  https://review.openstack.org/57100512:36
*** Odd_Bloke has left #openstack-infra12:38
*** ykarel has joined #openstack-infra12:38
*** lifeless_ has quit IRC12:41
*** amoralej|lunch is now known as amoralej12:41
*** salv-orlando has joined #openstack-infra12:43
*** salv-orlando has quit IRC12:43
*** salv-orlando has joined #openstack-infra12:44
*** salv-orlando has quit IRC12:44
*** armaan has quit IRC12:44
*** salv-orlando has joined #openstack-infra12:44
*** salv-orlando has quit IRC12:44
*** salv-orlando has joined #openstack-infra12:44
*** apple01 has quit IRC12:44
*** salv-orlando has quit IRC12:45
*** salv-orlando has joined #openstack-infra12:45
*** salv-orlando has quit IRC12:45
*** salv-orlando has joined #openstack-infra12:46
*** eharney has quit IRC12:46
*** salv-orlando has quit IRC12:46
*** udesale has quit IRC12:47
*** udesale has joined #openstack-infra12:47
*** edmondsw has joined #openstack-infra12:50
fungidtantsur: i think you want to propose it to https://git.openstack.org/cgit/openstack/openstack-manuals/tree/www/.htaccess but better to ask the docs team in #openstack-doc12:52
*** jamesmcarthur has joined #openstack-infra12:53
*** markvoelker has joined #openstack-infra12:54
dtantsurk thanks fungi12:54
*** vickyc has joined #openstack-infra12:57
*** mriedem has joined #openstack-infra12:59
openstackgerritAlex Schultz proposed openstack-infra/project-config master: Add python jobs to tripleo-image-elements  https://review.openstack.org/57119812:59
*** yamamoto has quit IRC13:04
*** jcoufal has joined #openstack-infra13:05
*** caphrim007 has quit IRC13:06
*** benj_ has quit IRC13:08
*** armaan has joined #openstack-infra13:08
*** benj_ has joined #openstack-infra13:09
*** ykarel_ has joined #openstack-infra13:13
*** ykarel has quit IRC13:16
*** rossella_s has quit IRC13:16
*** wolverineav has joined #openstack-infra13:18
*** rossella_s has joined #openstack-infra13:20
*** ykarel_ is now known as ykarel13:21
*** cshastri has quit IRC13:23
mordredclarkb: sorry about the issue with the inventory script - the main issue is that 'import openstack' breaks in a script called 'openstack'13:24
mordredthis is a good reminder that we should switch from the inventory script to the inventory plugin though - I'll put that into the ansible spec13:24
*** markvoelker has quit IRC13:24
*** rossella_s has quit IRC13:25
*** esberglu has joined #openstack-infra13:26
*** rossella_s has joined #openstack-infra13:27
*** links has quit IRC13:29
*** links has joined #openstack-infra13:30
*** eharney has joined #openstack-infra13:32
*** yamamoto has joined #openstack-infra13:35
*** cshastri has joined #openstack-infra13:35
*** yamamoto has quit IRC13:39
*** nicolasbock has quit IRC13:40
mordredfungi, stephenfin: https://review.openstack.org/#/c/559348/ should be ready to land now - are we comfortable doing that?13:40
stephenfinmordred: Yeah, I think it's good. Have tested with various DNM patches and submitted the patch to the zuul ML more than 2 weeks ago13:41
mordredkk. I'm going to pull the trigger. let's keep our eyes out for unexpected carnage13:42
*** rossella_s has quit IRC13:43
*** rossella_s has joined #openstack-infra13:44
*** jesslampe has joined #openstack-infra13:44
*** links has quit IRC13:44
*** cshastri has quit IRC13:45
fungiseems like it's time, yes13:47
*** rossella_s has quit IRC13:49
*** rossella_s has joined #openstack-infra13:51
*** nicolasbock has joined #openstack-infra13:53
corvusAn unexpected error occurred: "https://registry.yarnpkg.com/@babel/traverse/-/traverse-7.0.0-beta.42.tgz: Request failed \"503 Service Unavailable\"".13:54
corvusit's not a good week for js package hosting13:54
*** VW has joined #openstack-infra13:56
*** sree has joined #openstack-infra13:59
*** lpetrut has quit IRC14:00
*** Tahvok has joined #openstack-infra14:00
*** bhujay has quit IRC14:08
*** EmilienM_PTO is now known as EmilienM14:08
*** Guest62139 has quit IRC14:13
*** janki has quit IRC14:15
*** armaan has quit IRC14:18
*** signed8bit has joined #openstack-infra14:18
*** r-daneel has joined #openstack-infra14:18
*** armaan has joined #openstack-infra14:19
*** flaper87 has joined #openstack-infra14:20
*** rajinir has joined #openstack-infra14:25
*** felipemonteiro has joined #openstack-infra14:25
*** kien-ha has joined #openstack-infra14:29
corvusthat's 2 failures.  http://status.yarnpkg.com/  is green.14:30
corvusaiui, the npm status page was also "green" during the recent bad times14:30
corvuswe should totally have an infra status page that's always green14:31
*** rpioso|afk is now known as rpioso14:33
*** hongbin has joined #openstack-infra14:33
*** kjackal has joined #openstack-infra14:33
*** dklyle has joined #openstack-infra14:33
mordredcorvus: we can make one of those :)14:34
*** pblaho has quit IRC14:35
mordredcorvus: incidentally though - the zuul jobs are not currently using the infra reverse-proxy mirror14:36
mordredit's on my list of things to work on today14:36
*** r-daneel_ has joined #openstack-infra14:36
*** r-daneel has quit IRC14:37
*** r-daneel_ is now known as r-daneel14:37
mordred(for npm things)14:38
*** felipemonteiro has quit IRC14:42
corvusmordred: and i guess yarn things too?  (i'm assuming yarn needs its own configuration separate from anything you'd do for npm?)14:42
*** jamesmcarthur has quit IRC14:42
*** benj_ has quit IRC14:44
*** sree has quit IRC14:45
*** sree has joined #openstack-infra14:45
*** eernst has joined #openstack-infra14:46
*** benj_ has joined #openstack-infra14:46
clarkbmordred: thats fine however the old script could've remained in place and continued to use shade14:48
clarkbmordred: allowing users to migrate from one to the other more gracefully14:48
clarkbbut I guess that still has import problems14:48
*** armaan has quit IRC14:49
clarkbmordred: as for switching to the _inventory.py script should it just work and I should test it and switch pupept to that or should I move ahead with the pin instead?14:49
*** sree has quit IRC14:50
fungiclarkb: where did the survey.o.o certs/key end up? didn't see them at puppetmaster:~root/certs/14:51
clarkbmordred: basically my bigger concern at this point is less the rename of the script and more that the script internals changed too. A legacy_openstack.py inventory script would be helpful14:52
clarkbfungi: oh they are still in my inbox because summit. I will copy them now14:52
fungino worries, just trying to get everything into hiera in preparation for a production build14:52
*** ykarel is now known as ykarel|away14:53
fungiif you want to just stuff them into /etc/puppet/hieradata/production/group/survey.yaml that works too14:53
mordredclarkb: yes - it should just work - no internals should have changed at all14:54
mordredclarkb: the shade code in openstacksdk is a direct copy of the code in shade14:55
*** dave-mcc_ is now known as dave-mccowan14:55
clarkbmordred: ok that commit message implied things were rewritten but I didn't have time to look at the diff yet14:55
clarkbfungi: I put them in /root/certs14:56
mordredclarkb: just to import from openstack instead of shade ... we could add an openstack.py script back that imports from shade - but we'd already been getting bug reports from people about the fact that it didn't work with newer shade because of the use of openstacksdk inside of os-client-config14:57
fungithanks, clarkb!14:57
clarkbmordred: ok I'm gonna run a --list --cloud somecloud test and if that works I'll convert the puppetry over to use _inventory14:58
*** jgrassler has quit IRC14:58
*** jamesmcarthur has joined #openstack-infra14:58
clarkbdoes not work :/14:59
clarkbkeystoneauth1.exceptions.auth_plugins.MissingRequiredOptions: Auth plugin requires parameters which were not given: auth_url14:59
clarkbI did --list --cloud openstackci-limestone as a regular user. auth_url is set for openstackci-limestone in /etc/openstack/clouds.yaml15:00
*** armaan has joined #openstack-infra15:01
mordredclarkb: ok. lemme look real quick - is this on puppetmaster?15:01
clarkbmordred: yes15:01
clarkbclarkb@puppetmaster:~$ /opt/ansible/contrib/inventory/openstack_inventory.py --list --cloud openstackci-limestone15:01
mordredis that user in the admin group?15:01
clarkbyes15:01
*** ykarel|away is now known as ykarel15:02
openstackgerritMerged openstack-infra/zuul-jobs master: Default warning-is-error to True for non-legacy Sphinx projects  https://review.openstack.org/55934815:02
clarkbI can read /etc/openstack/clouds.yaml as this user15:02
mordredyes. I can too - and I can confirm the same error15:02
*** gyee has joined #openstack-infra15:05
*** zhangfei has joined #openstack-infra15:06
*** apple01 has joined #openstack-infra15:07
mordredclarkb: python -m openstack.config.loader is printing out config correctly including openstackci-limestone - I'm digging further15:08
*** markvoelker has joined #openstack-infra15:08
*** lpetrut has joined #openstack-infra15:09
*** bhujay has joined #openstack-infra15:12
*** bhujay has quit IRC15:16
*** r-daneel has quit IRC15:16
*** bhujay has joined #openstack-infra15:17
fungiheading out to grab some food, back in a bit15:19
*** dpawlik has quit IRC15:19
*** rfolco_ has joined #openstack-infra15:21
mordredclarkb: ok. it's a bug. https://github.com/ansible/ansible/pull/40892 https://github.com/ansible/ansible/pull/4089415:21
clarkbmordred: weird, is it maybe not looking in the auth: dict?15:21
clarkboh cool you've figured it out15:21
mordredShrews: ^^ got a sec to fasttrack land a couple of tiny patches?15:21
Shrewsya15:22
Shrewsmordred: let me know when shippable gives the ok15:23
*** dhinesh has joined #openstack-infra15:23
clarkbah so it wasn't looking in the cloud that was specified. Was the behavior in that case to expect a single cloud in the clouds.yaml file?15:23
clarkbI was still in keystoneauth wondering why I couldn't find where auth_url is defined as a required opt and that is because it is auth-url when defined as an opt15:23
*** rfolco has quit IRC15:23
mordred:)15:24
mordredclarkb: yah - which is not the intent...15:24
clarkbI wonder if I can teach git grep to treat _ and - the same when I use -i15:24
*** r-daneel has joined #openstack-infra15:25
mordredclarkb: that would be the best feature to git grep ever15:25
*** Guest48153 is now known as melwitt15:25
*** zhangfei has quit IRC15:25
clarkbmordred: Shrews ansible might want to drop fedora 24 and 25 testing at this point, those releases are no longer supported aiui15:27
clarkboh those are the only fedora tests, probably want to add 26-28 first15:28
*** jesslampe has quit IRC15:28
*** jamesmcarthur has quit IRC15:31
*** janki has joined #openstack-infra15:33
*** pcaruana has quit IRC15:33
hrwclarkb: abandon f26 too - it is close to EOL15:33
mordredclarkb, Shrews: https://review.openstack.org/#/q/topic:ansible-openstacksdk+status:open are testing updated to make sure we're cross-testing ansible and openstacksdk15:36
mordred(and also to test stable-2.5 of ansible against shade)15:36
*** udesale has quit IRC15:36
Shrewsmordred: do we have any openstack-inventory tests?15:37
Shrewsobviously nothing that could have caught the bug above15:37
mordredShrews: I believe we have no openstack-inventory tests15:39
mordredShrews: or, rather, we have no tests of that script - we do have tests of the underlying inventory code itself15:40
mordredbut the ansible inventory script wrapper stuff (which is what broke here) is missing a test15:40
*** dhinesh has quit IRC15:41
*** markvoelker has quit IRC15:41
*** udesale has joined #openstack-infra15:43
*** caphrim007 has joined #openstack-infra15:44
*** jgrassler has joined #openstack-infra15:44
*** jodh has joined #openstack-infra15:44
*** tung_doan has joined #openstack-infra15:45
*** jodh has left #openstack-infra15:46
tung_doanHi Infra team, could you guys give one more +2 to our patch: https://review.openstack.org/#/c/570363/15:46
tung_doanThanks15:46
clarkbtung_doan: is there any reason to not add that config in your project itself?15:47
clarkbI think we are going to have to start slowly pushing back on these chagnes that should be made in repo15:47
clarkbbut before I say you should do it in repo I want to make sure I'm not missing something important15:48
clarkboh monty approved15:49
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Run openstacksdk ansible tests on stable-2.6 of ansible  https://review.openstack.org/57123815:49
clarkbtung_doan: in general I think the preference would be to have that config in your project repos. We are in a transition state currently where config is still centrally managed in many cases but we should be shifting away from that over time15:50
mordredclarkb: oh - I can remove the +W if you want15:50
clarkbmordred: I think its fine if people are reviewing them15:50
*** ykarel is now known as ykarel|away15:50
*** jgrassler has quit IRC15:50
clarkbits mostly an anti pattern for us to have to review those especially as part of being pinged imo though15:50
clarkbif it is important and can be done in repo just get it done in repo :)15:50
Shrews++15:50
mordredkk. and I agree - I'd like to be as little in the business of central job management as possible15:50
*** felipemonteiro has joined #openstack-infra15:50
tung_doanclarkb: we are planning to add functional tests in repo. But for py35 and py27, you mean I can add them to my own repo?15:51
mordredShrews: one more ansible test patch: https://review.openstack.org/5712315:51
clarkbtung_doan: yes15:51
*** apple01 has quit IRC15:51
Shrewsmordred: ???15:51
Shrewsbad linky15:52
mordredShrews: I suck at copy-pasta https://review.openstack.org/57123815:52
clarkbmordred: wrong change link15:52
mordredyah15:52
Shrewsmordred: zuul'd -1, so i skipped it15:52
Shrewszuul -1'd, rather15:52
mordredoh - that one needed a depends-on - I'll circle back around on it15:52
openstackgerritMerged openstack-infra/project-config master: Add pep8, py27, and py35 test to apmec project  https://review.openstack.org/57036315:53
tung_doanclarkb: I did not know that before. I investigate that many projects just put func. tests in repo and move unit tests to infra :)15:53
clarkbtung_doan: the only repo specific requirement (and I'm sure I'll get this wrong due to other corner cases) are roles playbooks used by jobs must be in the same repo as the job definition and secrets can't be used by jobs defined in other repos15:55
*** lpetrut has quit IRC15:55
clarkbtung_doan: you can consume jobs that are defined in other repos just fine15:55
Shrewsmordred: -1 from zuul on https://review.openstack.org/570682 also too as well15:56
*** mriedem has quit IRC15:56
*** mriedem has joined #openstack-infra15:56
* Shrews runs out of adjectives15:56
*** jamesmcarthur has joined #openstack-infra15:56
Shrewsor adverbs15:57
Shrewsor whatever15:57
tung_doanclarkb: yep. got it15:57
*** dhinesh has joined #openstack-infra15:58
*** hasharAway is now known as hashar16:01
*** felipemonteiro has quit IRC16:01
*** felipemonteiro has joined #openstack-infra16:01
*** jamesmcarthur has quit IRC16:01
*** ykarel|away has quit IRC16:02
*** kiennt26 has joined #openstack-infra16:02
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Run openstacksdk ansible tests on stable-2.6 of ansible  https://review.openstack.org/57123816:02
clarkbmordred: Shrews should I expect the ansible bug to be fixed quickly enough to switch over to that inventory script or should we consider https://review.openstack.org/#/c/571045/ to get puppet running again?16:03
mordredShrews: ok. that ^^ and https://review.openstack.org/571240 should be happier now16:03
clarkbthe background here is I'm trying to update the default apache vhost on our git mirrors so that clietns without SNI support hit git.openstack.org16:03
mordredclarkb: either way - we'll get the ansible patch landed as soon as shippale returns green16:04
clarkbthey are the existing clients of the platform and zuul and airship and others can make sure their clients can SNI16:04
*** zoli is now known as zoli|gone16:04
*** zoli|gone is now known as zoli16:04
clarkbmordred: it hasn't started any tests yet :/16:04
clarkbI'm not sure what shippables throughoput is typically like16:04
mordredit might be a sec16:05
clarkbdoesn't look like they give me that global state info as an unauthenticated user16:05
mordredclarkb: so maybe let's just land fungi's patch for now, and we can revert it as soon as things are landed16:05
mordredoh - or your patch that is16:05
clarkbI'm happy to do that and separately get the inventory stuff updated16:05
mordredyah16:05
mordredbecause honestly I want to get us updated to using inventory instead of inventory script anyway16:06
*** camunoz has joined #openstack-infra16:06
clarkbansible has non contrib openstack inventory support? or you mean explicit file based inventory?16:06
*** dhinesh_ has joined #openstack-infra16:07
*** tung_doan has quit IRC16:07
mordredno, I mean there is now non-contrib openstack inventory support - and the new system also supports stacking inventory sources16:07
*** electrofelix has quit IRC16:08
mordredhttps://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/inventory/openstack.py16:08
mordredclarkb: (but I'm going to include notes aout that in the next version of the config management spec, since it should ultimately allow us to have a less janky inventory and host/group var experience)16:10
*** dhinesh__ has joined #openstack-infra16:10
*** dhinesh has quit IRC16:10
*** dhinesh__ has quit IRC16:11
*** dhinesh_ has quit IRC16:12
fungiback, catching up on which "fungi's patch" this is16:18
stephenfinfungi: quick question - do infra consume zuul master or zuul packages?16:19
*** panda is now known as panda|off16:19
mordredfungi: it's the one that's not fungi's patch but that mordred misidentified as such16:19
fungiclarkb: git grep will treat . the same as - and _ if that helps! ;)16:19
clarkbfungi: was my patch to pin ansible. We are going to do that short term to get puppet working again then switch to the new inventory script when mordred's bug fixes get in upstream16:19
mordredstephenfin: master-ish16:19
mordredstephenfin: we run from master, but don't restart the scheduler necessarily on every patch16:19
stephenfinmordred: So we'll see https://review.openstack.org/#/c/559348/ pop up before...3.0.3 (or whatever the next release) is cut?16:20
*** kjackal has quit IRC16:20
mordredstephenfin: although we do make sure to restart it before cutting release tags16:20
mordredstephenfin: ah - zuul-jobs is from master16:20
mordredstephenfin: zuul will automatically update its config the instant that lnads16:20
mordredso - it should be live now16:20
stephenfinmordred: Ah, different things of course. Cool cool16:20
*** dhajare_ has quit IRC16:20
*** kiennt26 has quit IRC16:21
stephenfindhellmann suggested I be a good citizen and tell people that was happening16:21
mordred++16:21
dhellmannbecause the answer to people complaining about the volume of email on openstack-dev is to fix the signal/noise ratio, and that means sending more email!16:22
fungiclarkb: mordred: oh, yes, the pin ansible patch. i think i +2'd already but will double-check16:22
clarkbfungi: you did16:22
fungiperfect, one less thing for me to check16:22
mordreddhellmann: https://xkcd.com/927/16:22
dhellmannmordred : s/standard/email thread/16:23
clarkbthat moment you discover you have +2 on openstacksdk. /me promises to only use it for adding these tests :P16:23
*** pcaruana has joined #openstack-infra16:23
mordredclarkb: you are more than welcome to use it any time you want - you're not there by accident ;)16:24
*** lpetrut has joined #openstack-infra16:24
*** trown is now known as trown|lunch16:26
*** udesale has quit IRC16:26
openstackgerritMerged openstack-infra/system-config master: Pin /opt/ansible as source of openstack inventory  https://review.openstack.org/57104516:28
* dtantsur knows one more person to ping for openstacksdk reviews, w00t :D16:28
clarkbmordred: I don't undertand what they one chagne keeps reporting it depends on a change that failed to merge16:31
clarkbits git parent ran tests fine and its depends on is running tests too16:32
fungidtantsur: the infra-core group are transitively core on openstacksdk by way of having been core on shade from back when it was still an infra project16:32
fungii doubt too many of us are exercising that, however16:32
clarkbmordred: (/Stage[main]/Openstack_project::Puppetmaster/Vcsrepo[/opt/ansible]/ensure) Please, commit your changes or stash them before you can switch branches.16:33
clarkbmordred: I'm assuming the loca changes on the puppet master are you?16:33
clarkbmordred: can we stash the delta or delete the changes?16:33
mordredclarkb: fine to delete16:34
clarkbok doing that now16:34
*** kjackal has joined #openstack-infra16:34
*** jgrassler has joined #openstack-infra16:35
clarkbok next puppet run should get us back to puppeting the git servers, we can check git08 is happy then deploy the sni fix to the rest of the cluster16:35
mordredwoot16:36
clarkbthen remove the ansible pin once openstack_inventory fixes merge and are tested16:36
clarkbI'll just fetch that file from ansible devel when it is updated and retest16:36
*** caphrim007 has quit IRC16:40
*** jpena is now known as jpena|off16:41
mordred++16:42
*** marshallmargenau has joined #openstack-infra16:44
*** signed8bit is now known as signed8bit_Zzz16:44
*** salv-orlando has joined #openstack-infra16:47
*** salv-orlando has quit IRC16:47
*** salv-orlando has joined #openstack-infra16:48
*** salv-orlando has quit IRC16:48
*** salv-orlando has joined #openstack-infra16:48
*** rlandy|rover is now known as rlandy|rover|brb16:49
*** salv-orlando has quit IRC16:49
*** salv-orlando has joined #openstack-infra16:49
*** salv-orlando has quit IRC16:50
*** salv-orlando has joined #openstack-infra16:50
*** armaan has quit IRC16:50
*** salv-orlando has quit IRC16:50
*** salv-orlando has joined #openstack-infra16:51
*** salv-orlando has quit IRC16:51
*** jpena|off is now known as jpena16:52
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Allow templates to be optional for zuul-migrate  https://review.openstack.org/57097916:54
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Allow for projects only names in zuul-migrate  https://review.openstack.org/57098016:54
clarkbmordred: any idea why ansible doesn't use the master branch?16:55
clarkbreviewing these new jobs it occurred to me that tool conventions are only going to be more important as we do more of this cross community stuff16:56
clarkband devel feels like a very unconventional git pattern16:56
*** sambetts is now known as sambetts|afk16:59
clarkbmordred: also http://logs.openstack.org/40/571240/1/check/openstacksdk-ansible-devel-functional-devstack/1a600cb/job-output.txt.gz#_2018-05-30_16_29_57_495957 do you want to merge the new jobs as is or fix that first?16:59
*** rpioso is now known as rpioso|eat16:59
*** dtantsur is now known as dtantsur|afk17:00
*** hashar is now known as hasharAway17:00
mordredclarkb: hysterical raisins, to the best of my knowledge17:01
mordredclarkb: I can tell you it always messes with my muscle memory17:01
*** derekh has quit IRC17:01
*** e0ne has quit IRC17:02
*** e0ne has joined #openstack-infra17:02
fungiclarkb: i wonder if they were using svn before17:02
clarkbI'm going to date myself. I jumped straight from rcs to bzr17:03
clarkbdid svn use devel?17:03
mordredno - it was a conscious choice on the part of early devs17:03
fungibasically it learned that pattern from rcs/cvs, sure. the predominant paradigm was to do development work on branches and commit stable code to the trunk17:03
mordredsomeone told me why thatperson thought it was a good idea a while back but I've long since forgotten17:04
fungiwhere git's "master" branch is the rough equivalent of the svn "trunk"17:04
clarkbit used file level checkouts though right?17:04
clarkbwhich are their own set of complications17:04
* clarkb glares at borland starteam17:04
clarkbok git08 got puppetted17:04
clarkband puppet appears to have moved the 50- prefix file to the 25- prefix17:05
clarkbtesting sni now17:05
fungiyeah, i'm doing my best not to bump th eneurons containing pre-git vcs memories17:05
clarkb`openssl s_client -connect git08.openstack.org:4443` gives me git.o.o not git.airshipit.org now.17:05
fungiyeah, that's looking right to me too17:05
clarkbI'm going to add git08 back to haproxy then remove 01-07 from the ansible emergency file17:05
fungisounds great17:06
fungiclarkb: /etc/httpd/conf.d/ doesn't seem to have ended up with two configs for git.o.o, so looking good17:07
*** amoralej is now known as amoralej|off17:07
fungithat puppet module is smarter than i'd expected17:07
clarkbyup17:07
clarkbI've gone ahead and put 08 back into haproxy and removed 01-07 from the emergency file. Next puppet run should update all the backends then I can circle back around on updating the inventory file source17:08
clarkbefried: in about half an hour I think you can retest if ssl certs are happier17:08
clarkbefried: I'll confirm with you when I think it is "fixed" on our end and you can double check17:08
*** tesseract has quit IRC17:10
clarkbIs it bad that I'm happy apache makes this intuitive17:10
clarkbmaybe happy is wrong word, surprised is probably better17:11
*** yamahata has joined #openstack-infra17:11
*** signed8bit_Zzz is now known as signed8bit17:12
fungiit is a bit shocking that it works this conveniently, sure17:14
*** felipemonteiro has quit IRC17:15
clarkbmordred: what is the get_one behavior if you pass nothing? is it lookin for a cloud with name key None?17:15
clarkbShrews: https://github.com/ansible/ansible/pull/40892 passed tests17:16
*** felipemonteiro has joined #openstack-infra17:16
*** jpena is now known as jpena|off17:18
*** felipemonteiro has quit IRC17:20
*** gyankum has quit IRC17:23
*** felipemonteiro has joined #openstack-infra17:25
*** ssbarnea_ has quit IRC17:27
Shrewsclarkb: yeah, the other didn't though (though it seems unrelated). i'd like to land them both at the same time17:27
Shrewsclarkb: gotta step out for a bit, but will check them again when i return17:28
clarkbShrews: thanks17:29
*** markvoelker has joined #openstack-infra17:30
mordredclarkb: yes. if you only have one cloud defined, it'll use that one, but otherwise it's usually not particularly useful thing to do17:30
openstackgerritClark Boylan proposed openstack-infra/system-config master: Switch to using new ansible openstack inventory script  https://review.openstack.org/57126317:31
clarkbI WIP'd that I could've actually dependson'd it now that I think about it17:31
clarkboh well17:31
*** jpich has quit IRC17:31
clarkbmordred: but you might want to go ahead and review that ^ lines up with what you expect to be the future state17:32
*** aojea has quit IRC17:33
*** signed8bit is now known as signed8bit_Zzz17:35
*** dhinesh has joined #openstack-infra17:38
*** felipemonteiro has quit IRC17:38
*** trown|lunch is now known as trown17:39
clarkb#status log git08 added back to git.o.o haproxy and all git backends updated to make git.openstack.org vhost their default vhost. This means that https clients that don't speak SNI will get the cert for git.o.o (and talk to git.o.o vhost) by default.17:40
clarkbefried: ^ I think you can check if things are happier for you now17:40
clarkbhrm am I not identified17:41
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul master: Add missing pathlib dependency  https://review.openstack.org/57053917:41
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul master: Add missing pathlib dependency  https://review.openstack.org/57053917:42
clarkb#status log git08 added back to git.o.o haproxy and all git backends updated to make git.openstack.org vhost their default vhost. This means that https clients that don't speak SNI will get the cert for git.o.o (and talk to git.o.o vhost) by default.17:42
openstackstatusclarkb: finished logging17:42
fungiclarkb: yeah, i think freenode wound up inadvertently clearing nickserv sessions over the weekend when doing services upgrades17:45
*** signed8bit_Zzz is now known as signed8bit17:45
fungii have autokick set on my nick so spotted it right away17:45
*** ssbarnea_ has joined #openstack-infra17:45
*** caphrim007 has joined #openstack-infra17:45
clarkbnext fixing storyboard17:46
clarkbfungi: you haven't had any great ideas re ^ have you?17:46
*** rlandy|rover|brb is now known as rlandy|rover17:46
openstackgerritMerged openstack/gertty master: Add message attribute to DisplayError  https://review.openstack.org/56968317:46
*** caphrim007 has quit IRC17:47
*** caphrim007 has joined #openstack-infra17:47
fungiclarkb: other than retooling storyboard-webclient as some sort of packaged distribution, not really17:50
corvuszxiiro: i sent an email to the odl helpdesk asking for streamEvents to be enabled for openstack-zuul, that hasn't happened yet, and i don't think i got an acknowledgement.  is that something you can check on?17:50
openstackgerritJonathan Bryce proposed openstack-infra/system-config master: Creates new mailing list for discussion of container security  https://review.openstack.org/57127317:50
fungimordred: ^ perhaps this is somewhere in your wheelhouse? trying to figure out the sanest way to deploy storyboard-webclient as a continuous delivery from git master17:51
fungithe hardest part is making sure the deployment location gets cleared of old cruft17:51
mordredfungi: well - we should be publishing tarballs of the built sourcing17:51
clarkbya we have the published tarballs17:51
fungiyeah, we're trying to deploy from those17:51
clarkbI kinda want to exec a script to do this17:52
fungii suppose we could have a puppet exec which uses rsync --delete though that seems hacky17:52
mordredthat's what we do for zuul iirc17:52
clarkbwhich can download the tarball to tmpdir, extract it there, rsync it with a delete to the dest and be done17:52
corvusi am not going to suggest afs :)17:52
fungihah17:52
fungii bet we could deploy via nntp17:52
*** sree has joined #openstack-infra17:52
mordred$zuul_web_filename <-- that's the thing to look for in puppet-zuul17:52
clarkbso rather than encode the download, delete, extract, copy in puppet we just have puppet call a script that does it17:53
*** lpetrut has quit IRC17:53
clarkbcommand     => "rm -rf ${zuul_web_src_root}/* && tar -C ${zuul_web_src_root} -xzf ./${zuul_web_filename}"17:53
clarkbthat is the trick that zuul uses17:53
clarkbthat is simple enough I'll make a patch to storyboard that does that17:53
fungihax0rz ftw!17:53
fungilet's call that precedent17:54
mordredclarkb: amusingly enough, I think the zuul_web puppet code was originally copied from storyboard :)17:54
fungiif so, that was long ago. i'm digging back through puppet-storyboard to see when that might have changed17:55
fungigone back as far as 2015 and not finding it17:56
*** sree has quit IRC17:57
openstackgerritClark Boylan proposed openstack-infra/puppet-storyboard master: Clean the webclient unpack dir safely.  https://review.openstack.org/57000417:57
*** psachin has quit IRC17:57
clarkbuses the old changeid but its a new patchset17:57
fungithx17:57
clarkbnow to double check if * gets expanded17:58
*** rpioso|eat is now known as rpioso17:58
fungiyeah, that looks sane assuming globbing works there17:58
clarkbhttps://puppet.com/docs/puppet/5.3/types/exec.html#exec-provider-shell it will work if you set the provider to shell. Unsure if it figures that out on its own17:59
clarkbI'll explicitly set it17:59
clarkbthen we can add it to the zuul resource if we need to17:59
mordredfungi: exec { 'get-webclient': and exec { 'unpack-webclient': were copied - I changed the zuul to use rsync instead of the puppet file resource because webpack for zuul produces hashed filenames so without --delete the directory would just grow infinitely17:59
*** markvoelker has quit IRC18:00
fungiworst case we do like puppet-zuul and remove the dir, then pass -C s that it's created again18:00
mordredyah - it's a temp dir anyway18:00
mordredclarkb: patch lgtm - want me to wait on you verifying * ?18:00
clarkbmordred: ya I'm just going to update it to explicitly set shell18:01
*** signed8bit is now known as signed8bit_Zzz18:01
*** signed8bit_Zzz is now known as signed8bit18:01
mordredkk18:01
openstackgerritClark Boylan proposed openstack-infra/puppet-storyboard master: Clean the webclient unpack dir safely.  https://review.openstack.org/57000418:02
clarkbstoryboard and sni were the two things I wanted to address that were largely ignored during the summit. Anything else I'm forgetting or continuing to ignore?18:04
openstackgerritMerged openstack-infra/zuul master: correct github review approval type is `approved`  https://review.openstack.org/57037818:06
funginot to my knowledge. i'm mostly focusing on limesurvey and mailman3 at this particular point in time18:08
*** felipemonteiro has joined #openstack-infra18:08
*** pcaruana has quit IRC18:09
*** ccamacho has quit IRC18:11
*** felipemonteiro has quit IRC18:13
*** lpetrut has joined #openstack-infra18:14
*** gfidente is now known as gfidente|pto18:17
*** gfidente|pto is now known as gfidente|afk18:17
zxiirocorvus: I could check. do you have a ticket?18:19
openstackgerritMerged openstack-infra/zuul master: Add missing pathlib dependency  https://review.openstack.org/57053918:19
*** mriedem1 has joined #openstack-infra18:21
*** mriedem has quit IRC18:23
zxiirocorvus: I don't see any open tickets. which email did you send your request to?18:23
*** mriedem1 is now known as mriedem18:24
*** lpetrut has quit IRC18:26
corvuszxiiro: i did not get a ticket response; i sent mail to helpdesk@opendaylight.org18:27
zxiirocorvus: odd, i guess the mail didn't make it over. because I don't see any tickets open about this. Can you try sending again?18:28
zxiiroI can watch the inbox and confirm it came in18:28
corvuszxiiro: sent18:29
*** ssbarnea_ has quit IRC18:31
zxiirocorvus: odd. I don't see it. I'm asking internally if someone can check if we're rejecting the mail somehow.18:32
corvuszxiiro: if it helps, here's the relevant log from my side (with the ack and queue id from the lf smpt server): 2018-05-30 18:23:11 1fO5kc-0004aB-MA => helpdesk@opendaylight.org R=dnslookup T=remote_smtp H=smtp1.linuxfoundation.org [140.211.169.13] X=TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128 C="250 2.0.0 Ok: queued as 04A13732"18:35
zxiirothanks, I'll pass it along18:36
*** signed8bit is now known as signed8bit_Zzz18:38
*** jcoufal_ has joined #openstack-infra18:40
*** signed8bit_Zzz is now known as signed8bit18:41
*** jcoufal has quit IRC18:44
zxiirocorvus: can you forward the email to jevans@linuxfoundation.org18:44
zxiirocorvus: Apparently it's getting through from smtp level but something's happening inside the ticket system that we need to figure out.18:45
openstackgerritMerged openstack-infra/puppet-storyboard master: Clean the webclient unpack dir safely.  https://review.openstack.org/57000418:46
*** iranzo has quit IRC18:46
corvuszxiiro: i sent another copy with the same text to jevans (i figured that would be most useful since i don't otherwise have a copy of exactly what was delivered)18:48
*** diablo_rojo has joined #openstack-infra18:48
*** dhinesh has quit IRC18:49
*** e0ne has quit IRC18:50
*** markvoelker has joined #openstack-infra18:50
*** rwsu has quit IRC18:51
zxiirocorvus: thanks18:51
*** markvoelker has quit IRC18:55
clarkbfungi: did you want to remove storyboard from the emergency hosts file now that ^ is in?18:56
clarkbfungi: I guess we'll see if it works on -dev first?18:56
*** yolanda has joined #openstack-infra18:57
clarkbI guess we also need a new storyboard client tarball to check it as well18:57
clarkbcan we get one of those simply by merging a commit or is it smarter than that?18:57
zxiirocorvus: alright, looks like we got the email in RT. and I've also went ahead and resolved it. openstack-zuul should have stream-events now.18:57
clarkbGerrit 2.15 GC feature results in file descriptor leakages...18:58
*** yolanda_ has quit IRC18:59
SotKclarkb: merging a webclient commit will cause a new tarball19:00
clarkbSotK: good to know thanks. We probably want to have that happen once we are ready to watch -dev then maybe against after we remove prod from the exclude list19:01
SotKthat should be easy enough, we have a few things waiting to merge I think19:02
*** felipemonteiro has joined #openstack-infra19:04
*** janki has quit IRC19:08
mordredSotK: I just rechecked a couple of the patches in the project-name-in-url series so we could see previews19:18
fungiclarkb: yeah, in theory we could merge a new sb-wc change to trigger deployment to sb-dev.o.o and see how it goes19:19
fungithough also possible deleting the old tarball will work19:19
mordredit should do ... the wget command will download it if tere isn't a local file that matches, and that shouldkick the remaining chain19:22
clarkbok I'll remove the existing tarball on -dev19:22
clarkbmoved it aside19:23
clarkbnext puppet run starting in 5 minutes should test everything out19:25
*** felipemonteiro has quit IRC19:26
*** felipemonteiro has joined #openstack-infra19:27
*** gfidente|afk has quit IRC19:31
openstackgerritPaul Belanger proposed openstack-infra/zuul master: Add the ability for a user to skip macros  https://review.openstack.org/57128719:31
*** lifeless has joined #openstack-infra19:31
*** nicolasbock has quit IRC19:33
*** ssbarnea_ has joined #openstack-infra19:33
*** wolverineav has quit IRC19:35
*** wolverineav has joined #openstack-infra19:35
*** wolverineav has quit IRC19:40
*** signed8bit is now known as signed8bit_Zzz19:40
*** signed8bit_Zzz is now known as signed8bit19:43
*** markvoelker has joined #openstack-infra19:45
*** slaweq_ has joined #openstack-infra19:47
*** slaweq has quit IRC19:48
*** nicolasbock has joined #openstack-infra19:48
*** eharney has quit IRC19:49
clarkbstoryboard-dev.openstack.org appears to have done the correct thing. I think we want it to have another puppet run to be sure that the next time around it won't break everything19:49
clarkbbut if that ends up being happy I think we can remove storyboard.openstack.org from the emergency file and manually remove its content tarball to force it to rerun there as well19:50
clarkbfungi: ^19:50
clarkbI'm going to find lunch now19:51
fungigreat!19:51
*** kjackal has quit IRC19:53
*** markvoelker has quit IRC19:54
*** kien-ha has quit IRC19:59
*** efried has quit IRC20:03
*** efried has joined #openstack-infra20:04
efriedclarkb: That seems to have worked.  Thanks!20:11
*** wolverineav has joined #openstack-infra20:13
*** roman_g has quit IRC20:14
*** armaan has joined #openstack-infra20:15
*** wolverin_ has joined #openstack-infra20:16
*** wolverineav has quit IRC20:16
*** felipemonteiro has quit IRC20:20
clarkbfungi: looks like -dev ran puppet again and nooped20:25
clarkband my browers is still happy with it after a hard refresh20:25
*** trown is now known as trown|outtypewww20:25
*** markvoelker has joined #openstack-infra20:25
clarkbdid you want to go ahead and remove the prod server from the emergency file? I am aroudn to help watch it though probably not primary on that as I need to start writing emails20:26
clarkbsummit is basically an email generator20:26
*** florianf has quit IRC20:34
mnasercorvus: time to bikeshed but instead of 'winterscale' we could have gone for 'snowtrain' as a reminder of our past 2 ptgs :P20:38
mnaserbut we do want to keep it seperate, but, i couldn't keep that comment to myself :)20:39
*** rwsu has joined #openstack-infra20:43
corvusmnaser: i love it.  bikeshedding over the codename used to talk about the thing about whose name we're bikeshedding.  :)20:43
mnaser:D20:44
mnaseroverall though i'm very +1 with the proposal, it's about time "openstack-infra" becomes its thing20:44
*** kgiusti has left #openstack-infra20:44
mnaserthings like zuul multitenancy and the state we're in means its the perfect time to do that20:45
*** felipemonteiro has joined #openstack-infra20:46
*** hasharAway has quit IRC20:51
fungiclarkb: yep, i'm unleashing the ansibles on sb.o.o again now20:52
clarkbcool20:52
fungiand done. thanks!20:53
fungi#status log storyboard.openstack.org has been removed from the emergency disable list now that storyboard-webclient tarball deployment is fixed20:54
openstackstatusfungi: finished logging20:54
clarkbfungi: are you going to remove the old tarball too?20:54
fungii think we've merged at least one sb-wc change since puppet was disabled. checking20:58
funginope, i'm wrong. current deployment seems to be tip of master already20:59
fungiso yeah i'll get the old tarball out of the way to make sure redeployment is exercised properly20:59
fungiand watching syslog to see when it fires21:00
*** salv-orlando has joined #openstack-infra21:00
*** salv-orlando has quit IRC21:01
*** salv-orlando has joined #openstack-infra21:01
*** neiloy has quit IRC21:06
fungilooks like it's starting21:10
*** salv-orlando has quit IRC21:13
*** markvoelker has quit IRC21:14
*** ssbarnea_ has quit IRC21:14
*** sree has joined #openstack-infra21:14
*** markvoelker has joined #openstack-infra21:14
*** ldnunes has quit IRC21:14
*** VW_ has joined #openstack-infra21:16
*** markvoelker has quit IRC21:18
fungiseems to have redeployed cleanly, and the webui is still working21:19
*** sree has quit IRC21:19
*** VW has quit IRC21:20
*** VW_ has quit IRC21:21
*** lifeless has quit IRC21:26
*** lifeless_ has joined #openstack-infra21:26
ianwclarkb / fungi : can you read over https://review.openstack.org/#/c/571046/ quickly which adds some notes from our reprepo discussion the other day21:27
fungisure21:28
ianwi'm also working on some notes for afs debugging from what i learnt during recent outage21:28
ianwand i also found some cool afs contrib utils, and one of them looks like a better basis for getting some stats into statsd and and a grafana dashboard than what i'd previously started21:28
openstackgerritMerged openstack-infra/system-config master: openstack_project: opensuse: Fix OBS repository url  https://review.openstack.org/56692421:30
*** lifeless has joined #openstack-infra21:38
*** lifeless_ has quit IRC21:38
*** armaan has quit IRC21:39
*** jcoufal_ has quit IRC21:41
*** esberglu has quit IRC21:43
clarkbfungi: yay re storyboard. I'm going to read ianw's change now then context switch over to testing the new ansible inventory stuff21:48
ianwclarkb: i didn't follow too much of that, but i'm around if i can help on that21:49
clarkbianw: I think we've mostly worked through the problems. Will let you know if new ones pop up21:50
*** harlowja has joined #openstack-infra21:52
fungiianw: a few nits/suggestions marked inline, but all in all i'm fine with it as-is21:55
*** caphrim007_ has joined #openstack-infra21:55
*** caphrim007 has quit IRC21:59
clarkbianw: fungi I've +2'd but not approved. Will let ianw decide if the comments are worth addressing prior to approval (otherwise approving si fine by me)21:59
openstackgerritIan Wienand proposed openstack-infra/system-config master: Add some rationale for reprepro  https://review.openstack.org/57104622:02
ianwthanks, added some updates.22:03
ianwclarkb: hmm, so is the ansible inventory stuff preventing updates on mirror-update?  i'd expected hworang's suse update fix to be deployed by now22:03
clarkbok openstack_inventory.py seems to work22:03
*** lifeless has quit IRC22:03
clarkbianw: we pinned to old ansible earlier today (maybe 6 hours ago?) which got it running again22:03
clarkbianw: but yes prior to that it would've been possible22:04
*** lifeless has joined #openstack-infra22:04
ianwhmm, ok22:05
clarkbianw: I just tseted that the new thing would work out of band (I think it will) so will unwip 57126322:05
ianwlast run was May 30 21:37:18, maybe it missed and i'm just impatient22:06
openstackgerritClark Boylan proposed openstack-infra/system-config master: Switch to using new ansible openstack inventory script  https://review.openstack.org/57126322:07
clarkbit had a bug I think ^ should be happy now and I tested that the usage of that inventory file works on puppetmaster22:07
clarkbmordred: ^ do you want to review that?22:07
*** kien-ha has joined #openstack-infra22:08
clarkb15 > 42 == True is my new favorite fact :)22:15
clarkb#status log storyboard is deploying latest webclient again after fixing the deployment process around the webclient.22:17
openstackstatusclarkb: finished logging22:17
*** rcernin has joined #openstack-infra22:22
*** dave-mccowan has quit IRC22:23
*** rlandy|rover is now known as rlandy|rover|bbl22:23
*** slaweq_ has quit IRC22:30
*** slaweq has joined #openstack-infra22:30
ianwthe remote rsync failed on the first attempt at the suse update, running it again22:31
*** salv-orlando has joined #openstack-infra22:32
*** lifeless has quit IRC22:33
*** slaweq has quit IRC22:34
*** lifeless has joined #openstack-infra22:35
*** salv-orlando has quit IRC22:37
*** hongbin has quit IRC22:38
*** jesslampe has joined #openstack-infra22:40
*** rossella_s has quit IRC22:41
*** rossella_s has joined #openstack-infra22:42
*** r-daneel has quit IRC22:42
openstackgerritAdam Spiers proposed openstack-infra/irc-meetings master: Remove weekly HA meeting  https://review.openstack.org/57132622:44
*** wolverin_ has quit IRC22:45
*** wolverineav has joined #openstack-infra22:45
openstackgerritAdam Spiers proposed openstack-infra/irc-meetings master: Remove weekly HA meeting  https://review.openstack.org/57132622:46
*** nicolasbock has quit IRC22:47
openstackgerritMerged openstack-infra/system-config master: Add some rationale for reprepro  https://review.openstack.org/57104622:48
*** mriedem is now known as mriedem_afk22:48
*** wolverineav has quit IRC22:50
*** rossella_s has quit IRC22:51
*** edmondsw has quit IRC22:52
*** edmondsw has joined #openstack-infra22:53
*** signed8bit is now known as signed8bit_Zzz22:53
*** rossella_s has joined #openstack-infra22:54
*** edmondsw has quit IRC22:57
*** tosky has quit IRC23:04
*** dpawlik has joined #openstack-infra23:10
*** lifeless has quit IRC23:11
*** jesslampe has quit IRC23:13
*** jesslampe has joined #openstack-infra23:13
tonybCan I be added to bootstrappers group for ~10mins to EOL(newton) Kolla and OSA as per http://lists.openstack.org/pipermail/openstack-dev/2018-April/129366.html and http://lists.openstack.org/pipermail/openstack-dev/2018-April/129981.html ?23:14
*** dpawlik has quit IRC23:14
openstackgerritAmy Marrich (spotz) proposed openstack-infra/system-config master: Update of codeblock for adding a new list  https://review.openstack.org/57132923:14
clarkbtonyb: sure23:14
tonybclarkb: Thanks23:15
clarkbtonyb: you are in there now. standard disclaimer and all that23:15
openstackgerritMerged openstack-infra/irc-meetings master: Remove weekly HA meeting  https://review.openstack.org/57132623:15
tonybclarkb: Awesome23:15
spotztonyb You be nice to OSA Newton, it was a good friend:)23:16
tonybspotz: I'm only doing as evrardjp asked ;P  but of course I'll do it as kindly as I can :)23:16
spotztonyb hehe23:17
clarkbcorvus: we both +2'd spotz change at the same time :P I'll approve it23:17
clarkbbah we both approved too23:17
corvusclarkb: if you do we'll both have approved it :)23:17
corvusha23:17
spotzclarkb corvus - thanks! there will be a patch in the next frw days for a new list for the mentoring program:) Wanted to make sure the docs were right as it'll be Ell's first patch:)23:18
*** wolverineav has joined #openstack-infra23:18
clarkbspotz: thank you for the fix23:18
spotzAnd I'll try to do some reviews myself after I hit the OSA reviews23:18
*** felipemonteiro has quit IRC23:18
spotzAnd fix the cinder install guide:)23:19
*** felipemonteiro has joined #openstack-infra23:25
*** markvoelker has joined #openstack-infra23:25
*** salv-orlando has joined #openstack-infra23:33
*** salv-orlando has quit IRC23:34
*** salv-orlando has joined #openstack-infra23:34
clarkbcorvus: zuul has been running for almost 3 weeks without a restart and is using less than 3GB resident memory for the scheduler and server total is under 5GB :)23:36
corvus\o/23:37
clarkbthinking out loud here we should probably consider reducing the size of that server. Also switching to our zookeeper cluster23:39
*** salv-orlando has quit IRC23:40
clarkbnot urgent but would be nice to free up some resources.23:40
clarkbsimilarly I think we may need to delete the old trusty gerrit server?23:40
corvusyep, and i think we have enough data for our baseline now23:40
* clarkb scribbles a note to followup on the gerrit thing soon23:40
*** lifeless has joined #openstack-infra23:41
clarkba 16GB instance should be plenty big and accomodate the cache being used23:41
ianwrsync: change_dir "/opensuse/repositories/Cloud:/OpenStack:/Pike/openSUSE_Leap_15.0" (in pub) failed: No such file or directory (2)23:42
ianwclarkb: ^ any particular a priori insight into that?23:42
clarkbno, sorry23:42
ianwOBS_REPOS=('Virtualization:/containers' 'Cloud:/OpenStack:/Pike' 'Cloud:/OpenStack:/Queens') ... i guess it is assumed all of those are valid for all suse releases23:43
tonybclarkb: Thanks  I did the OSA ones but not kolla, and I've removed myself from that group23:43
*** markvoelker has quit IRC23:43
clarkbianw: that is probably a good guess, UCA has similar "issues" iirc23:43
clarkbtonyb: thanks23:43
*** markvoelker has joined #openstack-infra23:44
clarkbianw: http://download.opensuse.org/repositories/Cloud:/OpenStack:/ browesable repos23:44
clarkband yup looks like leap 15 starts with queens so pike is too old23:44
ianwhmm, not sure why the mirror has "Staging"  "http://download.opensuse.org/repositories/Cloud:/OpenStack:/Queens:/Staging/"23:47
tonybclarkb: Actually can you add me back again, I just worked out what I was doign wrong with kolla23:47
*** caphrim007_ has quit IRC23:47
clarkbtonyb: sure23:47
clarkbianw: http://download.opensuse.org/repositories/Cloud:/OpenStack:/Queens/ : vs without :23:48
clarkbtonyb: done23:48
*** markvoelker has quit IRC23:48
openstackgerritMerged openstack-infra/system-config master: Update of codeblock for adding a new list  https://review.openstack.org/57132923:50
tonybclarkb: Thanks, kolla has been tagged all done (for real this time)23:54
*** rpioso is now known as rpioso|afk23:55
*** lifeless has quit IRC23:58
*** eharney has joined #openstack-infra23:58
*** lifeless has joined #openstack-infra23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!