Tuesday, 2017-12-19

openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add an authoritative hidden master  https://review.openstack.org/52843800:00
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884600:00
*** jcoufal has joined #openstack-infra00:00
jeblairclarkb, fungi, dmsimard: ^ those changes are manually-puppet-applied on adns1.openstack.org and ns1.openstack.org.  values are in hiera.00:01
jeblairi'm sure the whitespace is wrong, but i think the functionality is there00:01
clarkbjeblair: can I dig zuul-ci.org against ns1.openstack.org yet?00:01
clarkbdig worked but zone isn't there yet00:02
clarkboh wait00:02
clarkbI asked for an A record00:02
clarkbnice it totally owrk00:02
jeblairclarkb: yes.  lists is a cname in there00:02
jeblairshould be able to get soa or lists00:02
clarkbyup works00:03
clarkb(I asked for NS)00:03
*** xarses_ has joined #openstack-infra00:04
*** d0ugal has joined #openstack-infra00:04
*** jcoufal has quit IRC00:05
*** andreww has joined #openstack-infra00:07
*** andreww has quit IRC00:07
jeblairclarkb: dig +dnssec zuul-ci.org @ns1.openstack.org    should also work00:07
*** xarses_ has quit IRC00:08
*** rcernin has quit IRC00:09
*** rcernin_ has joined #openstack-infra00:09
*** andreww has joined #openstack-infra00:09
clarkbjeblair: in your docs it says run dnssec-keygen -fx, reading the manpage -f takes KSK or REVOKE?00:09
jeblairtypo, should be -fk00:10
jeblairwill fix00:10
clarkbis that short for ksk?00:11
jeblairi'm guessing so.  it's what all the tutorials say to do to generate a ksk00:11
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add an authoritative hidden master  https://review.openstack.org/52843800:12
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884600:12
jeblairclarkb: you can examine the output on adns1 or in hiera.  the generated key does say that it is a ksk00:12
*** ijw has joined #openstack-infra00:13
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884600:14
clarkbjeblair: left one more comment on the old patchset00:17
fungiokay, back and catching up00:17
*** ijw has quit IRC00:18
jeblairclarkb: ah yep00:18
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add an authoritative hidden master  https://review.openstack.org/52843800:21
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884600:21
fungijeblair: ahh, yeah, if we authenticate axfr then i don't think we need to bother with ip address identification anyway. i mean it would be nice to have both i guess so that it's harder for someone to try to brute-force your key, but not really critical00:22
clarkbjeblair: lgtm00:22
*** lukebrowning has joined #openstack-infra00:24
fungireviewing now, then i'll catch up on the zuul meeting log00:24
jeblairfungi: we do have iptables rules on adns1, so we have a form of ip protection + tsig.00:24
clarkbjeblair: isn't it in the wrong direction though?00:25
fungioh, that's a great point. i kinda trust iptables to do a better job at that anyway00:25
clarkb(I mean its probably a good idea to have the rule regardless but I think tcp connection will initiate the other direction won't it?)00:25
*** ijw has joined #openstack-infra00:26
jeblairclarkb: the axfr requests come from ns* to adns1, and adns1 only allows tcp/53 from ns*00:26
fungiclarkb: the silent master doesn't need to allow 53 )tcp or udp) from anybody except the slaves00:26
fungior what jeblair managed to say in far fewer typos00:26
clarkbbut the notify is the other way? I guess the transfer is what matters ultimately00:27
funginotifies go from adns1 to ns*, and then axfr queries come from ns* to adns100:27
jeblairthe slaves receive notify requests from the master, which are not tsig-signed, since puppet-nsd doesn't expose that option.  but nsd is set up to do ip acls on those.00:27
clarkbgotcha00:27
fungiworst case someone spoofs a bunch of notifies to the slaves and they axfr to the silent master more often00:28
*** lukebrowning has quit IRC00:28
jeblair(nsd does support requiring tsig on notifies, but for some reason, the puppet module doesn't expose that.  that *might* be different in the latest version (requires puppet4).  but a lot has changed and i didn't trace it all through.00:28
jeblairfungi: ya00:28
jeblairi was a bit surprised -- the puppet-nsd module is otherwise amazing00:29
*** ijw has quit IRC00:30
fungithe also-notify is probably also not strictly necessary since i'm pretty sure bind's default behavior is to notify any nameservers with ns records for the zone being updated00:31
fungibut it can't hurt00:32
jeblairfungi: yes i thought so too -- but fore some reason i wasn't seeing it happen in my testing.  i could try again, though i don't believe i did anything to "fix" that other than add also-notify00:32
fungimeh, i wouldn't sweat it then00:33
fungijeblair: i love the implementation here. we get the flexibility of bind's vast featureset coupled with the more limited risk profile of nsd00:34
*** marst has quit IRC00:35
clarkbjeblair: oh, one question what tells bind to pick up the new dns zones in the first place? doesn't it need a service reload for that (or the more specific managmeent command I can't remember because service reload worked on $distro)00:36
jeblairfungi: yeah, i'm digging it!  i think it would also be easy to do a bind slave -- and the hidden master approach makes that nice.00:36
clarkboh there is a puppet notify service thing in there00:37
jeblairclarkb: yep00:37
clarkbso I guess puppet will do a service restart but since bind isn't the public facing ns thats fine00:37
clarkband as long as we update zones via puppet nad not some other system too there won't be a race on those restarts so should be fine00:37
jeblairand restarts send out notifies00:37
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Xenial: add a service file  https://review.openstack.org/52862600:37
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Add basic rspec test  https://review.openstack.org/52813000:37
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: [WIP] try a later version  https://review.openstack.org/52815600:37
*** bobh has joined #openstack-infra00:39
*** d0ugal has quit IRC00:40
fungiand i'm caught up on the happy holidays edition of the zuul meeting log as well now00:43
*** xarses has joined #openstack-infra00:45
*** pvaneck has quit IRC00:50
*** dhill_ has quit IRC00:50
*** rcernin_ has quit IRC00:50
*** d0ugal has joined #openstack-infra00:51
*** jamesmcarthur has joined #openstack-infra00:52
*** psachin has joined #openstack-infra00:52
*** annp has quit IRC00:53
*** annp has joined #openstack-infra00:53
*** daidv has quit IRC00:53
*** daidv has joined #openstack-infra00:54
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Xenial: add a service file  https://review.openstack.org/52862600:55
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Add basic rspec test  https://review.openstack.org/52813000:55
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: [WIP] try a later version  https://review.openstack.org/52815600:55
*** jamesmcarthur has quit IRC00:56
*** catintheroof has joined #openstack-infra00:56
fungioh, interesting... minor hole in gerrit's commit message update detection00:57
fungii have two changes in a series. moving some of the delta from the parent change to the child results in gerrit thinking i only updated the commit message of the child because the diff of the resulting code between patchsets didn't change00:58
*** edmondsw has joined #openstack-infra00:58
clarkbha00:58
fungi526140 for an example00:58
fungipatchset 4 includes half a dozen lines moved into it from the parent change's tox.ini00:59
*** gyee has quit IRC01:01
*** kiennt26 has joined #openstack-infra01:03
*** lukebrowning has joined #openstack-infra01:04
*** cuongnv has joined #openstack-infra01:05
fungidoing it again01:07
openstackgerritJeremy Stanley proposed openstack-infra/zuul-base-jobs master: Initial boilerplate, packaging and testing  https://review.openstack.org/52613901:07
openstackgerritJeremy Stanley proposed openstack-infra/zuul-base-jobs master: Add generic base and base-test jobs/playbooks  https://review.openstack.org/52614001:07
fungiyep. "Uploaded patch set 5: Commit message was updated."01:08
fungithis one even moved some entire files from the parent change to the child01:08
*** lukebrowning has quit IRC01:09
openstackgerritCao Xuan Hoang proposed openstack-infra/project-config master: Add neutron-vpnaas/vpnaas-dashboard to gerritbot  https://review.openstack.org/52786701:10
*** lukebrowning has joined #openstack-infra01:10
clarkbfungi: does this make you the gerrit upgrade tester?01:11
*** andreww has quit IRC01:11
*** salv-orlando has quit IRC01:13
*** salv-orlando has joined #openstack-infra01:14
*** bobh has quit IRC01:14
*** bobh has joined #openstack-infra01:14
*** lukebrowning has quit IRC01:15
*** lukebrowning has joined #openstack-infra01:17
*** iyamahat has joined #openstack-infra01:17
*** salv-orlando has quit IRC01:18
*** Swimingly has quit IRC01:20
*** Swimingly has joined #openstack-infra01:20
*** catintheroof has quit IRC01:21
*** lukebrowning has quit IRC01:21
*** lukebrowning has joined #openstack-infra01:23
*** ijw has joined #openstack-infra01:26
*** lukebrowning has quit IRC01:28
*** lukebrowning has joined #openstack-infra01:29
*** ijw has quit IRC01:32
*** Apoorva_ has joined #openstack-infra01:32
*** linkmark has quit IRC01:33
*** lukebrowning has quit IRC01:34
*** lukebrowning has joined #openstack-infra01:35
*** Apoorva has quit IRC01:36
*** Apoorva_ has quit IRC01:37
*** lukebrowning has quit IRC01:40
*** lukebrowning has joined #openstack-infra01:42
*** lukebrowning has quit IRC01:46
*** liujiong has joined #openstack-infra01:47
*** lukebrowning has joined #openstack-infra01:48
*** claudiub has joined #openstack-infra01:48
*** lukebrowning has quit IRC01:53
*** edmondsw has quit IRC01:53
*** ijw has joined #openstack-infra01:54
*** lukebrowning has joined #openstack-infra01:54
*** andreas_s has joined #openstack-infra01:55
*** jamesmcarthur has joined #openstack-infra01:56
*** wolverineav has joined #openstack-infra01:56
*** lukebrowning has quit IRC01:59
*** andreas_s has quit IRC01:59
*** wolverineav has quit IRC02:01
*** jamesmcarthur has quit IRC02:01
*** ijw has quit IRC02:01
*** lukebrowning has joined #openstack-infra02:02
*** lukebrowning has quit IRC02:06
*** ijw has joined #openstack-infra02:06
*** lukebrowning has joined #openstack-infra02:08
*** rcernin has joined #openstack-infra02:11
*** lukebrowning has quit IRC02:13
*** lukebrowning has joined #openstack-infra02:14
*** salv-orlando has joined #openstack-infra02:14
*** ijw has quit IRC02:15
*** r-daneel has quit IRC02:17
*** psachin has quit IRC02:18
*** lukebrowning has quit IRC02:19
*** salv-orlando has quit IRC02:20
*** bobh has quit IRC02:20
*** lukebrowning has joined #openstack-infra02:20
*** daidv has quit IRC02:20
*** bobh has joined #openstack-infra02:21
*** daidv has joined #openstack-infra02:21
*** bobh has quit IRC02:24
*** bobh has joined #openstack-infra02:24
*** lukebrowning has quit IRC02:25
*** claudiub has quit IRC02:25
*** lukebrowning has joined #openstack-infra02:27
*** gcb has joined #openstack-infra02:27
*** Apoorva has joined #openstack-infra02:29
*** lukebrowning has quit IRC02:31
*** lukebrowning has joined #openstack-infra02:33
*** psachin has joined #openstack-infra02:36
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Xenial: add a service file  https://review.openstack.org/52862602:36
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Add basic rspec test  https://review.openstack.org/52813002:36
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: [WIP] try a later version  https://review.openstack.org/52815602:36
*** ramishra has joined #openstack-infra02:37
*** lukebrowning has quit IRC02:38
*** lukebrowning has joined #openstack-infra02:39
*** ramishra has quit IRC02:40
*** liujiong has quit IRC02:42
*** ramishra has joined #openstack-infra02:43
*** lukebrowning has quit IRC02:44
*** lukebrowning has joined #openstack-infra02:45
*** jascott1_ has joined #openstack-infra02:47
*** jascott1 has quit IRC02:47
lbragstado/ i'm seeing some documentation failures that i can't recreate locally https://review.openstack.org/#/c/512468/602:48
lbragstadis that possibly related to http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html ?02:48
*** liujiong has joined #openstack-infra02:48
*** lukebrowning has quit IRC02:50
*** lukebrowning has joined #openstack-infra02:52
*** bobh has quit IRC02:52
*** bobh has joined #openstack-infra02:53
*** ijw has joined #openstack-infra02:54
*** liujiong has quit IRC02:54
*** bmjen has quit IRC02:56
*** Hunnur has quit IRC02:56
*** mriedem has quit IRC02:56
*** lukebrowning has quit IRC02:56
*** liujiong has joined #openstack-infra02:56
*** ijw has quit IRC02:59
*** lukebrowning has joined #openstack-infra03:03
*** lukebrowning has quit IRC03:07
*** lukebrowning has joined #openstack-infra03:09
*** rlandy|rover has quit IRC03:10
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Xenial: add a service file  https://review.openstack.org/52862603:11
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Add basic rspec test  https://review.openstack.org/52813003:11
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: [WIP] try a later version  https://review.openstack.org/52815603:11
*** lukebrowning has quit IRC03:13
clarkblbragstad: http://logs.openstack.org/68/512468/6/check/build-openstack-sphinx-docs/0773382/job-output.txt.gz#_2017-12-19_00_23_32_716523 is the error03:14
*** edmondsw has joined #openstack-infra03:15
clarkblbragstad: which should've been fixed by https://review.openstack.org/#/c/528796 I think03:15
*** ramishra has quit IRC03:15
*** salv-orlando has joined #openstack-infra03:15
*** bobh has quit IRC03:17
*** yamahata has joined #openstack-infra03:17
clarkbsphinx_pbr_autodoc was set to false according to ara03:18
clarkbI see why. I'll get a patch up shortly03:19
*** lukebrowning has joined #openstack-infra03:19
*** edmondsw has quit IRC03:19
*** salv-orlando has quit IRC03:20
lbragstadclarkb: aha - so hold off on a recheck then?03:20
*** armax has quit IRC03:22
clarkbya I don't think a recheck will work03:23
*** markvoelker has joined #openstack-infra03:24
*** lukebrowning has quit IRC03:25
openstackgerritClark Boylan proposed openstack-infra/zuul-jobs master: Support autodoc_tree_index_modules in sphinx role  https://review.openstack.org/52894603:26
clarkblbragstad: AJaeger mordred config-core ^ I think taht should do it03:26
*** lukebrowning has joined #openstack-infra03:26
*** yamahata has quit IRC03:27
*** iyamahat has quit IRC03:27
*** iyamahat has joined #openstack-infra03:27
*** lukebrowning has quit IRC03:30
*** lukebrowning has joined #openstack-infra03:32
ianwwow, there's no end of options, are there03:34
*** iyamahat has quit IRC03:35
clarkbianw: configure all the things03:37
clarkbI'm still not sure I understand the difference reading pbr's docs03:38
clarkbbut keystone definitely uses the one I Just added into the ansible role03:38
ianwyeah, i have fought with those before03:38
*** lukebrowning has quit IRC03:41
*** e0ne has joined #openstack-infra03:42
*** lukebrowning has joined #openstack-infra03:42
*** ramishra has joined #openstack-infra03:43
*** e0ne has quit IRC03:47
*** lukebrowning has quit IRC03:47
ianw2017-12-19 03:30:50.967916 | TASK [sphinx : Attempt to get warning-is-error from config file]03:48
ianw2017-12-19 03:30:51.954655 | ubuntu-xenial | MODULE FAILURE03:48
ianwthat doesn't give much help03:48
clarkbdid my change fail?03:48
ianwclarkb: http://logs.openstack.org/46/528946/1/check/build-openstack-sphinx-docs/a5ff336/job-output.txt.gz#_2017-12-19_03_30_50_96791603:48
clarkbI probably made a python fail03:49
*** lukebrowning has joined #openstack-infra03:49
clarkbyup03:50
clarkbara has details I will get a patch up03:50
openstackgerritClark Boylan proposed openstack-infra/zuul-jobs master: Support autodoc_tree_index_modules in sphinx role  https://review.openstack.org/52894603:51
*** dave-mccowan has quit IRC03:51
clarkbianw: ^ needed to set the variable first as without that in your setup.cfg the code to set the value won't be run03:51
ianwahh, that makes sense03:52
*** lukebrowning has quit IRC03:54
*** ijw has joined #openstack-infra03:55
*** lukebrowning has joined #openstack-infra03:55
*** markvoelker has quit IRC03:58
*** ijw has quit IRC03:59
*** lukebrowning has quit IRC04:00
*** bobh has joined #openstack-infra04:00
*** e0ne has joined #openstack-infra04:01
*** armax has joined #openstack-infra04:01
*** lukebrowning has joined #openstack-infra04:03
*** bobh has quit IRC04:04
*** Apoorva has quit IRC04:04
*** lukebrowning has quit IRC04:08
*** lukebrowning has joined #openstack-infra04:09
*** e0ne has quit IRC04:13
*** lukebrowning has quit IRC04:14
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: zk: automatically retry command when connection is lost  https://review.openstack.org/52364004:14
*** lukebrowning has joined #openstack-infra04:15
*** e0ne has joined #openstack-infra04:16
*** salv-orlando has joined #openstack-infra04:16
*** gouthamr has quit IRC04:18
*** lukebrowning has quit IRC04:20
*** salv-orlando has quit IRC04:21
*** e0ne has quit IRC04:21
*** lukebrowning has joined #openstack-infra04:21
*** lukebrowning has quit IRC04:26
*** e0ne has joined #openstack-infra04:26
*** lukebrowning has joined #openstack-infra04:28
*** psachin has quit IRC04:30
*** e0ne has quit IRC04:31
*** coolsvap has joined #openstack-infra04:32
*** jamesmcarthur has joined #openstack-infra04:32
*** lukebrowning has quit IRC04:33
*** lukebrowning has joined #openstack-infra04:34
*** psachin has joined #openstack-infra04:35
*** jamesmcarthur has quit IRC04:36
*** lukebrowning has quit IRC04:39
*** lukebrowning has joined #openstack-infra04:40
*** andreas_s has joined #openstack-infra04:40
*** e0ne has joined #openstack-infra04:41
*** iyamahat has joined #openstack-infra04:42
*** janki has joined #openstack-infra04:43
*** ykarel has joined #openstack-infra04:44
*** lukebrowning has quit IRC04:45
*** andreas_s has quit IRC04:45
*** e0ne has quit IRC04:46
*** lukebrowning has joined #openstack-infra04:46
*** annp has quit IRC04:51
*** lukebrowning has quit IRC04:51
*** annp has joined #openstack-infra04:52
*** lukebrowning has joined #openstack-infra04:53
*** armax has quit IRC04:53
*** kiennt26 has quit IRC04:54
*** daidv has quit IRC04:54
*** cuongnv has quit IRC04:54
*** daidv has joined #openstack-infra04:55
*** cuongnv has joined #openstack-infra04:55
*** kiennt26 has joined #openstack-infra04:55
*** markvoelker has joined #openstack-infra04:55
*** e0ne has joined #openstack-infra04:56
*** gcb has quit IRC04:56
*** yamamoto has joined #openstack-infra04:57
*** lukebrowning has quit IRC04:58
*** gcb has joined #openstack-infra04:58
*** pgadiya has joined #openstack-infra05:01
*** e0ne has quit IRC05:01
*** e0ne_ has joined #openstack-infra05:01
*** lukebrowning has joined #openstack-infra05:04
*** jascott1_ has left #openstack-infra05:06
*** e0ne_ has quit IRC05:07
*** yamamoto has quit IRC05:07
*** psachin has quit IRC05:07
*** yamamoto has joined #openstack-infra05:08
*** lukebrowning has quit IRC05:09
*** janki has quit IRC05:09
*** lukebrowning has joined #openstack-infra05:10
*** e0ne has joined #openstack-infra05:10
*** janki has joined #openstack-infra05:10
*** lukebrowning has quit IRC05:15
*** e0ne has quit IRC05:15
*** psachin has joined #openstack-infra05:16
*** lukebrowning has joined #openstack-infra05:16
*** salv-orlando has joined #openstack-infra05:17
*** lukebrowning has quit IRC05:21
*** salv-orlando has quit IRC05:22
*** armax has joined #openstack-infra05:22
*** lukebrowning has joined #openstack-infra05:23
*** e0ne has joined #openstack-infra05:23
*** yamamoto has quit IRC05:27
*** lukebrowning has quit IRC05:28
*** e0ne has quit IRC05:28
*** markvoelker has quit IRC05:28
*** armax has quit IRC05:28
*** lukebrowning has joined #openstack-infra05:29
*** yamamoto has joined #openstack-infra05:30
*** e0ne has joined #openstack-infra05:32
*** lukebrowning has quit IRC05:34
*** lukebrowning has joined #openstack-infra05:35
*** yamamoto has quit IRC05:36
*** e0ne has quit IRC05:37
*** lukebrowning has quit IRC05:40
*** {aborted} is now known as chandankumar05:43
*** e0ne has joined #openstack-infra05:44
*** links has joined #openstack-infra05:47
*** lukebrowning has joined #openstack-infra05:48
*** e0ne has quit IRC05:49
*** lukebrowning has quit IRC05:53
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Update default version of nodejs  https://review.openstack.org/52815605:53
openstackgerritIan Wienand proposed openstack-infra/puppet-etherpad_lite master: Add basic rspec test  https://review.openstack.org/52813005:53
*** iyamahat_ has joined #openstack-infra05:53
*** iyamahat has quit IRC05:53
*** lukebrowning has joined #openstack-infra05:54
*** e0ne has joined #openstack-infra05:54
openstackgerritAndreas Jaeger proposed openstack-infra/irc-meetings master: Move irc meetings tox job in-tree  https://review.openstack.org/52849805:56
openstackgerritIan Wienand proposed openstack-infra/system-config master: Update etherpad versions for Xenial  https://review.openstack.org/52895905:57
AJaegertonyb: figured it out, the check_chairs.py call parsed .zuul.yaml ;( ^05:57
*** wolverineav has joined #openstack-infra05:57
AJaegerclarkb: thanks, +305:57
*** lukebrowning has quit IRC05:59
*** e0ne has quit IRC05:59
*** e0ne has joined #openstack-infra06:01
ianwclarkb: ^^ i think etherpad is getting close, but it's going to need a lot of babysitting06:02
*** wolverineav has quit IRC06:02
*** lukebrowning has joined #openstack-infra06:04
tonybAJaeger: Oh :(  That'd do it  sounds like a bug to me ;P06:04
openstackgerritMerged openstack-infra/zuul-jobs master: Support autodoc_tree_index_modules in sphinx role  https://review.openstack.org/52894606:06
*** annp has quit IRC06:08
*** annp has joined #openstack-infra06:09
*** lukebrowning has quit IRC06:09
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/52896306:10
*** lukebrowning has joined #openstack-infra06:10
*** afazekas has quit IRC06:11
*** afazekas has joined #openstack-infra06:11
*** e0ne has quit IRC06:13
*** namnh has joined #openstack-infra06:13
*** lukebrowning has quit IRC06:15
*** e0ne has joined #openstack-infra06:16
*** lukebrowning has joined #openstack-infra06:16
*** armax has joined #openstack-infra06:21
*** lukebrowning has quit IRC06:21
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Fix docs building  https://review.openstack.org/52882206:21
*** e0ne has quit IRC06:21
*** kiennt26 has quit IRC06:22
*** kiennt26 has joined #openstack-infra06:22
*** janki has quit IRC06:22
openstackgerritAndreas Jaeger proposed openstack-infra/irc-meetings master: Move irc meetings tox job in-tree  https://review.openstack.org/52849806:23
AJaegertonyb: now really fixed ^06:23
* AJaeger forgot to push a change06:24
*** lukebrowning has joined #openstack-infra06:24
AJaegerianw, frickler could you review two job removal changes for openstack-zuul-jobs: https://review.openstack.org/528581 and https://review.openstack.org/#/c/522471/ , please?06:25
*** markvoelker has joined #openstack-infra06:25
AJaegerlbragstad: the fix merged, I'll recheck your change06:26
*** armax has quit IRC06:28
*** CrayZee has joined #openstack-infra06:28
*** e0ne has joined #openstack-infra06:28
*** lukebrowning has quit IRC06:29
openstackgerritMerged openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/52896306:30
AJaegerianw: dmsimard asked that you review https://review.openstack.org/525658  - could you do so, please? His -1 is procedural06:30
*** lukebrowning has joined #openstack-infra06:30
AJaegertonyb: 528498  passes testsuite now06:32
*** e0ne has quit IRC06:33
*** yamamoto has joined #openstack-infra06:35
*** lukebrowning has quit IRC06:35
*** lukebrowning has joined #openstack-infra06:36
openstackgerritAndreas Jaeger proposed openstack-infra/devstack-gate master: Move commands to the base testenv  https://review.openstack.org/52881906:38
openstackgerritAndreas Jaeger proposed openstack-infra/devstack-gate master: Remove run-tests and py3-run-tests  https://review.openstack.org/52881706:38
*** lukebrowning has quit IRC06:41
*** yamamoto has quit IRC06:41
*** e0ne has joined #openstack-infra06:42
*** lukebrowning has joined #openstack-infra06:43
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: config: add statsd-server config parameter  https://review.openstack.org/52896906:44
*** yolanda has quit IRC06:44
*** yolanda has joined #openstack-infra06:47
*** e0ne has quit IRC06:47
*** lukebrowning has quit IRC06:48
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Git driver  https://review.openstack.org/52561406:48
*** lukebrowning has joined #openstack-infra06:49
*** edmondsw has joined #openstack-infra06:51
*** e0ne has joined #openstack-infra06:53
*** lukebrowning has quit IRC06:54
*** armaan has quit IRC06:55
*** lukebrowning has joined #openstack-infra06:55
*** armaan has joined #openstack-infra06:55
*** edmondsw has quit IRC06:55
*** e0ne has quit IRC06:58
*** markvoelker has quit IRC06:59
*** e0ne has joined #openstack-infra06:59
*** jchhatbar has joined #openstack-infra07:00
*** lukebrowning has quit IRC07:00
*** e0ne has quit IRC07:04
*** lukebrowning has joined #openstack-infra07:04
*** lukebrowning has quit IRC07:09
*** armax has joined #openstack-infra07:10
*** lukebrowning has joined #openstack-infra07:10
*** armax has quit IRC07:11
*** e0ne has joined #openstack-infra07:14
*** lukebrowning has quit IRC07:15
*** lukebrowning has joined #openstack-infra07:17
*** dsariel has quit IRC07:17
*** kjackal has joined #openstack-infra07:17
*** andreas_s has joined #openstack-infra07:18
*** salv-orlando has joined #openstack-infra07:19
*** e0ne has quit IRC07:20
*** claudiub has joined #openstack-infra07:20
*** lukebrowning has quit IRC07:21
*** lukebrowning has joined #openstack-infra07:22
*** salv-orlando has quit IRC07:23
*** lukebrowning has quit IRC07:27
*** e0ne has joined #openstack-infra07:28
*** rcernin has quit IRC07:31
*** lukebrowning has joined #openstack-infra07:32
*** e0ne has quit IRC07:33
*** e0ne has joined #openstack-infra07:35
*** lukebrowning has quit IRC07:36
*** shardy has joined #openstack-infra07:37
*** shardy has quit IRC07:37
*** lukebrowning has joined #openstack-infra07:38
*** e0ne has quit IRC07:41
*** shardy has joined #openstack-infra07:41
*** lukebrowning has quit IRC07:42
openstackgerritChandan Kumar proposed openstack-infra/project-config master: Add new repo for Blazar tempest plugin  https://review.openstack.org/52867207:43
*** yamamoto has joined #openstack-infra07:45
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement an OpenContainer driver  https://review.openstack.org/46875307:46
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a Kubernetes driver  https://review.openstack.org/52135607:46
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement an Amazon EC2 driver  https://review.openstack.org/52898207:46
*** slaweq has joined #openstack-infra07:46
*** mmethot_ has joined #openstack-infra07:49
*** lukebrowning has joined #openstack-infra07:50
*** mmethot has quit IRC07:51
*** e0ne has joined #openstack-infra07:51
*** salv-orlando has joined #openstack-infra07:54
*** e0ne has quit IRC07:55
*** lukebrowning has quit IRC07:57
*** e0ne has joined #openstack-infra07:58
*** lukebrowning has joined #openstack-infra07:59
*** rcernin has joined #openstack-infra08:02
*** d0ugal has quit IRC08:03
*** e0ne has quit IRC08:03
*** lukebrowning has quit IRC08:04
*** lukebrowning has joined #openstack-infra08:05
*** d0ugal has joined #openstack-infra08:07
*** e0ne has joined #openstack-infra08:07
*** slaweq_ has joined #openstack-infra08:09
*** HeOS has joined #openstack-infra08:09
*** ralonsoh has joined #openstack-infra08:10
*** lukebrowning has quit IRC08:10
*** cuongnv has quit IRC08:10
*** daidv has quit IRC08:10
*** daidv has joined #openstack-infra08:11
*** cuongnv has joined #openstack-infra08:11
*** lukebrowning has joined #openstack-infra08:11
*** slaweq_ has quit IRC08:14
*** e0ne has quit IRC08:16
*** lukebrowning has quit IRC08:16
*** lukebrowning has joined #openstack-infra08:18
*** xinliang has quit IRC08:22
*** alexchadin has joined #openstack-infra08:22
*** lukebrowning has quit IRC08:22
*** lukebrowning has joined #openstack-infra08:23
*** amoralej|off is now known as amoralej08:25
*** e0ne has joined #openstack-infra08:28
*** lukebrowning has quit IRC08:28
*** andreas__ has joined #openstack-infra08:30
openstackgerritmegan guiney proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation  https://review.openstack.org/52486208:30
*** hashar has joined #openstack-infra08:32
*** jamesmcarthur has joined #openstack-infra08:33
openstackgerritJens Harbott (frickler) proposed openstack-infra/openstack-zuul-jobs master: Remove neutron-dynamic-routing legacy jobs  https://review.openstack.org/52898808:33
*** lukebrowning has joined #openstack-infra08:33
openstackgerritJens Harbott (frickler) proposed openstack-infra/project-config master: Remove neutron-dynamic-routing legacy jobs  https://review.openstack.org/52898908:33
*** yolanda__ has joined #openstack-infra08:34
*** andreas_s has quit IRC08:34
*** xinliang has joined #openstack-infra08:34
*** yolanda has quit IRC08:36
*** florianf has joined #openstack-infra08:37
*** jamesmcarthur has quit IRC08:37
*** lukebrowning has quit IRC08:38
*** ccamacho has joined #openstack-infra08:39
*** lukebrowning has joined #openstack-infra08:39
*** edmondsw has joined #openstack-infra08:39
*** jtomasek has joined #openstack-infra08:40
*** lukebrowning has quit IRC08:44
*** edmondsw has quit IRC08:44
*** salv-orlando has quit IRC08:45
*** lukebrowning has joined #openstack-infra08:45
*** salv-orlando has joined #openstack-infra08:46
*** alexchadin has quit IRC08:47
*** alexchadin has joined #openstack-infra08:48
*** priteau has joined #openstack-infra08:50
*** lukebrowning has quit IRC08:50
*** jpena|off is now known as jpena08:51
*** salv-orlando has quit IRC08:51
*** lukebrowning has joined #openstack-infra08:52
*** hashar has quit IRC08:54
*** markvoelker has joined #openstack-infra08:56
*** lukebrowning has quit IRC08:57
*** lukebrowning has joined #openstack-infra08:58
*** salv-orlando has joined #openstack-infra08:58
*** hashar has joined #openstack-infra08:59
chandankumarAJaeger: hello08:59
chandankumarAJaeger: anything more to add in this review https://review.openstack.org/#/c/528672/08:59
AJaegerhi chandankumar , let me check08:59
chandankumaras per last dmsimard comment i have again fixed it08:59
*** e0ne has quit IRC09:00
AJaegerchandankumar: sorry, you misunderstood my comment - and we fixed the problem properly, so we're back to where we started with the two changes09:00
*** andreas_s has joined #openstack-infra09:00
chandankumarAJaeger: cool, thanks :-)09:01
AJaegerchandankumar: so, change is fine, +2. Anything else open from you that I can check or clarify?09:02
chandankumarAJaeger: both are done from your side :-)09:02
*** lukebrowning has quit IRC09:03
chandankumari hope blazar would be the last one from tempest plugin goal repo creation09:03
AJaegeryeah ;)09:04
*** andreas__ has quit IRC09:04
AJaegerchandankumar: thanks a lot for driving this!09:04
*** lukebrowning has joined #openstack-infra09:05
*** lukebrowning has quit IRC09:09
openstackgerritJens Harbott (frickler) proposed openstack-infra/devstack-gate master: Add neutron-tempest-plugin-api job  https://review.openstack.org/52900009:09
*** lukebrowning has joined #openstack-infra09:11
*** lukebrowning has quit IRC09:16
*** lukebrowning has joined #openstack-infra09:17
openstackgerritJavier Peña proposed openstack-infra/puppet-openstackci master: Add AFS mirror support  https://review.openstack.org/52873909:23
*** lukebrowning has quit IRC09:28
*** lukebrowning has joined #openstack-infra09:29
*** markvoelker has quit IRC09:29
*** jpich has joined #openstack-infra09:29
AJaegerfrickler: could you review https://review.openstack.org/#/c/521557/ , please? We now have updated the check/gate job and should update the post job the same way...09:32
*** lucas-afk is now known as lucasagomes09:32
AJaegermordred: ^09:32
*** armaan has quit IRC09:33
*** armaan has joined #openstack-infra09:34
*** lukebrowning has quit IRC09:34
*** lukebrowning has joined #openstack-infra09:35
*** derekh has joined #openstack-infra09:36
*** lukebrowning has quit IRC09:40
*** lukebrowning has joined #openstack-infra09:42
*** armaan has quit IRC09:45
*** armaan has joined #openstack-infra09:46
*** lukebrowning has quit IRC09:46
*** lukebrowning has joined #openstack-infra09:48
*** wolverineav has joined #openstack-infra09:50
*** jascott1 has joined #openstack-infra09:52
*** lukebrowning has quit IRC09:53
*** liujiong has quit IRC09:53
*** lukebrowning has joined #openstack-infra09:54
*** wolverineav has quit IRC09:55
*** jascott1 has quit IRC09:57
*** calbers has quit IRC09:57
openstackgerritTristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: web: add swagger api documentation  https://review.openstack.org/52901309:58
*** lukebrowning has quit IRC09:59
*** Shrews has quit IRC10:00
*** afazekas has quit IRC10:01
*** namnh has quit IRC10:01
*** kiennt26 has quit IRC10:02
*** calbers has joined #openstack-infra10:03
fricklerAJaeger: is there a test for https://review.openstack.org/521557 somewhere? otherwise, how about adding the new job first and verifying it? I'd really like to stop breaking things every other day10:04
*** lukebrowning has joined #openstack-infra10:05
*** afazekas has joined #openstack-infra10:07
*** lukebrowning has quit IRC10:09
*** cuongnv has quit IRC10:09
*** Shrews has joined #openstack-infra10:10
*** bhavik has joined #openstack-infra10:10
AJaegerfrickler: no test - and it runs as post job ;(10:12
AJaegerfrickler: this should mirror what we do in build-sphinx-docs since yesterday - and is needed due to the changes we merged yesterday10:13
AJaegerfrickler: we can wait with merging until mordred is around in case of problems - so, a +2 alone without the +W.10:14
*** psachin has quit IRC10:24
*** bhavik has quit IRC10:26
*** markvoelker has joined #openstack-infra10:26
*** edmondsw has joined #openstack-infra10:27
*** lukebrowning has joined #openstack-infra10:29
fricklerAJaeger: I'm not convinced yet. what prevents you from changing the post job only for a single project and test it there?10:30
*** e0ne has joined #openstack-infra10:31
*** edmondsw has quit IRC10:32
*** lukebrowning has quit IRC10:34
*** lukebrowning has joined #openstack-infra10:36
gcbAJaeger: about comments in https://review.openstack.org/#/c/528970/  should we adjust the proposal change in http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html ?10:37
*** jtomasek has quit IRC10:38
*** sambetts|afk is now known as sambetts10:39
*** jtomasek has joined #openstack-infra10:39
*** jpich has quit IRC10:40
*** lukebrowning has quit IRC10:41
*** jchhatbar is now known as janki10:41
*** niska has quit IRC10:42
*** lukebrowning has joined #openstack-infra10:42
*** stakeda has quit IRC10:44
AJaegergcb: yes, we should - let's ask mordred once he's awake ^10:46
*** electrofelix has joined #openstack-infra10:46
AJaegergcb: just pushed a few changes to remove tox_install.sh from remaining oslo projects, those are needed as well10:46
AJaegergcb, see topic:rm-tox_install10:46
*** lukebrowning has quit IRC10:47
gcbAJaeger: thanks  will look at them10:47
*** lukebrowning has joined #openstack-infra10:48
AJaegerfrickler: let's talk with mordred later...10:49
*** niska has joined #openstack-infra10:52
*** lukebrowning has quit IRC10:53
*** yamamoto has quit IRC10:55
*** lukebrowning has joined #openstack-infra10:55
*** yamamoto has joined #openstack-infra10:55
*** yamamoto has quit IRC10:56
*** yamamoto has joined #openstack-infra10:56
*** lukebrowning has quit IRC10:59
*** markvoelker has quit IRC11:00
ttxI'm trying to fix the governance doc builds -- I can't really reproduce errors locally though.11:00
ttxRunning tox -evenv -- sphinx-build -W -b html doc/source doc/build/html11:01
ttxThings just work11:01
ttxIs there a better way to simulate the new thing ?11:01
AJaegerttx, governance needs python3.11:01
AJaegerbut our new job by default uses python2. Let me check with mordred how to fix this best later today...11:02
ttxAJaeger: ok11:02
AJaegergcb: better wait with removing the requirements.txt line, I'll wait for mordred to comment11:03
gcbAJaeger: okay11:03
AJaegergcb: i see you just updated - let's see what mordred says11:04
AJaegermordred: I updated https://etherpad.openstack.org/p/merge-updated-pti-topic with some comments11:04
AJaegerbbl11:04
ttxAJaeger: the releases repo is in the same situation.... smcginnis included some workarounds for open() calls now being run under python 2.7 and failing11:04
ttxso let's see if there is a more general way to solve this and allow the doc job to run under py311:05
*** lukebrowning has joined #openstack-infra11:05
*** salv-orlando has quit IRC11:09
*** salv-orlando has joined #openstack-infra11:09
*** yamamoto has quit IRC11:09
openstackgerritJavier Peña proposed openstack-infra/system-config master: Move AFS mirror code to puppet-openstackci  https://review.openstack.org/52903211:10
*** lukebrowning has quit IRC11:10
*** andreas_s has quit IRC11:10
*** andreas_s has joined #openstack-infra11:11
*** lukebrowning has joined #openstack-infra11:11
*** yamamoto has joined #openstack-infra11:13
*** salv-orlando has quit IRC11:14
*** lukebrowning has quit IRC11:17
*** yamamoto has quit IRC11:18
*** bhavik has joined #openstack-infra11:19
*** ldnunes has joined #openstack-infra11:22
*** pgadiya has quit IRC11:22
*** andreas_s has quit IRC11:24
*** andreas_s has joined #openstack-infra11:26
*** armaan has quit IRC11:28
*** armaan has joined #openstack-infra11:28
*** jtomasek has quit IRC11:29
*** andreas_s has quit IRC11:30
*** jamesmcarthur has joined #openstack-infra11:33
*** armaan has quit IRC11:33
*** armaan has joined #openstack-infra11:33
*** jamesmcarthur has quit IRC11:38
*** lukebrowning has joined #openstack-infra11:39
*** andreas_s has joined #openstack-infra11:40
*** e0ne has quit IRC11:41
*** lukebrowning has quit IRC11:43
*** alexchadin has quit IRC11:44
*** armaan has quit IRC11:45
*** lukebrowning has joined #openstack-infra11:45
*** pbourke has joined #openstack-infra11:47
*** andreas_s has quit IRC11:50
*** yolanda__ is now known as yolanda11:50
*** lukebrowning has quit IRC11:50
*** andreas_s has joined #openstack-infra11:51
AJaegerttx: there is - mordred did the ground work already. I just wonder how to configure in the best way. So, let's wait for him and discuss later today...11:51
*** e0ne has joined #openstack-infra11:51
*** lukebrowning has joined #openstack-infra11:51
*** andreas_s has quit IRC11:53
*** andreas_s has joined #openstack-infra11:53
*** lukebrowning has quit IRC11:56
*** smatzek has joined #openstack-infra11:56
*** markvoelker has joined #openstack-infra11:57
*** lukebrowning has joined #openstack-infra11:58
*** bhavik has quit IRC12:02
*** lukebrowning has quit IRC12:03
*** lukebrowning has joined #openstack-infra12:06
*** yamamoto has joined #openstack-infra12:06
*** alexchadin has joined #openstack-infra12:06
*** salv-orlando has joined #openstack-infra12:10
*** yamamoto has quit IRC12:10
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Small refactor of translation scripts  https://review.openstack.org/52904712:14
*** salv-orlando has quit IRC12:14
*** lukebrowning has quit IRC12:15
*** edmondsw has joined #openstack-infra12:16
*** annp has quit IRC12:17
*** rkukura has quit IRC12:18
*** edmondsw has quit IRC12:20
*** e0ne has quit IRC12:22
*** links has quit IRC12:23
*** katkapilatova has joined #openstack-infra12:27
*** amoralej is now known as amoralej|off12:29
*** markvoelker has quit IRC12:29
*** yamamoto has joined #openstack-infra12:38
*** yamamoto has quit IRC12:39
*** lucasagomes is now known as lucas-hungry12:42
*** alexchadin has quit IRC12:46
*** jpena is now known as jpena|lunch12:48
*** weshay_pto is now known as weshay12:48
*** hichihara has joined #openstack-infra12:50
*** aarefiev has joined #openstack-infra12:50
*** bobh has joined #openstack-infra12:52
*** bobh has quit IRC12:53
*** ralonsoh has quit IRC12:53
*** jascott1 has joined #openstack-infra12:54
*** claudiub|2 has joined #openstack-infra12:57
*** jascott1 has quit IRC12:59
*** zhurong has joined #openstack-infra13:00
*** claudiub has quit IRC13:00
*** zhurong has quit IRC13:02
*** wolverineav has joined #openstack-infra13:02
*** zhurong has joined #openstack-infra13:03
*** claudiub has joined #openstack-infra13:04
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Do not exit early tenant configuration in case of config issue  https://review.openstack.org/52906013:04
*** jcoufal has joined #openstack-infra13:06
*** claudiub|2 has quit IRC13:07
*** salv-orlando has joined #openstack-infra13:11
*** janki has quit IRC13:11
*** janki has joined #openstack-infra13:11
*** catintheroof has joined #openstack-infra13:11
*** catintheroof has quit IRC13:12
*** catintheroof has joined #openstack-infra13:12
*** lukebrowning has joined #openstack-infra13:13
*** d0ugal has quit IRC13:14
*** salv-orlando has quit IRC13:15
*** rossella_s has joined #openstack-infra13:18
*** lukebrowning has quit IRC13:18
*** links has joined #openstack-infra13:18
*** r-daneel has joined #openstack-infra13:19
*** rcernin has quit IRC13:20
*** lukebrowning has joined #openstack-infra13:20
*** r-daneel has quit IRC13:20
*** d0ugal has joined #openstack-infra13:23
*** lukebrowning has quit IRC13:24
*** zhurong has quit IRC13:25
*** dprince has joined #openstack-infra13:25
*** hashar has quit IRC13:25
*** hashar has joined #openstack-infra13:26
*** lukebrowning has joined #openstack-infra13:26
*** markvoelker has joined #openstack-infra13:27
*** markvoelker has quit IRC13:28
*** fultonj has quit IRC13:28
*** rlandy has joined #openstack-infra13:29
*** markvoelker has joined #openstack-infra13:29
*** rlandy is now known as rlandy|rover13:29
*** dsariel has joined #openstack-infra13:30
*** lukebrowning has quit IRC13:31
*** coolsvap has quit IRC13:32
*** lukebrowning has joined #openstack-infra13:32
*** armaan has joined #openstack-infra13:34
*** r-daneel has joined #openstack-infra13:34
*** lucas-hungry is now known as lucasagomes13:34
*** trown|outtypewww is now known as trown|ruck13:36
*** rkukura has joined #openstack-infra13:37
andreafAJaeger, frickler: could you have a look at if you have a sec? https://review.openstack.org/#/c/528748/ - it's a very simple change and a long list of approved devstack patches depends on it :)13:37
*** d0ugal has quit IRC13:37
*** lukebrowning has quit IRC13:37
*** lukebrowning has joined #openstack-infra13:39
*** ralonsoh has joined #openstack-infra13:39
*** fultonj has joined #openstack-infra13:39
*** d0ugal has joined #openstack-infra13:39
*** yamamoto has joined #openstack-infra13:40
fricklerandreaf: done13:43
*** lukebrowning has quit IRC13:43
*** lukebrowning has joined #openstack-infra13:44
*** armaan has quit IRC13:45
*** dave-mccowan has joined #openstack-infra13:45
*** kgiusti has joined #openstack-infra13:46
*** mihalis68 has joined #openstack-infra13:48
*** yamamoto has quit IRC13:48
*** ykarel has quit IRC13:48
*** jpena|lunch is now known as jpena13:49
*** lukebrowning has quit IRC13:50
openstackgerritMerged openstack-infra/devstack-gate master: Import tox jobs  https://review.openstack.org/52855313:50
*** rkukura has quit IRC13:50
*** lukebrowning has joined #openstack-infra13:51
*** makowals has joined #openstack-infra13:53
*** dsariel has quit IRC13:53
*** annp has joined #openstack-infra13:54
*** lukebrowning has quit IRC13:55
*** lukebrowning has joined #openstack-infra13:57
openstackgerritMerged openstack-infra/zuul-jobs master: Dereference symlinks when staging artefacts  https://review.openstack.org/52874813:58
*** mriedem has joined #openstack-infra13:58
*** bh526r has joined #openstack-infra13:58
*** lukebrowning has quit IRC14:01
*** rkukura has joined #openstack-infra14:03
*** links has quit IRC14:03
*** dhill_ has joined #openstack-infra14:06
*** salv-orlando has joined #openstack-infra14:11
openstackgerritAkihiro Motoki proposed openstack-infra/project-config master: Drop django_openstack_auth master branch jobs  https://review.openstack.org/52908014:13
*** katkapilatova has quit IRC14:14
*** salv-orlando has quit IRC14:16
*** links has joined #openstack-infra14:17
*** lukebrowning has joined #openstack-infra14:17
*** lukebrowning has quit IRC14:22
*** dhill_ has quit IRC14:22
andreykurilinhi folks! does anyone can explain me about writting custom roles.  I mean how can I assign the custom role to task?14:22
*** salv-orlando has joined #openstack-infra14:23
*** lukebrowning has joined #openstack-infra14:23
*** jtomasek has joined #openstack-infra14:23
*** lukebrowning has quit IRC14:28
*** dhill_ has joined #openstack-infra14:29
*** smatzek has quit IRC14:29
*** lukebrowning has joined #openstack-infra14:30
*** andreas_s has quit IRC14:30
*** andreas_s has joined #openstack-infra14:30
lbragstadclarkb: AJaeger thanks for the assist14:33
*** lukebrowning has quit IRC14:34
*** lukebrowning has joined #openstack-infra14:36
*** kiennt26 has joined #openstack-infra14:37
*** abelur_ has quit IRC14:37
*** lukebrowning has quit IRC14:41
*** lukebrowning has joined #openstack-infra14:42
andreaffrickler: cool, thanks14:43
*** andreas_s has quit IRC14:45
*** andreas_s has joined #openstack-infra14:45
*** sshnaidm has quit IRC14:46
*** gouthamr has joined #openstack-infra14:47
*** ramishra has quit IRC14:47
*** lukebrowning has quit IRC14:47
*** felipemonteiro has joined #openstack-infra14:47
*** sshnaidm has joined #openstack-infra14:48
*** andreas_s has quit IRC14:48
*** andreas_s has joined #openstack-infra14:48
*** hichihara has quit IRC14:52
*** witek has joined #openstack-infra14:54
*** lukebrowning has joined #openstack-infra14:55
*** jascott1 has joined #openstack-infra14:56
*** smatzek has joined #openstack-infra14:58
*** lukebrowning has quit IRC14:59
*** jascott1 has quit IRC15:01
*** lukebrowning has joined #openstack-infra15:01
*** andreas_s has quit IRC15:03
*** andreas_s has joined #openstack-infra15:03
*** salv-orlando has quit IRC15:04
*** salv-orlando has joined #openstack-infra15:04
*** lukebrowning has quit IRC15:06
cmurphyAJaeger: keystone doc builds are still broken, can you tell us if https://review.openstack.org/#/c/528960 is the way to go? basically our setup.cfg [extras] aren't getting installed15:07
*** lukebrowning has joined #openstack-infra15:08
*** efoley has joined #openstack-infra15:08
*** salv-orlando has quit IRC15:09
AJaegercmurphy: just started loooking at that patch ;) give me a few mins15:09
cmurphy:)15:10
*** marst has joined #openstack-infra15:10
*** kiennt26 has quit IRC15:11
cmurphyAJaeger: recent failure is https://review.openstack.org/#/c/528866/ btw15:11
*** kiennt26 has joined #openstack-infra15:11
*** andreas_s has quit IRC15:12
AJaegermordred: master debugger of docs related odditities, are you already around?15:12
*** lukebrowning has quit IRC15:12
AJaegerchange 528960 just move all the extras from setup.cfg into doc/requirements.txt so that the build actually works. Quite a hack ;( But it generates the documentation ;)15:13
*** andreas_s has joined #openstack-infra15:13
AJaegercmurphy: I agree, tox.ini should use the previous python.py setup.py build_sphinx15:14
*** lukebrowning has joined #openstack-infra15:14
AJaegercmurphy: let me update 528960 with that single line change and wait for mordred on whether he has a better idea. This works but I'm unhappy ;(15:15
*** salv-orlando has joined #openstack-infra15:15
*** gmmaha has left #openstack-infra15:16
cmurphyAJaeger: cool :)15:16
AJaegercmurphy: pushed the change - it will take some time until it passes, I suggest we hear whether mordred has a better idea - but if he's not reacting, go ahead and merge to unblock keystone and we can revert/adjust later.15:18
andreafAJaeger, frickler: any idea how to trigger the gate on https://review.openstack.org/#/c/527106? The parent and depends-on patches are merged but nothing happens15:18
cmurphyAJaeger: alright thank you15:19
*** lukebrowning has quit IRC15:19
AJaegerandreaf: checking...15:19
AJaegerandreaf: a recheck should be enough - or another +A. You're core there, aren't you? Just +A to generate the event15:20
*** aarefiev has quit IRC15:20
*** lukebrowning has joined #openstack-infra15:20
*** armax has joined #openstack-infra15:20
openstackgerritMatthew Treinish proposed openstack-infra/devstack-gate master: Handle a stestr results directory  https://review.openstack.org/52364615:21
andreafAJaeger ok thx15:21
*** andreas_s has quit IRC15:23
AJaegerandreaf: the two repos - the dependent one and the one waiting - do not share a queue. And then there's no notification when dependent job merged...15:24
AJaegerandreaf: a recheck would work as well, but then you would need to wait for the job to run through check again, so the extra +A triggers the gate here15:24
*** lukebrowning has quit IRC15:25
AJaegerconfig-core, could you review https://review.openstack.org/#/c/528518 and https://review.openstack.org/#/c/529047/1 for translations, please? Fixes one specific cornercase (only one repo has the situation).15:25
*** bobh has joined #openstack-infra15:26
*** lukebrowning has joined #openstack-infra15:26
*** abelur_ has joined #openstack-infra15:27
*** andreas_s has joined #openstack-infra15:27
*** kiennt26 has quit IRC15:27
*** rbrndt has joined #openstack-infra15:28
*** felipemonteiro has quit IRC15:29
*** felipemonteiro has joined #openstack-infra15:30
*** lukebrowning has quit IRC15:32
*** lukebrowning has joined #openstack-infra15:33
*** andreas_s has quit IRC15:34
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Create project-templates for sphinx building with python3  https://review.openstack.org/52909315:34
jeblairandreykurilin: are you asking about ansible roles?15:34
*** andreas_s has joined #openstack-infra15:34
andreykurilinjeblair: yes. I already tried guessing, but it looks like wasting compute resources without any progress. I have a patch - https://review.openstack.org/#/c/524637 . It adds one role to existing job15:36
*** ykarel has joined #openstack-infra15:37
AJaegerttx, for releases repo, do something similar to 52909315:37
AJaegerttx, or reuse the template15:37
*** lukebrowning has quit IRC15:38
*** eharney has joined #openstack-infra15:38
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Create project-templates for sphinx building with python3  https://review.openstack.org/52909315:39
jeblairandreykurilin: looks like you've got it running -- what's the problem?15:39
andreykurilinjeblair: I have post_failure status15:40
* andreykurilin jeblair: http://logs.openstack.org/37/524637/6/check/rally-install-ubuntu-xenial/f4aff39/15:40
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Use python3 sphinx building for governance  https://review.openstack.org/52909515:41
andreykurilinalso, there is nothing at ara15:41
AJaegerttx, and 529095 fixes governance ^15:42
jeblairandreykurilin: hrm, i agree, there should be more output in that log telling you what went wrong (cc mordred)15:42
jeblairandreykurilin: we have some longstanding bugs about ansible error output not making it into the logs15:43
jeblairandreykurilin: i'll see if i can find something on the executor15:43
andreykurilinjeblair: and I expect to see custom index.html at the top dir of logs (we did it before in legacy jobs), but there is nothing, so it means that my role is wrong or it was not executed correctly15:43
andreykurilinjeblair: thanks15:43
*** lukebrowning has joined #openstack-infra15:44
*** liusheng has quit IRC15:45
*** liusheng has joined #openstack-infra15:45
*** lukebrowning has quit IRC15:48
*** lukebrowning has joined #openstack-infra15:50
ttxAJaeger: looks like it fails due to a recursion in job definition in https://review.openstack.org/#/c/529093/2 ?15:50
*** jtomasek has quit IRC15:50
ttxsmcginnis: see for fixing the releases repo ^15:51
*** edmondsw has joined #openstack-infra15:52
*** salv-orlando has quit IRC15:52
*** salv-orlando has joined #openstack-infra15:53
smcginnisWatching15:53
ttxsmcginnis: probably needing something like https://review.openstack.org/#/c/529095/15:53
ttxBut 529093 needs to be fixed first15:53
*** jamesmcarthur has joined #openstack-infra15:54
*** lukebrowning has quit IRC15:55
jeblairandreykurilin: this is the error: http://paste.openstack.org/show/629322/15:55
jeblairandreykurilin: the problem is subtle15:55
jeblairandreykurilin: if you look at the first line, it has: ...ansible/post_playbook_0/role_0:15:55
jeblairandreykurilin: followed by: ansible/post_playbook_0/role_1/project-config/roles:15:56
*** edmondsw has quit IRC15:56
jeblairandreykurilin: the second element (role_1) has a project name in it15:56
*** lukebrowning has joined #openstack-infra15:56
jeblairandreykurilin: the first element, which is the element in the path for this job (it orders them backwards with this repo first, and others later), has no project name15:57
*** salv-orlando has quit IRC15:57
*** xarses_ has joined #openstack-infra15:57
jeblairandreykurilin: that means that zuul thought this repo was a "bare role" -- explained at https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.roles15:57
jeblairandreykurilin: it thought that because there is a tasks/ directory at the root of the repo15:57
andreykurilinjeblair: hm... based on the first like (PATH), I can put the roles in openstack/rally/tests/ci/playbooks/base_new/roles and it should become discoverable. right?15:59
jeblairandreykurilin: in other words, this repo is not structured to be an ansible shared role15:59
jeblairandreykurilin: yes15:59
jeblairandreykurilin: in that case, the role would only be able to be used by that playbook15:59
andreykurilinjeblair: ok. so `zuul:openstack/rally` become redundant16:00
jeblairandreykurilin: yes -- it's actually redundant either way -- the job's repo is always automatically added as a role16:00
jeblairandreykurilin: (it's harmless to have it)16:00
jeblair(the last pgraph of that doc describes that)16:01
*** lukebrowning has quit IRC16:01
andreykurilinI read the docs, but it looks like I missed this.16:01
jeblairthey are... dense.16:01
andreykurilinjeblair: one more question. if such role will be used by a job and some other project will try to use such job, what will happen?16:02
jeblairandreykurilin: should be fine16:02
*** lukebrowning has joined #openstack-infra16:02
andreykurilinjeblair: cool, thanks16:03
openstackgerritThierry Carrez proposed openstack-infra/openstack-zuul-jobs master: Create project-templates for sphinx building with python3  https://review.openstack.org/52909316:03
AJaegerttx, yeah, that's the usual dance with project-config - we need to merge one change, recheck others, merge, recheck...16:04
ttxAh/16:04
andreykurilinjeblair: the last question, I hope. Does it possible to write something "abort check" to stop all the jobs for a patch?16:05
ttxI thought you added -python3 to the publish job erroneously and pushed a new patchset... please check16:05
ttxAJaeger: https://review.openstack.org/#/c/529093/16:05
jeblairandreykurilin: not at the moment16:05
jeblairandreykurilin: a new patchset will abort them (but of course, start them again with the new patchset)16:06
AJaegerttx, your change is wrong - the depends-on defines that job. and since project-config is special, Zuul does not know about it...16:06
ttxdamn16:06
ttxI'll revert16:06
jeblairandreykurilin: if all your jobs are defined in the zuul.yaml file in the repo you're changing, you can temporarily remove jobs you aren't interested in while you work on a change16:06
AJaegerttx, so the change will pass with that one - but then once the project-config change is in, we have to do change it again...16:07
andreykurilinjeblair: :( ok. hope that in near future, we will port all the jobs to be hosted in our repo, so it will be possible to specify what jobs to launch while experimenting.16:07
jeblairandreykurilin: yeah, once you get to that point, experimenting becomes a lot easier16:07
andreykurilinjeblair: and less expensive :)16:07
*** lukebrowning has quit IRC16:07
AJaegerttx, so if we don't merge https://review.openstack.org/#/c/521557/ quickly, I would use your variant - better a half-working solution (publishing broken but check/gate working) than everything broken as is now.16:07
*** apetrich has joined #openstack-infra16:08
*** lukebrowning has joined #openstack-infra16:09
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add an authoritative hidden master  https://review.openstack.org/52843816:11
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884616:11
*** ykarel has quit IRC16:11
ttxAJaeger: so should I revert it?16:12
AJaegerttx, I still have it locally here - let me push...16:12
ttxok! That other patch looks close enough16:13
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Create project-templates for sphinx building with python3  https://review.openstack.org/52909316:13
*** lukebrowning has quit IRC16:13
AJaegerttx, want me to do a change for releases repo?16:14
*** lukebrowning has joined #openstack-infra16:15
ttxAJaeger: if you're on a roll, please do. Otherwise I expect smcginnis to catch it (he has another change in flight he needs to abandon)16:17
clarkblbragstad: AJaeger mordred http://logs.openstack.org/68/512468/6/check/build-openstack-sphinx-docs/e724ee3/ara/ still not happy, in this case looks like it wants some ldap library for autodocing that isn't installed16:17
clarkbmay need to add a dependency?16:17
*** hashar is now known as hasharAway16:18
lbragstadclarkb: https://review.openstack.org/#/c/528960/516:18
AJaegerclarkb: as lbragstad says. While I'm not happy with it, I have no other solution. Waiting for mordred to comment16:19
AJaegerclarkb: it's all this [extra] referencing setup.cfg16:19
AJaegerttx, ok, will fix16:20
*** lukebrowning has quit IRC16:20
clarkbAJaeger: lbragstad was the dep missing from the normal project requirements?16:20
lbragstadi think it's optional16:21
lbragstadfrom a keystone perspective16:21
*** lukebrowning has joined #openstack-infra16:21
AJaegerlbragstad: I wonder why it's optional from keystone but required for docs building...16:22
AJaegerlbragstad: I know you can do this, just looks odd16:22
lbragstadwe don't have a hard dependency on it since keystone can be run without ldap16:24
AJaegerfrickler: rereading 521557 again: It does not use the new sphinx role everywhere and just adds this for a few specific repos as you suggested. 525734 is intended as the change to switch the rest.16:24
*** salv-orlando has joined #openstack-infra16:25
*** andreas_s has quit IRC16:26
AJaegerttx, https://review.openstack.org/52910416:26
*** jamesmcarthur has quit IRC16:26
*** lukebrowning has quit IRC16:26
*** andreas_s has joined #openstack-infra16:26
*** jamesmcarthur has joined #openstack-infra16:27
ttxAJaeger: should we just do a similar change in governance ? i.e. using zuul.d ?16:28
*** lukebrowning has joined #openstack-infra16:28
AJaegerttx, 529095 fixes governance16:29
ttxAJaeger: ok, was just wondering if that would avoid the dependency and allow to quickfix16:30
ttx(to do a local .zuul.yaml change)16:31
*** janki has quit IRC16:31
*** jamesmcarthur has quit IRC16:31
ttxavoids using the template and all16:31
AJaegerttx, yes, it would. So, you could do as hotfix16:32
ttxAJaeger: ok thanks. I'd like to unblock asap16:32
*** lukebrowning has quit IRC16:32
AJaegerttx but publishing will not work this way!16:32
ttxAh! again16:32
ttxI'm confused16:32
smcginnisttx, AJaeger: Looks like for releases we will need to get sphinxocntrib.datatemplates added to g-r first, then get that landed in releases to include it in doc/requirements.txt.16:33
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add an authoritative hidden master  https://review.openstack.org/52843816:33
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884616:33
smcginnisI will un-squash that change with the io.open one.16:33
openstackgerritJens Harbott (frickler) proposed openstack-infra/devstack-gate master: Add neutron-tempest-plugin-api job  https://review.openstack.org/52900016:33
ttxOh well, probably too late for me to fix this today16:33
AJaegersmcginnis: argh ;(16:33
ttxI'll wait until those changes make it to project-config16:33
*** lukebrowning has joined #openstack-infra16:34
AJaegerttx, I'll advocate to push this through today.16:34
*** armaan has joined #openstack-infra16:37
AJaegerjeblair: for translations, I want to change the templates. Can I have two post jobs like in http://paste.openstack.org/show/629326/ - I might then add translation-jobs-master to all translated repos; and have translation-jobs-ocata to some added. So, that we run the push job only for master and stable/ocata (if both templates are used).16:37
AJaegerclarkb: that's basically implementing your suggestion ^16:37
*** r-daneel has quit IRC16:38
*** lukebrowning has quit IRC16:39
AJaegerclarkb: and will not use a single line of change to project-config and how we do branching ;)16:39
*** andreas_s has quit IRC16:40
*** jamesmcarthur has joined #openstack-infra16:41
clarkbAJaeger: went with different josb instead?16:41
*** gyee has joined #openstack-infra16:42
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Rework translation templates  https://review.openstack.org/52911116:43
AJaegerclarkb: different *templates* ;) Please review ^16:43
fungido we actually enforce requirements matching on doc/requirements.txt?16:43
fungiwondering why sphinxcontrib.datatemplates would need to go in global reqs16:43
AJaegerfungi: we currently do - I was too quick ;( Some discussed here to skip that one...16:43
*** annp has quit IRC16:44
AJaegerfungi, what we might want to do: sync but not enforce. That's a bit more work for the scripts...16:44
AJaegerfungi: meaning: I don't think they can handle this case today, so needs extending of scripts.16:45
smcginnisSeems like it would make sense to skip docs requirements.16:45
fungii guess for the specific purpose of ensuring runtime compatibility between projects we could scale back to only top-level requirements.txt matching global-reqs even, though anything not in the transitive global requirements set also misses out on constraints pinning16:45
clarkbfungi: wouldn't we want to enforce those requirements for all the of the same reasons we enforce them otherwise?16:45
clarkb(ensures that packagers know what to package and don't have to package a bunch of redundant/duplicate functionality, license checks, and so on)16:45
fungiclarkb: depends on whether we consider those reasons to only be important for run-time dependencies16:46
clarkbdocs somewhat complicate that as html/js/css are runtime in many cases16:47
jeblairAJaeger: i think your proposed change would work.  you could also move the use of the translation-jobs template in-repo, and only put it on the branches you want.16:47
clarkbso even if we said its only run time deps we wouldn't be avoided that in docs builds16:48
*** lukebrowning has joined #openstack-infra16:48
clarkb*avoiding16:48
AJaegerjeblair: I don't want that yet - the translators tranlate only master and last branch, so having a central place to  disable/enable branches is important16:49
AJaegerjeblair: could you review then how I did this in https://review.openstack.org/529111, please?16:49
*** lukebrowning has quit IRC16:52
*** andreas_s has joined #openstack-infra16:53
*** lukebrowning has joined #openstack-infra16:54
*** lucasagomes is now known as lucas-afk16:56
*** lukebrowning has quit IRC16:59
* mordred waves to everyone16:59
AJaegermordred: do you have time to chat?16:59
AJaegermordred: https://etherpad.openstack.org/p/merge-updated-pti-topic is updated, we need https://review.openstack.org/#/c/521557 to go in for post jobs, then https://review.openstack.org/529093 and  https://review.openstack.org/529095 for repos with python3 for sphinx building. Could you review the later two - and should we ask for merging of 521557?17:01
*** salv-orlando has quit IRC17:01
*** salv-orl_ has joined #openstack-infra17:01
fricklerjust noticed that eavesdrop is posting unexpected links: http://eavesdrop01.openstack.org/meetings/neutron_ci/2017/neutron_ci.2017-12-19-16.01.html17:02
fricklershouldn't that be eavesdrop.o.o?17:02
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add an authoritative hidden master  https://review.openstack.org/52843817:02
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884617:02
*** andreas_s has quit IRC17:03
*** Goneri has joined #openstack-infra17:03
AJaegerfrickler: I agree, it should.17:04
*** iyamahat_ has quit IRC17:04
AJaegerfrickler: links work but once we setup eavesdrop02, it might not...17:04
*** jascott1 has joined #openstack-infra17:05
mordredAJaeger: yes - I think https://review.openstack.org/#/c/521557 is ready to go in17:06
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Attempt to improve tenant configuration in case of config issue  https://review.openstack.org/52906017:06
*** r-daneel has joined #openstack-infra17:07
*** slaweq has quit IRC17:08
*** bh526r has quit IRC17:08
jeblairinfra-root: frickler's issue above is an important error we should fix asap17:09
AJaegerinfra-root, can we merge 521557 , please? This is one step in bringing post jobs and check/gate together17:09
mordredAJaeger: the other two lgtm17:09
clarkbjeblair: frickler its just from the irc bot side right? apache isn't redirecting you from eavesdrop.o. to eavesdrop01.o.o is it?17:10
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Set eavesdrop vhost  https://review.openstack.org/52911817:10
jeblairinfra-root: ^ i think that will fix future occurances.  we will need to edit files on disk to correct content generated with wrong links.17:11
jeblairif there is any -- i don't know17:11
jeblairit's possible this only affects what the bot says in channel?17:11
*** ccamacho has left #openstack-infra17:11
clarkblooks like there is maybe one link written to disk that is affected, the link back to the full logs17:12
jeblairclarkb: oh i think that's relative17:12
AJaegermordred: thanks. can you help pushing those in? I'll be offline now for an hour or two17:12
mordredAJaeger: yup17:13
mordredjeblair: +A17:13
*** jascott1 has quit IRC17:13
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Attempt to improve tenant configuration in case of config issue  https://review.openstack.org/52906017:14
fungido we need someone to check the files on disk for incorrect hyperlinks and fix them? i can take care of that once the fix is in production17:15
openstackgerritFabien Boucher proposed openstack-infra/zuul feature/zuulv3: Attempt to improve tenant config loading in case of config issue  https://review.openstack.org/52906017:17
openstackgerritMerged openstack-infra/elastic-recheck master: Track tripleo timeout or canceled deployments  https://review.openstack.org/51352417:17
clarkbfungi: I'm not finding any cases of that through my web browser17:18
clarkbI think it is just what the bot says in channel17:18
clarkbso the meeting logs themselves may have the wrong links in the log17:18
fungicool. i'm doing a recursive grep just in case17:19
fricklerjeblair: does your patch change /var/lib/meetbot/openstack.org/ircmeeting/meetingLocalConfig.py ? cause that contains the bad logUrlPrefix17:20
fungiclarkb: bad news... http://eavesdrop.openstack.org/meetings/neutron_ci/2017/neutron_ci.2017-12-12-16.00.txt17:21
clarkbfrickler: yes https://git.openstack.org/cgit/openstack-infra/puppet-meetbot/tree/templates/meetingLocalConfig.py.erb17:21
fungier, wrong link17:21
clarkbfungi: ok I was confused for a second17:22
mordredfungi: I was about to say - that seemed like good news :)17:22
fungihttp://eavesdrop.openstack.org/meetings/neutron_ci/2017/neutron_ci.2017-12-19-16.01.txt is the one i meant :(17:22
*** jascott1 has joined #openstack-infra17:22
*** trown|ruck is now known as trown|lunch17:22
fungii can work on categorizing places where it leaked in and stream edit the affected files17:22
*** dsariel has joined #openstack-infra17:22
fungibut that's the first one that jumped out at me in my grep17:23
clarkbah ok its the same message printed in irc that is printed in those logs17:23
mordredfrickler, clarkb, fungi, jeblair: when you have a sec, could you also land https://review.openstack.org/#/c/521557/7 - we need it to unbreak governance repo builds17:23
*** andreas_s has joined #openstack-infra17:26
*** ralonsoh has quit IRC17:30
pabelangermordred: when do you expect the next shade release?17:32
openstackgerritAlex Schultz proposed openstack-infra/elastic-recheck master: Query for 1719469  https://review.openstack.org/50729417:32
mordredpabelanger: I need to land one revert patch (fixing merge conflict right now) then will cut the release17:32
pabelangermordred: ack, thanks17:33
mordredpabelanger: while I've got you, could I get you to review https://review.openstack.org/#/c/521557/7 please?17:33
pabelangerlooking17:33
*** lukebrowning has joined #openstack-infra17:34
*** jamesmcarthur has quit IRC17:34
pabelangermordred: +317:35
*** jamesmcarthur has joined #openstack-infra17:35
mordredpabelanger: thank you17:35
clarkbmordred: pabelanger we need to restart nodepool launchers to pick up my bug fix and the quota stuff17:35
clarkbdo we want to lump shade into that too? or should we restart now then do shade after things settle in?17:35
mordredclarkb: yah - let's lump the shade release into that too17:36
clarkbok17:36
pabelanger++17:36
pabelangermy afteroon is clear so able to support17:36
mordredawesome17:36
clarkbalright, I'm likely to be afk ish this afternoon to help smoke salmon17:36
pabelangerGoing to also look at getting kdc03 online (new master) and see what is need to migrate kdc01 to it17:36
clarkbbut will have laptop too17:36
* fungi wants smoked salmon now17:37
fungii candied and smoked salmon cheeks here a few months ago and they came out amazingly17:38
fungier, no, it was salmon bellies17:39
*** jamesmcarthur has quit IRC17:39
*** lukebrowning has quit IRC17:39
mordredfungi: mmm. salmon bellies17:40
*** edmondsw has joined #openstack-infra17:40
clarkbthis is a freezer full of the filets that didn't get eaten after fishing trips. I expect a bunch of it will be given away at family gatherings over the next week or so17:40
mordrednice17:40
clarkbProblem is I'm so sick of salmon now :)17:40
fungiyou do get a lot more of them there than we do on this end of the rock17:41
mordredclarkb: I'll trade you - I've got plenty of beef in these parts17:41
fungibut atlantic salmon are still pretty awesome17:41
openstackgerritMerged openstack-infra/project-config master: Use the new sphinx PTI for special publish jobs  https://review.openstack.org/52155717:42
*** andreas_s has quit IRC17:43
*** Apoorva has joined #openstack-infra17:43
*** Apoorva has quit IRC17:43
*** Apoorva has joined #openstack-infra17:44
andreykurilinfolks. what is the right way to copy some file <project_src>/foo.file to the results of jobs? I'm trying to use copy task with `src: "{{  zuul.executor.src_root }}foo.file"` but it doesn't work. Any suggestions?17:44
*** edmondsw has quit IRC17:44
mordredandreykurilin: the file is created as part of the job?17:44
andreykurilinmordred: no. just constant file17:45
mordredah. hrm. got a link?17:45
*** lukebrowning has joined #openstack-infra17:45
andreykurilinmordred: https://review.openstack.org/#/c/524637/12/tests/ci/playbooks/base_new/roles/fetch-rally-reports/tasks/main.yaml17:46
andreykurilinmordred: https://github.com/openstack/rally/blob/master/rally/ui/templates/ci/index.html17:46
andreykurilinmordred: and the results of that role: http://logs.openstack.org/37/524637/12/check/rally-install-ubuntu-xenial/f28d3fd/ara/17:46
*** derekh has quit IRC17:47
*** aviau has quit IRC17:47
*** aviau has joined #openstack-infra17:47
mordredandreykurilin: k. so - the copy task is for copying files between computers - and that's going to be going in the wrong direction ...17:48
*** andreas_s has joined #openstack-infra17:48
mordredandreykurilin: I think instead you should maybe to synchronize: mode: pull src: "{{ zuul.project.src_dir }}/rally/ui/templates/ci/index.html" dest: "{{ zuul.executor.log_root }}/index.html"17:49
andreykurilinmordred: so it should look something like https://review.openstack.org/#/c/524637/9/tests/ci/playbooks/base_new/roles/fetch-rally-reports/tasks/main.yaml17:49
*** catintheroof has quit IRC17:50
andreykurilinit was several attempts before17:50
*** lukebrowning has quit IRC17:50
mordredandreykurilin: yes - but zuul.executor.src_root  shoujld be zuul.project.src_dir17:50
mordredandreykurilin: there is a stack of stuff we'll hopefully get landed soon that should make this easier - but for now I think if you just do it like that patch but wiht zuul.project.src_dir it should work for you17:51
*** catintheroof has joined #openstack-infra17:51
*** lukebrowning has joined #openstack-infra17:52
andreykurilinmordred: thanks. last time, it found `--ignore-missing-args` option unknown and I started to find another solution.17:52
mordredandreykurilin: weird - we use --ignore-missing-args like that in other places17:53
fungiclarkb: the good news is that the only occurrences i'm finding are in the plaintext versions of the meeting minutes, not in the html minutes nor either version of the meeting logs17:56
mnaseri have some interesting questions to the folks who previously operated osic, does anyone in infra know who i can throw some interesting questions about the deployment?17:56
fungiwell, also the channel logs, which are probably not worth fixing17:56
andreykurilinmordred: yes, I copied it from somewhere. unfortunately, I'm trying to resubmitte new patch-sets as soon as target job failed to reduce the cost of other jobs, so there is no posted link to results of failure with `--ignore-missing-args` :(17:56
fungimnaser: you want cloudnull17:56
*** lukebrowning has quit IRC17:56
mordredpabelanger: wanna do another one? https://review.openstack.org/#/c/52909317:57
mnaserfungi: thank you, cloudnull, i have a question in #openstack-operators that i'd be interested in hearing more from you about (or free free to pm, tia :D)17:57
*** jtomasek has joined #openstack-infra17:58
*** lukebrowning has joined #openstack-infra17:58
pabelangermordred: +317:59
*** raissa has joined #openstack-infra18:00
*** raissa has quit IRC18:01
*** r-daneel_ has joined #openstack-infra18:01
*** raissa has joined #openstack-infra18:01
*** r-daneel has quit IRC18:01
*** r-daneel_ is now known as r-daneel18:01
*** raissa has quit IRC18:01
*** andreas_s has quit IRC18:02
*** lukebrowning has quit IRC18:03
mordredpabelanger: \o/18:03
*** sambetts is now known as sambetts|afk18:03
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Create project-templates for sphinx building with python3  https://review.openstack.org/52909318:03
*** efoley has quit IRC18:04
*** catintheroof has quit IRC18:07
*** catintheroof has joined #openstack-infra18:08
*** dsariel has quit IRC18:09
*** lukebrowning has joined #openstack-infra18:09
*** raissa has joined #openstack-infra18:10
clarkbfungi: ya the actual channel logs probably don't want to be changed18:14
kmalloco/18:14
*** lukebrowning has quit IRC18:14
kmallocoh wait nvm18:14
kmallocugh18:14
fungikmalloc: you're welcome here any time! ;)18:14
*** harlowja has joined #openstack-infra18:15
kmalloc:P18:15
*** lukebrowning has joined #openstack-infra18:15
*** dsariel has joined #openstack-infra18:17
clarkbmwhahaha: for the e-r query change that just went up, dropping that into kibana returns no results for me18:18
clarkbmwhahaha: 507294 specifically18:18
*** lukebrowning has quit IRC18:20
*** links has quit IRC18:23
AJaegerone more change needed to unbreak governance building. clarkb, pabelanger, please review https://review.openstack.org/#/c/52909518:24
*** armaan has quit IRC18:24
AJaegermordred: do you want to keep your IRC priviledges? It's awkward to see "@mordred" here ;)18:24
AJaegermordred: did you see https://review.openstack.org/528960 ? Do you have a better solution for that one?18:26
*** dsariel has quit IRC18:27
*** rbrndt has quit IRC18:28
AJaegerdmsimard: tristanC is fine with https://review.openstack.org/527313 as one step - could you change your -1 to a +2A, please?18:30
*** rmcall has joined #openstack-infra18:30
*** ChanServ sets mode: -o mordred18:31
*** lukebrowning has joined #openstack-infra18:31
mordredAJaeger: thanks for the reminder :)18:31
*** bobh has quit IRC18:32
*** jpena is now known as jpena|off18:33
mordredAJaeger: what's the issue with that patch? (also, wow that's a LOT of doc requirements)18:33
AJaegermordred: doc/requirements.txt includes the [extra] dependencies so that keystone can be installed.18:35
AJaegermordred: I hate those [extra] dependencies via setup.cfg but we cannot just ignore them. So, the change works. But I wonder whether there's a superior solution than putting all optional parts into doc/requirement if you want to autodoc?18:36
*** lukebrowning has quit IRC18:36
mordredAJaeger: ah - yah. I don't have a better solution there - I totally see the benefit of having the requirements broken out into different extras for optional depends, but based on how sphinx imports things you really do have to install the union of them all ...18:37
AJaegermordred: and I'm out of wits on why https://review.openstack.org/#/c/528735/ fails. do you have an idea?18:37
AJaegermordred: ok, so be it - thanks18:37
mordredAJaeger: OR - you have to do *something* in the code itself to perhaps protect against import errors18:37
mordredAJaeger: so that sphinx can import and doc the code - maybe that's a thing we can play with18:37
*** lukebrowning has joined #openstack-infra18:37
*** andreas_s has joined #openstack-infra18:39
*** iyamahat has joined #openstack-infra18:40
AJaegermaybe ;)18:40
*** trown|lunch is now known as trown18:40
*** trown is now known as trown|ruck18:40
AJaegermordred: do you hae any idea on taskflow? I'm not seeing why it fails - and works without the change ;(18:42
AJaegerI can reproduce locally with tox -e docs18:42
mordredAJaeger: cool - I'm doing the same18:42
*** lukebrowning has quit IRC18:42
*** andreas_s has quit IRC18:43
*** lukebrowning has joined #openstack-infra18:44
AJaegermordred: pydotplus is not in doc/requirements.txt - checking whether adding that one helps18:47
*** inc0 has quit IRC18:48
* AJaeger run a diff between the pip freeze output and looked for hints18:48
AJaegerdoes not help ;818:48
*** inc0 has joined #openstack-infra18:48
mwhahahaclarkb: right so that's a bug when the ovb the clouds have problems spinning up envs. We don't have to necessarily merge that but it's one of those nice to know when it happens18:48
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884618:48
*** lukebrowning has quit IRC18:48
clarkbmwhahaha: I see so no current results isn't suprising?18:48
mwhahahaclarkb: no it's not18:49
*** lukebrowning has joined #openstack-infra18:50
clarkbInfra meeting in ~10 minutes over in #openstack-meeting18:50
AJaegermordred: does not help ;( I'm puzzled18:51
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: requirements: remove paramiko <2.0 cap  https://review.openstack.org/52871918:54
clarkbmwhahaha: I'll let mriedem weigh in since he already did a review on it, but looks fine to me if that is expected.18:55
*** lukebrowning has quit IRC18:55
mriedem?18:56
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: requirements: remove paramiko <2.0 cap  https://review.openstack.org/52872918:56
*** dtantsur is now known as dtantsur|afk18:56
clarkbmriedem: https://review.openstack.org/#/c/507294/218:56
*** catintheroof has quit IRC18:56
*** wolverineav has quit IRC18:56
*** lukebrowning has joined #openstack-infra18:56
AJaegermordred: adding *all* missing requirements fixes it ;( So, same bad change as for keystone....18:56
*** r-daneel has quit IRC18:57
AJaegermordred: so, I'll have an IMHO ugly patch in a bit...18:57
*** armaan has joined #openstack-infra18:57
*** wolverineav has joined #openstack-infra18:58
mriedemclarkb: mwhahaha: why would we merge a query that doesn't have any hits?18:58
clarkbmriedem: because the issue happens occasionally and knowing when that happens is nice?18:58
mwhahahaso when it happens again we know why18:58
mwhahahaie regressions18:58
mwhahahaor enviromental failures18:58
mwhahahaso folks don't spin their wheels on a known problem that has happened in the past18:59
fungiand now is the time on infra when we meet19:00
*** jokke_ has quit IRC19:00
*** lukebrowning has quit IRC19:01
*** jokke_ has joined #openstack-infra19:01
*** andreas_s has joined #openstack-infra19:01
AJaegerianw, good morning: dmsimard asked that you review https://review.openstack.org/525658  - could you do so, please? His -1 is procedural...19:02
*** sbezverk has joined #openstack-infra19:02
mordredAJaeger: ok. seems like a thing for us to consider19:03
*** r-daneel has joined #openstack-infra19:03
*** lukebrowning has joined #openstack-infra19:05
ianwAJaeger: done ... i'll try to update the testing on the other job ... but it's my last day until early jan so don't hold your breath :)  not urgent, anyway19:06
AJaegerdmsimard: ianw is happy with https://review.openstack.org/#/c/525658/ - could you remove your -1, please? ianw ,thanks for reviewing19:06
AJaegeryes, not urgent...19:06
AJaegerjust nice ;)19:06
*** lukebrowning has quit IRC19:10
mordredAJaeger: what if we added an optional zuul job variable that is a list of extras ... so similar to how taskflow has in its original tox.ini file:19:10
mordredAJaeger: deps = .[test,workers,zookeeper,database,redis,eventlet,doc]19:10
mordredAJaeger: for projects that have decided to organize their dependencies in this way, the could just say "vars: some_good_name: workers,zookeeper,database,redis,eventlet,doc"19:11
*** lukebrowning has joined #openstack-infra19:11
*** iyamahat has quit IRC19:11
mordredAJaeger: and then in install-if-python role, instead of just pip install . - we could do pip install .[{{some_good_name}}]19:12
clarkbdmsimard: can you join us in #openstack-meeting to talk about your meeting agenda items19:12
AJaegermordred: that will not work for post jobs ;/ the docs publishing job is final19:12
mordredAJaeger: oh - right19:13
mordredAJaeger: well - it'll work for post jobs once we get the base job rework finished19:14
*** shardy has quit IRC19:14
mordredAJaeger: so maybe for now we just make really big doc/requirements.txt files, but with a TODO to shrink them again once we've got the base job rework done19:14
*** andreas_s has quit IRC19:15
*** jamesmcarthur has joined #openstack-infra19:15
AJaegermordred: yeah19:15
*** lukebrowning has quit IRC19:16
*** lukebrowning has joined #openstack-infra19:18
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck master: Remove old queries for fixed bugs  https://review.openstack.org/52913619:18
*** jamesmcarthur has quit IRC19:20
openstackgerritAndreas Jaeger proposed openstack-infra/infra-manual master: Better explain needed patchsets  https://review.openstack.org/52837519:21
AJaegerjeblair: thanks for spotting, fixed ^19:21
openstackgerritMerged openstack-infra/system-config master: Set eavesdrop vhost  https://review.openstack.org/52911819:22
*** lukebrowning has quit IRC19:23
*** lukebrowning has joined #openstack-infra19:24
jeblairAJaeger: now the question is -- should we merge that, or WIP it again pending the merge of 513368?  :)19:24
jeblairAJaeger: documentation driven development!  if you document the terrible process, it gets better :)19:25
AJaegerjeblair: I'm fine if my documents results in great ideas and better code ;)19:25
fungitrue that writing down the process makes it easier to see how terrible it is ;)19:25
AJaegerso, if we merge that, let's abandon my change - once we *use* that change.19:25
* AJaeger volunteers to add the implicit target and remove system-required and update docs...19:26
*** jamesmcarthur has joined #openstack-infra19:27
*** edmondsw has joined #openstack-infra19:28
*** iyamahat has joined #openstack-infra19:28
*** lukebrowning has quit IRC19:29
*** catintheroof has joined #openstack-infra19:29
*** rbrndt has joined #openstack-infra19:29
*** lukebrowning has joined #openstack-infra19:31
*** edmondsw has quit IRC19:32
AJaegerjeblair: depends also how quickly you merge 513368 and apply it...19:35
*** lukebrowning has quit IRC19:36
*** lukebrowning has joined #openstack-infra19:37
*** edmondsw has joined #openstack-infra19:37
*** openstack has joined #openstack-infra19:42
*** ChanServ sets mode: +o openstack19:42
*** edmondsw has quit IRC19:44
*** gouthamr has quit IRC19:45
openstackgerritMerged openstack-infra/irc-meetings master: Move irc meetings tox job in-tree  https://review.openstack.org/52849819:45
AJaegerconfig-core https://review.openstack.org/#/c/528500/ removes a converted job, would appreciate if you put this on your review list for some time this week. thanks19:47
*** yamahata has joined #openstack-infra19:47
*** tdasilva has quit IRC19:47
*** hasharAway is now known as hashar19:47
*** tdasilva has joined #openstack-infra19:47
*** claudiub|2 has joined #openstack-infra19:48
*** electrofelix has quit IRC19:48
jlvillalfungi: So did your script only remove -1 Jenkins and not +1 Jenkins?  https://review.openstack.org/#/c/429770/19:48
*** rlandy|rover is now known as rlandy|rover|bia19:49
AJaegerjlvillal: might be - those are not bad...19:49
AJaegerI see no reason to update 1000s of reviews for that - do you?19:50
*** claudiub has quit IRC19:50
inc0hey, qq about https://review.openstack.org/#/c/526469/10 - latest ps has sytrax error, I assume that's about vars: in line 13...question, how can I pass varible to job that is imported from another project?:)19:53
fungijlvillal: yes19:53
jlvillalAJaeger: fungi thanks. Just curious after seeing the run that removed the Jenkins votes. I wasn't sure if it only did -1s19:54
fungijlvillal: it specifically addressed "nobody is reviewing my change because there's still a -1 from jenkins i can't clear without rebasing or otherwise creating a new patchset"19:54
AJaegerinc0: missing ":" in line 1419:54
jlvillalfungi: Makes sense.19:55
*** dprince has quit IRC19:55
inc0AJaeger: ah..right.. thank you.19:55
clarkbdmsimard: maybe you want to send a thred to the infra mailing list about the puppet modules for new services topic?19:59
AJaegerinfra-root, one more change needed to unbreak governance building, please review https://review.openstack.org/#/c/52909520:00
*** gouthamr has joined #openstack-infra20:00
*** lukebrowning has joined #openstack-infra20:00
AJaegerfungi, pabelanger , could you help, please? ttx has a large pile of changes waiting for the merge ^20:01
*** gouthamr has quit IRC20:01
fungiin _good_ news, looks like we may have only impacted ourselves by triggering a meetbot restart. there was only one other meeting starting at 19:00 and it ended early, before the bot departed20:01
fungiAJaeger: yes, i started looking earlier. will go through that one now20:02
AJaegerthanks, fungi20:02
fungithanks for the reminder20:02
*** lukebrowning has quit IRC20:05
*** caphrim007 has joined #openstack-infra20:10
*** lukebrowning has joined #openstack-infra20:14
*** jtomasek has quit IRC20:15
AJaegerteam, one more docs change for review, please: https://review.openstack.org/#/c/52573520:16
openstackgerritAlex Schultz proposed openstack-infra/elastic-recheck master: Query for 1719469  https://review.openstack.org/50729420:17
*** lukebrowning has quit IRC20:19
*** claudiub|2 has quit IRC20:23
*** dprince has joined #openstack-infra20:23
AJaegerconfig-core, and two review request for translations, please: https://review.openstack.org/#/c/528518 and https://review.openstack.org/#/c/529047 - and then https://review.openstack.org/52911120:23
* AJaeger cannot count until three ;(20:24
openstackgerritMerged openstack-infra/project-config master: Use python3 sphinx building for governance  https://review.openstack.org/52909520:24
*** ldnunes has quit IRC20:26
openstackgerritMerged openstack-infra/elastic-recheck master: Remove old queries for fixed bugs  https://review.openstack.org/52913620:26
*** armaan has quit IRC20:27
openstackgerritMatthew Treinish proposed openstack-infra/system-config master: Add subunit2sql check database server and workers  https://review.openstack.org/52019220:27
*** slaweq_ has joined #openstack-infra20:27
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove jenkins related puppet manifests  https://review.openstack.org/52914520:30
*** lukebrowning has joined #openstack-infra20:30
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Remove jenkins related puppet manifests  https://review.openstack.org/52914520:32
*** lukebrowning has quit IRC20:34
AJaegermordred: did you see my -1 on https://review.openstack.org/#/c/525734 ?20:35
*** shardy has joined #openstack-infra20:36
*** lukebrowning has joined #openstack-infra20:36
*** armaan has joined #openstack-infra20:38
*** rlandy|rover|bia is now known as rlandy|rover20:38
fungi#status log Manually repaired eavesdrop URLs in recent plaintext meeting minutes after https://review.openstack.org/529118 merged20:39
openstackstatusfungi: finished logging20:39
clarkbjeblair: I'm looking at https://review.openstack.org/#/c/528846/8/manifests/site.pp and puppet typically doesn't let you redefine a variable. Does it let yuo go from undef to defined value (tsig_name in particular has me asking that)20:40
*** lukebrowning has quit IRC20:41
clarkbjeblair: also maybe now is a good time to approve https://review.openstack.org/#/c/527498/ ?20:41
*** abelur_ has quit IRC20:43
openstackgerritMatthew Treinish proposed openstack-infra/system-config master: Add subunit2sql check database server and workers  https://review.openstack.org/52019220:48
mtreinishclarkb, fungi: ^^^ I think that should work, I'm just fighting the puppet syntax. Before I commit too much time to that though, do you think the apprach is basically what we discussed in syd?20:48
pabelangerinfra-root: https://review.openstack.org/529145/ deletes a whole lot of jenkins code in system-config, figured might be a good time to land it :D20:49
*** Apoorva has quit IRC20:51
AJaegermordred: http://logs.openstack.org/03/03478673435a7e852529c4132d2fe050dd83471f/post/publish-governance-tc/249e2fb fails ;(20:51
AJaegermordred: I'll sign off for the day now, do you have time to look into that one, please?20:51
*** abelur_ has joined #openstack-infra20:52
AJaegermordred: that's still using pyhon2 instead of python3 - did we not convert the job properly?20:52
clarkbianw: I've reviewed the etherpad stack, left comments on them as I went but only -1 on https://review.openstack.org/#/c/528625/220:54
*** smatzek has quit IRC20:54
AJaegermordred: at least check/gate jobs worked ;)20:54
*** lukebrowning has joined #openstack-infra20:54
* AJaeger waves good night20:55
pabelangerclarkb: so, should we run yum update on a single git server first, before proposing puppet changes? or just let puppet handle it20:55
clarkbmtreinish: one question I have is whether or not we need different workers for this and insteast can have a common worker pool that write to the correct location. I guess check could slow the gate workers if we do that though20:55
clarkbpabelanger: probably not a bad idea though I hope that cgit, git and httpd don't change so much they stop working20:55
clarkbpabelanger: one thing to test is shallow clones20:56
*** jamesmcarthur has quit IRC20:56
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884620:56
jeblairclarkb: ^ fixed20:56
pabelangerclarkb: do we have a process for taking a server out of rotation?20:56
jeblairclarkb: the replacement that we do on hiera values in site.pp makes it difficult to use complex values :(20:56
mtreinishclarkb: It's possible to do it all in one worker, I was just trying to minimize the complexity in the log capture mechanism, it's already complicated enough20:57
clarkbpabelanger: I'm not sure its written down but you disable the three backends for the node (http, https, git://) in haproxy20:57
clarkbpabelanger: you do that via the command socket20:57
mtreinishclarkb: and yeah performance is an unknown quantity for it right now, the check queue has a lot more stuff. The owrkers can keep up with gate well enough, but with gate and check it's not clear20:58
mordredpabelanger: hah. I actually wrote that same patch yesterday but forgot to push it up20:58
mtreinishI also don't know how big the check db will gate even with it's much shorter duration20:58
mtreinishs/gate/get/g20:58
jeblairmtreinish, clarkb: what about the database do we need to manually control?20:59
clarkbjeblair: I think thats largely a mordred thing?20:59
clarkb(wanting to get away from trove)20:59
mtreinishjeblair: well the pain point for subunit2sql in the past is the temp filesystem size20:59
jeblairclarkb: yes, i recall some discussion about that in the past, mostly around backups, etc...20:59
*** lukebrowning has quit IRC20:59
fungipabelanger: i may have some example socat calls for that in my command history on git.o.o... checking now21:00
mordredyah - and being able to do schema upgrades on larger datasets like subunit2sql without downtime by applying the update on a slave, then promoting that slave to master then apply the schema update to the new slave21:00
mtreinishwe can't run certain operations in trove given the size of the db because the tempdir is a ramdisk and the actual os is hidden21:00
*** lukebrowning has joined #openstack-infra21:01
cmurphyAJaeger: mordred i guess our other keystone* docs jobs are broken too, is there a non-hacky fix on the way for including [extras] requirements or should i do the same as https://review.openstack.org/#/c/528960/ for our other projects21:01
*** wolverineav has quit IRC21:01
cmurphylike we have http://logs.openstack.org/31/526631/2/gate/build-openstack-sphinx-docs/153f89b/job-output.txt.gz broken because oslo_messaging is only required in setup.cfg here http://git.openstack.org/cgit/openstack/keystonemiddleware/tree/setup.cfg#n2721:01
*** CrayZee has quit IRC21:02
jeblairmtreinish: ack.  the word "manually" in the commit message threw me for a loop, but looking further at the change, i see you mean "run using configuration management".21:02
fungipabelanger: no such luck. looks like i probably haven't done it since we switched back off the split frontend servers21:02
fungihaproxy docs are your friend21:02
*** catintheroof has quit IRC21:02
mordredcmurphy: we've discussed some non-hacky solutions, but for now I think you should just do the same as https://review.openstack.org/#/c/528960/ - as i tmight take a few days to get the non-hacky fix in place21:02
mtreinishjeblair: ah yeah it's manual as opposed to trove. Heh, I guess it's amatter of degrees21:02
cmurphymordred: kk ty21:03
*** catintheroof has joined #openstack-infra21:03
fungimtreinish: actually _less_ manual for us than trove. spinning up trove instances is not something we have orchestrated in our automation21:03
clarkbpabelanger: https://cbonte.github.io/haproxy-dconv/1.6/management.html#9.2 has info21:04
jeblairbut more manual for server migrations :|21:04
fungifair21:04
*** rossella_s has quit IRC21:04
mtreinishfungi: heh, I assumed there was a playbook or something for the roots to just say give me a trove21:04
fungithough putting the database on a cinder volume probably also makes server migrations pretty painless21:04
clarkbpabelanger: if you look at git.openstack.org:/etc/haproxy/haproxy.cfg there are three listen directives21:04
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Get rid of references to Jenkins slaves  https://review.openstack.org/52914821:05
fungimtreinish: nope, we generally use the command-line client or the service provider's (non-horizon) dashboard21:05
mordredpabelanger: ^^ that goes along with your system-config patch21:05
clarkbpabelanger: each listen directive names a backend then the servers lsited are all the servers you can disable21:05
*** lukebrowning has quit IRC21:05
fungigranted, i'm told there's a non-trove implementation behind that particular trove api anyway21:05
*** rossella_s has joined #openstack-infra21:06
clarkbpabelanger: so I think you can do `disable server balance_git_daemon/git08.openstack.org` against the command socket and do that for the other two backend agaist git08.openstack.org then use git08.openstack.org for testing21:06
mtreinishfungi: well it used to be a fork from a release several years old at this point. Not sure if that changed though21:06
mtreinishit's not like I want to use that service, my rax account lapsed a long time ago too21:07
clarkbpabelanger: echo "disable server balance_git_daemon/git08.openstack.org" | socat /var/lib/haproxy/stats stdio21:07
*** catintheroof has quit IRC21:07
mtreinishwhile I don't see an trove openstack ansible module, I do see: https://docs.ansible.com/ansible/latest/rax_cdb_database_module.html21:08
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Remove additional dead jenkins related code  https://review.openstack.org/52915021:08
clarkbpabelanger: I've just tested `echo "show stat" | socat /var/lib/haproxy/stats stdio` and that works so ya I think the command above and two more for the other backends should work21:09
mordredmtreinish: yah - we haven't added any trove ansible modules yet21:09
*** lukebrowning has joined #openstack-infra21:12
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Set python3 on publish-governance-tc  https://review.openstack.org/52915521:12
mordredAJaeger: ^^ that's what we missed21:12
mordredclarkb, pabelanger, fungi: could I get a quick +3 on that?21:12
clarkbpabelanger: also if you feed it something it doesn't understand it errors gracefully21:14
jeblairfungi: can you review https://review.openstack.org/526483 ? it should be okay to merge now (indeed, i'd like to merge it before the big change to lists.o.o)21:14
mordredjeblair: you wrote a lot of ruby21:15
jeblairmordred: i took one for the team21:15
*** kgiusti has left #openstack-infra21:15
jeblairsadly, i wrote a lot more ruby than actually made it into that change :|21:16
fungiyup, going back over what's changed in the last few patchsets now21:16
ianwjeblair: puppet's failing to connect to adns1 & ns1 due to not knowing their host keys.  is it still wip or should i just accept them?21:16
fungijeblair: but the good news is that we ended up needing far less ruby? ;)21:16
*** armaan has quit IRC21:16
*** lukebrowning has quit IRC21:17
openstackgerritMerged openstack-infra/puppet-exim master: Move user-supplied routers to the end  https://review.openstack.org/52749821:17
openstackgerritMerged openstack-infra/system-config master: Add an authoritative hidden master  https://review.openstack.org/52843821:17
fungiianw: the dns server changes merged a few minutes ago21:17
*** armaan_ has joined #openstack-infra21:17
fungior were approved at least21:17
*** edmondsw has joined #openstack-infra21:17
fungimaybe gating now21:17
ianwok ... there's actually an odd selection of things failing due to host keys -> http://paste.openstack.org/show/629377/21:18
ianwwas just looking at the logs21:18
*** lukebrowning has joined #openstack-infra21:18
jeblairianw: i will accept them once the patches merge21:18
jeblair(for adns1 and ns1)21:19
clarkbfor paste.o.o and grafana.o.o I think that implies the old servers are still in the inventory list and the cname points at the new servers which then fails?21:19
clarkbeither the servers didn't get deleted or ansible hasn't caught up?21:19
*** Apoorva has joined #openstack-infra21:20
pabelangerclarkb: fungi: okay, let me read up on it and see how to do it, I am not sure I'll start now, but maybe in the morning tomorrow21:20
jeblairclarkb, ianw: paste.o.o was deleted.  i don't know why it would show up21:20
ianwyeah, we have both21:20
ianw017-12-19 21:12:24,409 p=16350 u=root |  grafana.openstack.org      : ok=0    changed=0    unreachable=1    failed=021:20
ianw2017-12-19 21:12:24,410 p=16350 u=root |  grafana01.openstack.org    : ok=8    changed=2    unreachable=0    failed=021:20
fungistale inventory cache?21:21
ianwoh, i wonder if this goes back to the stale credentials in all-clouds ... did we fix that?21:21
jeblairgrafana.o.o was also deleted21:22
ianwcan we remove the infra-files-[rw|ro] entries from all-clouds?21:22
*** lukebrowning has quit IRC21:22
*** edmondsw has quit IRC21:22
fungithat was for swift-based log publishing, right?21:23
clarkbianw: thats the swift account we were using21:23
clarkbI think at somepoint we will want to look at using it again but isn't necessary now21:23
pabelangerianw: yah, I think we use it for swift things, but maybe others know21:24
*** lukebrowning has joined #openstack-infra21:24
ianwwell we can't be updating anything with it because the -rw credentials are wrong21:24
ianwthis was a problem when i was using /etc/ansible/hosts/openstack --list for updating the keypairs21:25
clarkbah in that case I think we can just remove them21:25
clarkband sort out swift later if we need to (though why would they suddenly be wrong?)21:25
*** jascott1 has quit IRC21:25
ianwalright, i'll remove them21:26
openstackgerritMerged openstack-infra/project-config master: Set python3 on publish-governance-tc  https://review.openstack.org/52915521:26
fungipossible we never used clouds.yaml for that so the entries for them have always been incorrect in it?21:26
*** armaan_ has quit IRC21:28
*** armaan has joined #openstack-infra21:29
*** lukebrowning has quit IRC21:29
ianwyeah, grafana doesn't show up in the openstack --list, only grafana0121:30
*** lukebrowning has joined #openstack-infra21:30
jeblairmordred: can you approve https://review.openstack.org/528138 ?21:31
jeblairclarkb: want to restart those nodepool launchers now?21:31
ianwfungi: oh right, all-clouds v clouds21:31
clarkbjeblair: sure I'm here for about another hour before I have to salmon21:31
mordredjeblair: done21:32
clarkbdid new shade release get installed /me checks21:32
jeblairshade==1.25.021:32
clarkbI think 1.26.0 is the new shade21:33
clarkbmordred: ^ do you still want us to wait for you on that?21:33
*** slaweq_ is now known as slaweq21:33
jeblairthat's the latest release, november 2921:33
smcginnisWe can't release shade until we get docs builds working. :/21:33
clarkbah maybe release hasn't happened yet then21:33
clarkbgotcha21:33
clarkbin that case I'm inclined to restart now and pick up the bug fixes and features that we can21:33
clarkbjeblair: ^21:33
jeblairclarkb: ya.  shall i type or you want to?21:34
clarkbI can type. Also should we do both at the same time or stagger them?21:34
pabelangersmcginnis: is there a patch up to fix the docs?21:34
jeblairclarkb: one at a time should be fine i think21:34
clarkbI'll run sudo service nodepool-launcher stop then start on nl01 now, check it starts then go on to 0221:34
*** myoung is now known as myoung|bbl21:35
smcginnispabelanger: We need this one to stop hitting random timeouts: https://review.openstack.org/528823 then this one to update the doc requirements https://review.openstack.org/52882421:35
*** lukebrowning has quit IRC21:35
smcginnisAnd I think AJaeger has one yet to switch the jobs to py3.21:36
clarkbjeblair: it seems ot haev started and is reconciling its state21:36
pabelangersmcginnis: k, I think python3 docs landed already21:37
clarkb2017-12-19 21:36:20,338 WARNING nodepool.CleanupWorker: Unable to grab lock to deallocate node 0001518581 from request 100-000165588921:37
*** lukebrowning has joined #openstack-infra21:37
smcginnispabelanger: No, I think this is the next step for that: https://review.openstack.org/#/c/529104/21:37
*** HeOS has quit IRC21:38
pabelangersmcginnis: ah, ya. to convert it21:38
*** slaweq has quit IRC21:38
smcginnispabelanger: And I think I will need to squash that with my requirements.txt update for either of them to pass. I'll do that now and hope the last recheck on the o/requirements patch goes through.21:38
clarkbah ok that is for cleaning up lost requests which is probably normal when service restarts21:38
clarkbjeblair: ready for 02 to be restarted?21:38
mordredsmcginnis: I actually just got that squash done locally21:39
mordredsmcginnis: but if you'd rather do it, totally cool21:39
pabelangersmcginnis: yah, think you are right: No module named 'sphinxcontrib.datatemplates'21:39
smcginnismordred: No, feel free.21:39
mordredsmcginnis: https://review.openstack.org/528824 Add doc job requirements.txt21:39
mordredsmcginnis: updated21:39
jeblairclarkb: yep21:39
clarkbjeblair: ok done21:40
smcginnismordred: Thanks, I'll keep my fingers crossed.21:40
clarkbjeblair: it also appears to be happy and is going through state reconciliation21:41
jeblairclarkb: cool.  we'll want to check graphs periodically and also watch out for errors near quota21:41
*** lukebrowning has quit IRC21:41
jeblairwe may want to do a zuul restart to (a) pick up pending changes and (b) drive us closer to quota :)21:41
clarkbha21:42
clarkbhow to drive up load, restart everything21:42
clarkbI'll leave the zuul restart to you if it is something you want to do21:42
* mordred can always submit some patch bombs ...21:42
*** lukebrowning has joined #openstack-infra21:43
*** trown|ruck is now known as trown|outtypewww21:43
clarkbjeblair: 02 complains about citycloud quota which was preexisting situation21:44
clarkbjeblair: I was somewhat hoping this would address that, but I'm beginning to think their nova may have the wrong quota calculation for us21:44
clarkb(asking about that was in the email I sent them though)21:44
*** ilbelkyr has left #openstack-infra21:45
jeblairclarkb: what's the complaint?21:45
*** wolverineav has joined #openstack-infra21:46
*** priteau_ has joined #openstack-infra21:46
clarkbjeblair: shade.exc.OpenStackCloudHTTPError: (403) Client Error for url: https://lon1.citycloud.com:8774/v2/bed89257500340af8d0fbe7141b1bfd6/servers Quota exceeded for cores, instances: Requested 8, 1, but already used 400, 50 of 400, 50 cores, instances21:46
jeblairclarkb, mordred: there is a shade-related exception: http://paste.openstack.org/show/629380/21:47
clarkbjeblair: but that error was preexisting so not a regression21:47
mordredjeblair's exception seems bad though21:47
mordredI agree with the exception - that is not a real thing21:48
clarkbthe good news is its an exception raising and exception21:48
openstackgerritMerged openstack-infra/system-config master: Make ns*.openstack.org authoritative slaves  https://review.openstack.org/52884621:48
clarkbso we probably handle it pretty similarly :)21:48
*** lukebrowning has quit IRC21:48
*** armaan has quit IRC21:48
*** armaan has joined #openstack-infra21:48
mordredI betcha I can squeeze a fix for that into 1.26 :)21:48
openstackgerritMerged openstack-infra/puppet-mailman master: Add multi-site support  https://review.openstack.org/52648321:49
jeblairclarkb: we do have an error in the new code: http://paste.openstack.org/show/629381/21:49
*** lukebrowning has joined #openstack-infra21:49
mordredhttps://review.openstack.org/529167 Throw OpenStackCloudCreateException on create errors21:49
*** priteau has quit IRC21:50
clarkbjeblair: OpenStackProvider has that method21:50
jeblairclarkb: but not OpenStackProviderConfig21:51
jeblairwe need a test21:51
clarkboh I see _provider isn't a provider21:51
clarkbthis was something I ran into with the lock stuff, the names we've used in places are tricky21:51
*** lukebrowning has quit IRC21:54
*** slaweq has joined #openstack-infra21:55
*** lukebrowning has joined #openstack-infra21:56
*** ijw has joined #openstack-infra21:56
openstackgerritMerged openstack-infra/puppet-zuul master: Update zuulv3 rewrite rules for keys  https://review.openstack.org/52813821:56
smcginnisAnyone know how much disk space is allocated on our test nodes?21:59
smcginnisSeeing this error: http://logs.openstack.org/38/523138/3/check/legacy-tempest-dsvm-full-sheepdog/13c4781/logs/screen-c-sch.txt.gz#_Nov_27_16_31_22_13484821:59
*** slaweq has quit IRC22:00
smcginnisNot sure if it's the test environment or something with this drivers devstack plugin and its configuration.22:00
clarkbsmcginnis: it varies based on cloud but should be ~80GB22:00
*** rossella_s has quit IRC22:00
*** lukebrowning has quit IRC22:01
smcginnisclarkb: OK, thanks. That should be more than plenty then.22:01
*** lukebrowning has joined #openstack-infra22:02
clarkbjeblair: I think we need to call invalidateQuotaCache on self._manager22:04
*** eharney has quit IRC22:05
jeblairclarkb: yeah.  i'm writing a test to make sure that code is exercised22:05
*** armaan has quit IRC22:05
clarkbjeblair: the terminology here is confusing but self._provider is actually just the provider configuration  and self._manager is I think the provider (by way of the provider_manager whcih gives you an OpenStackProvider)22:06
clarkbjeblair: cool, it might be nice to also rename things so that self._provider is self._provider_config and self._manager is self._provider?22:06
*** armaan has joined #openstack-infra22:06
clarkbthat may be a fairly large change though so probably good for followup22:06
jeblairclarkb: yes, i would be happy to have variables more reflective of what they are22:06
*** lukebrowning has quit IRC22:07
clarkboh ya ProviderManager only has static methods so its not really something we have multiple objects of22:07
*** priteau_ has quit IRC22:09
*** pcichy has joined #openstack-infra22:09
*** priteau has joined #openstack-infra22:09
ianwso is /etc/ansible/hosts/generated-groups *only* updated when we run http://git.openstack.org/cgit/openstack-infra/system-config/tree/launch/launch-node.py#n152 ?22:10
clarkbI've got to go help with the salmon shortly so won't dive into updating names now, but will keep an eye out for your patch to help get that reviewed22:10
ianwthat would explain why we have old hosts22:10
clarkbianw: oh ya since we don't delete the old host until later22:10
ianwseems like we should probably slip a cron job in to regenerate that occasionally, to kick deleted hosts before the next node launch22:12
ianwnot that it matters much ... just a bit confusing22:12
*** claudiub|2 has joined #openstack-infra22:12
*** jcoufal has quit IRC22:13
*** priteau has quit IRC22:14
*** jascott1 has joined #openstack-infra22:19
ianwno that's not it22:21
ianwhttp://paste.openstack.org/show/629391/22:21
ianwansible --list shows it, /etc/ansible/hosts/openstack --list doesn't22:22
*** florianf has quit IRC22:22
ianwgrafana.o.o does not appear in /var/cache/ansible-inventory/ansible-inventory.cache22:23
*** rcernin has joined #openstack-infra22:28
*** HeOS has joined #openstack-infra22:32
*** slaweq has joined #openstack-infra22:32
*** lukebrowning has joined #openstack-infra22:35
*** salv-orl_ has quit IRC22:36
*** salv-orlando has joined #openstack-infra22:37
*** shardy has quit IRC22:37
*** slaweq has quit IRC22:37
*** pcichy has quit IRC22:38
*** lukebrowning has quit IRC22:40
*** marst has quit IRC22:41
*** salv-orlando has quit IRC22:41
*** lukebrowning has joined #openstack-infra22:42
*** salv-orlando has joined #openstack-infra22:42
*** lukebrowning has quit IRC22:47
*** lukebrowning has joined #openstack-infra22:48
ianwyou know what ... ansible reads /etc/ansible/hosts/generated-groups as it's creating generated-groups.  if the regex matches in  t/usr/local/bin/expand-groups.sh, the host gets put back into it indefinitely22:50
*** lukebrowning has quit IRC22:53
*** slaweq has joined #openstack-infra22:53
*** lukebrowning has joined #openstack-infra22:54
openstackgerritJames E. Blair proposed openstack-infra/nodepool feature/zuulv3: Fail on quota-exceeded (partial revert)  https://review.openstack.org/52917322:56
jeblairclarkb, mordred, tobiash: ^22:56
*** myoung|bbl is now known as myoung22:56
*** slaweq has quit IRC22:58
*** lennyb has quit IRC22:58
*** lukebrowning has quit IRC22:59
mordredjeblair: reading22:59
jeblairhrm.  the nodepool graphs are discontinuitous after the restart23:00
*** lukebrowning has joined #openstack-infra23:00
*** lennyb has joined #openstack-infra23:00
mordredjeblair: patch lgtm23:03
jeblairmordred: does get_complute_limits use task manager?23:05
jeblairget_compute_limits23:05
mordredjeblair: everything uses task manager23:05
*** lukebrowning has quit IRC23:05
jeblairah there it is, sorry, i missed the task23:05
*** edmondsw has joined #openstack-infra23:06
mordredjeblair: yah- all the rest calls go through task manager at the adapter layer - it would take a massive effort at this point to avoid task manager :)23:06
jeblairi can't actually tell what nodepool thinks the quotas are23:07
jeblairit outputs stuff to the log, but does not include the provider name23:07
jeblairi suspect that there are problems23:07
jeblairfor instance, i don't think we've spun up a node on rackspace since the changes went in23:07
mordredjeblair: oh - you know what - I think rax reports bogus quota values23:08
jeblair2017-12-19 23:06:21,619 DEBUG nodepool.driver.openstack.OpenStackProvider: Available nodepool quota: {'compute': {'ram': 2572288, 'instances': 219, 'cores': -25}}23:09
jeblairit's possible that's rax-dfw, but i don't know for sure23:09
mordredjeblair: nod. I just checked a rax dfw acount...23:10
*** edmondsw has quit IRC23:10
mordredjeblair: http://paste.openstack.org/raw/629397/  - which does sure return max_total_cores -123:11
mordredalthough 'total_cores_used': 629 - so it does report *something(*23:11
jeblairnodepool has 72 cores in use in dfw right now23:12
jeblair-1 - 72 = -73 != -25  ... so i don't know where it got the -25 from23:13
jeblairbut maybe that wasn't ord23:13
jeblairer dfw23:13
jeblairanyway, it seems problematic23:13
mordredyah23:13
jeblairperhaps -1 means "not limited" ?23:14
mordredmaybe?23:14
*** lukebrowning has joined #openstack-infra23:14
mordredmriedem: ^^ does '-1' have a meaning for nova for quota/usage?23:14
jeblairi think i'll start with 2 changes: a) log the provider name :)  b) assume -1 means unlimited23:15
jeblairand see where that gets us23:15
mordredjeblair: sounds good to me23:15
*** hashar has quit IRC23:15
mriedemmordred: unlimited23:16
mordredjeblair: you win23:16
mordredmriedem: thanks23:17
openstackgerritKendall Nelson proposed openstack-infra/storyboard master: Cleanup Migration Docs  https://review.openstack.org/52917723:18
*** lukebrowning has quit IRC23:19
*** lukebrowning has joined #openstack-infra23:20
openstackgerritJames E. Blair proposed openstack-infra/nodepool feature/zuulv3: Log provider names with quota  https://review.openstack.org/52917823:21
clarkbjeblair: reading nodepool fix test, why do we modify the client max instances when we have the quota mocked?23:22
clarkbjeblair: is it caching the quota value from the mock then not asking again?23:23
*** felipemonteiro has quit IRC23:24
*** felipemonteiro has joined #openstack-infra23:24
*** lukebrowning has quit IRC23:25
*** lukebrowning has joined #openstack-infra23:26
openstackgerritJames E. Blair proposed openstack-infra/nodepool feature/zuulv3: Assume a quota limit of -1 means unlimited  https://review.openstack.org/52918023:27
*** andreas_s has joined #openstack-infra23:28
jeblairclarkb: yes, the mock is used only when the provider is constructed.23:28
jeblairclarkb: that's not a mock of a real function -- it's basically just a way to pass a parameter to the provider creation out of band.23:29
jeblairclarkb, mordred: both changes up ^23:30
jeblairinfra-root: ^ recent quota handling changes to nodepool have caused us to stop using rax; fixes in progress; if they don't merge soon, we should revert.23:31
*** lukebrowning has quit IRC23:31
*** andreas_s has quit IRC23:32
jeblairi would revert to a20d81c32c7caeb330bbc6b7373ddb975c533e20 if necessary23:32
*** lukebrowning has joined #openstack-infra23:33
clarkbjeblair: the -1 change us what we need for rax?23:33
jeblairclarkb: that's my understanding, based on some, i think, reasonable assumptions i'm making from the incomplete log output23:34
clarkbok reviewing the rest of stack now23:34
*** HeOS has quit IRC23:34
*** stakeda has joined #openstack-infra23:35
clarkbjeblair changes lgtm23:37
clarkbI approved the first one should I remove approval in case we want to revert?23:37
clarkb(though getting fixes in seems like a good idea)23:37
jeblairclarkb: nah, let's see if we can roll forward23:37
*** lukebrowning has quit IRC23:38
openstackgerritMerged openstack-infra/project-config master: Improve zanata-cli download  https://review.openstack.org/52565823:38
openstackgerritMerged openstack-infra/project-config master: Add log-classify check and gate jobs  https://review.openstack.org/52731323:38
*** lukebrowning has joined #openstack-infra23:39
*** pahuang has joined #openstack-infra23:42
*** catinthe_ has joined #openstack-infra23:43
*** lukebrowning has quit IRC23:44
*** lukebrowning has joined #openstack-infra23:45
*** rbrndt has quit IRC23:46
openstackgerritIan Wienand proposed openstack-infra/system-config master: Avoid generated-groups when creating generated-groups  https://review.openstack.org/52918623:48
openstackgerritIan Wienand proposed openstack-infra/system-config master: Run expand-groups.sh occasionaly under cron  https://review.openstack.org/52918723:48
openstackgerritIan Wienand proposed openstack-infra/system-config master: Run expand-groups.sh occasionally under cron  https://review.openstack.org/52918723:49
*** lukebrowning has quit IRC23:50
pabelangerjeblair: clarkb: +3 on rax fixes23:51
jeblairpabelanger: great, thanks!  hopefully those will go in and we don't have to do a manual revert23:51
openstackgerritIan Wienand proposed openstack-infra/system-config master: Fix codesearch stderr capturing  https://review.openstack.org/52918823:52
*** catintheroof has joined #openstack-infra23:52
*** catintheroof has quit IRC23:52
*** catintheroof has joined #openstack-infra23:53
*** catinthe_ has quit IRC23:55
ianw#status log removed infra-files-ro and infra-files-rw from all-clouds.yaml as they are invalid, and cause issues deploying new keys.  saved in a backup file on puppetmaster.o.o if required23:56
openstackstatusianw: finished logging23:56
*** lukebrowning has joined #openstack-infra23:56
*** catintheroof has quit IRC23:58
pabelangerianw: is there a patch up for all-clouds.yaml in system-config?23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!