| guilhermesp | hello everyone! so I'm from openstack-ansible team :) | 00:02 |
|---|---|---|
| guilhermesp | well, as placement is not part of the nova than, we are going to create an openstack-ansible-os_placement role. | 00:02 |
| guilhermesp | I'm following that guide to add the project to the openstack ci | 00:03 |
| guilhermesp | https://docs.openstack.org/infra/manual/creators.html#add-the-project-to-the-master-projects-list | 00:03 |
| guilhermesp | just to confirm, that's the repo to add the project, isn't it? https://review.openstack.org/#/admin/projects/openstack-infra/project-config | 00:03 |
| tonyb | clarkb: any chance you can +1 https://review.openstack.org/#/c/614887/ | 00:04 |
| clarkb | tonyb: done | 00:05 |
| tonyb | clarkb: Thanks | 00:06 |
| clarkb | guilhermesp: yes that is the repo to modify | 00:06 |
| clarkb | guilhermesp: our automation picks up changes to that repo then updates gerrit based on those changes | 00:06 |
| guilhermesp | thanks clarkb ! I'm just trying to pick every information about the whole process :) Hopefully I can use another osa projects as a reference to kick of the placement role | 00:07 |
| guilhermesp | yeah, seems straight forward as we have a bunch of osa projects, they all share the same acls | 00:12 |
| *** hongbin has joined #openstack-infra | 00:17 | |
| *** ssbarnea has quit IRC | 00:25 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings master: add tc meetings https://review.openstack.org/608682 | 00:33 |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings master: Use yaml2ical 0.9.0 to get monthly events https://review.openstack.org/614892 | 00:33 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 00:41 |
| *** longkb has joined #openstack-infra | 00:44 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 00:49 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 00:56 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack-infra/project-config master: Add os_placement role to OpenStack-Ansible https://review.openstack.org/614896 | 00:59 |
| *** agopi|pto is now known as agopi|off | 01:11 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 01:12 |
| *** zhangfei has joined #openstack-infra | 01:15 | |
| *** diablo_rojo has quit IRC | 01:16 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 01:30 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack-infra/project-config master: Add os_placement role to OpenStack-Ansible https://review.openstack.org/614896 | 01:30 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack-infra/project-config master: Add os_placement role to OpenStack-Ansible https://review.openstack.org/614896 | 01:32 |
| *** hongbin has quit IRC | 01:32 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 02:00 |
| *** hongbin has joined #openstack-infra | 02:12 | |
| *** carl_cai has joined #openstack-infra | 02:13 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 02:18 |
| *** yamamoto has joined #openstack-infra | 02:31 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 02:33 |
| *** xarses_ has joined #openstack-infra | 02:36 | |
| *** bobh has joined #openstack-infra | 02:37 | |
| *** jamesmcarthur has joined #openstack-infra | 02:43 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] test against ansible master https://review.openstack.org/614894 | 02:43 |
| *** jamesmcarthur has quit IRC | 02:44 | |
| *** jamesmcarthur has joined #openstack-infra | 02:45 | |
| *** mrsoul has joined #openstack-infra | 02:51 | |
| *** bobh has quit IRC | 02:57 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: uses queues uid to preserve state on change https://review.openstack.org/614933 | 02:59 |
| *** bhavikdbavishi has joined #openstack-infra | 02:59 | |
| *** eernst has joined #openstack-infra | 03:01 | |
| *** eernst has quit IRC | 03:08 | |
| *** yamamoto has quit IRC | 03:09 | |
| *** yamamoto has joined #openstack-infra | 03:09 | |
| *** yamamoto has quit IRC | 03:09 | |
| tonyb | In the past the irc-meetings-tox-ical job in irc-meetings just to copy generated html and iCal files ontp the logs server so we could check the thinsg looked about right. | 03:21 |
| tonyb | What's the right way to do that again? Is there a role I can add to the job? | 03:21 |
| *** jamesmcarthur has quit IRC | 03:25 | |
| *** jamesmcarthur has joined #openstack-infra | 03:26 | |
| *** yamamoto has joined #openstack-infra | 03:30 | |
| *** jamesmcarthur has quit IRC | 03:39 | |
| *** ramishra has joined #openstack-infra | 03:50 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Pin bridge.o.o to ansible 2.7.0, add devel testing job https://review.openstack.org/614894 | 03:51 |
| *** udesale has joined #openstack-infra | 03:56 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 03:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 04:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Remove start_services https://review.openstack.org/614961 | 04:02 |
| *** bhavikdbavishi1 has joined #openstack-infra | 04:07 | |
| *** david-lyle has joined #openstack-infra | 04:08 | |
| *** bhavikdbavishi has quit IRC | 04:09 | |
| *** dklyle has quit IRC | 04:09 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 04:09 | |
| *** auristor has quit IRC | 04:10 | |
| *** toabctl has quit IRC | 04:13 | |
| *** toabctl has joined #openstack-infra | 04:14 | |
| *** hongbin has quit IRC | 04:17 | |
| *** ykarel|away has joined #openstack-infra | 04:24 | |
| openstackgerrit | Kim Bao Long proposed openstack-infra/ansible-role-puppet master: Update min tox version to 2.0 https://review.openstack.org/614970 | 04:34 |
| *** zhangfei has quit IRC | 04:47 | |
| *** ykarel|away is now known as ykarel | 04:47 | |
| *** janki has joined #openstack-infra | 04:48 | |
| *** zhangfei has joined #openstack-infra | 04:57 | |
| *** Keitaro has quit IRC | 05:03 | |
| *** carl_cai has quit IRC | 05:22 | |
| *** auristor has joined #openstack-infra | 05:26 | |
| *** threestrands has quit IRC | 05:40 | |
| *** kjackal has joined #openstack-infra | 05:45 | |
| *** quiquell|off is now known as quiquell | 06:07 | |
| *** ykarel is now known as ykarel|afk | 06:21 | |
| *** ramishra has quit IRC | 06:22 | |
| *** dpawlik has joined #openstack-infra | 06:26 | |
| *** dpawlik has quit IRC | 06:26 | |
| *** dpawlik has joined #openstack-infra | 06:27 | |
| *** apetrich has quit IRC | 06:27 | |
| *** ykarel|afk has quit IRC | 06:31 | |
| *** ykarel|afk has joined #openstack-infra | 06:31 | |
| openstackgerrit | Kim Bao Long proposed openstack-infra/system-config master: Update the min version of tox to 2.0 https://review.openstack.org/615023 | 06:40 |
| *** annp has joined #openstack-infra | 06:43 | |
| *** apetrich has joined #openstack-infra | 06:43 | |
| *** udesale has quit IRC | 06:49 | |
| openstackgerrit | Kim Bao Long proposed openstack-infra/storyboard-webclient master: Update the min version of tox to 2.0 https://review.openstack.org/615035 | 06:49 |
| *** ramishra has joined #openstack-infra | 06:53 | |
| openstackgerrit | Vieri proposed openstack/os-performance-tools master: Update min tox version to 2.0 https://review.openstack.org/615045 | 06:58 |
| *** erlon has joined #openstack-infra | 07:14 | |
| *** gbutnaru has quit IRC | 07:15 | |
| *** pcaruana has joined #openstack-infra | 07:20 | |
| *** Douhet has quit IRC | 07:22 | |
| *** Douhet has joined #openstack-infra | 07:22 | |
| *** auristor has quit IRC | 07:24 | |
| *** Douhet has quit IRC | 07:33 | |
| *** ccamacho has joined #openstack-infra | 07:34 | |
| *** auristor has joined #openstack-infra | 07:35 | |
| *** Douhet has joined #openstack-infra | 07:37 | |
| *** lennyb has quit IRC | 07:39 | |
| *** lennyb has joined #openstack-infra | 07:40 | |
| *** armax has quit IRC | 07:43 | |
| *** ramishra has quit IRC | 07:46 | |
| *** ramishra has joined #openstack-infra | 07:53 | |
| *** kjackal has quit IRC | 07:55 | |
| *** coolsvap has joined #openstack-infra | 07:55 | |
| openstackgerrit | Vieri proposed openstack-dev/specs-cookiecutter master: Update min tox version to 2.0 https://review.openstack.org/615083 | 07:57 |
| *** ykarel|afk is now known as ykarel | 08:11 | |
| *** slaweq has joined #openstack-infra | 08:17 | |
| *** bhavikdbavishi has quit IRC | 08:25 | |
| *** ralonsoh has joined #openstack-infra | 08:26 | |
| *** kjackal has joined #openstack-infra | 08:29 | |
| *** betherly has joined #openstack-infra | 08:30 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Add systemd-containers functional tests https://review.openstack.org/614051 | 08:37 |
| *** florianf|afk is now known as florianf | 08:40 | |
| *** bauzas is now known as bauwser | 08:52 | |
| *** jpich has joined #openstack-infra | 09:00 | |
| *** xek__ has joined #openstack-infra | 09:08 | |
| *** shrasool has joined #openstack-infra | 09:11 | |
| *** erlon has quit IRC | 09:13 | |
| *** ssbarnea has joined #openstack-infra | 09:15 | |
| *** ykarel_ has joined #openstack-infra | 09:20 | |
| *** ykarel has quit IRC | 09:23 | |
| *** derekh has joined #openstack-infra | 09:25 | |
| *** dpawlik has quit IRC | 09:28 | |
| *** ykarel_ is now known as ykarel | 09:31 | |
| *** ykarel_ has joined #openstack-infra | 09:34 | |
| *** ykarel_ has quit IRC | 09:35 | |
| *** ykarel has quit IRC | 09:36 | |
| quiquell | Good morning | 09:41 |
| quiquell | Do we have a URL with the centos7 images we use at zuul ? | 09:41 |
| *** zhangfei has quit IRC | 09:42 | |
| eumel8 | o/ | 09:43 |
| eumel8 | something is broken on the user-survey: https://www.openstack.org/user-survey/ | 09:43 |
| eumel8 | seems like database backend issue | 09:43 |
| *** dtantsur|afk is now known as dtantsur | 09:48 | |
| *** e0ne has joined #openstack-infra | 09:51 | |
| *** udesale has joined #openstack-infra | 09:53 | |
| *** shrasool has quit IRC | 09:54 | |
| openstackgerrit | Pavlo Shchelokovskyy proposed openstack/diskimage-builder master: Disable tgt service under systemd for ironic-agent https://review.openstack.org/615119 | 09:54 |
| *** ykarel has joined #openstack-infra | 09:55 | |
| bauwser | AJaeger: morning | 09:55 |
| bauwser | ttx: morning too | 09:55 |
| bauwser | AJaeger: ttx: something got messed up with the website | 09:56 |
| bauwser | https://www.openstack.org/summit/berlin-2018/ gives me a weird "SilverStripe Framework requires a $databaseConfig defined." | 09:56 |
| bauwser | ttx: AJaeger: previously, I got a connection refused so I suspect someone is looking at it anyway | 09:57 |
| eumel8 | bauwser: had the same | 09:59 |
| ttx | yes confirmed. Will ping internally | 10:00 |
| eumel8 | maybe frickler can help, or other from the EU team | 10:00 |
| bauwser | ttx: ack, thanks | 10:01 |
| bauwser | context: it was for subscribing for the pub crawl event | 10:01 |
| *** shrasool has joined #openstack-infra | 10:01 | |
| bauwser | tl;dr : beer triggered monitoring system | 10:01 |
| bauwser | subscribing to* | 10:01 |
| ttx | affects all of www.openstack.org -- that's not run by the infra team | 10:02 |
| bauwser | ttx: ack, gtk | 10:02 |
| bauwser | (oh btw. bauwser == bauzas on Fridays) | 10:02 |
| ttx | thanks for flagging it! | 10:02 |
| bauwser | ttx: I was thinking code was hosted by infra? | 10:03 |
| frickler | yeah, nothing I can do about that site I fear, will need some foundation folks | 10:03 |
| bauwser | but infra isn't, gotcha | 10:03 |
| *** panda|off is now known as panda | 10:03 | |
| *** jamesmcarthur has joined #openstack-infra | 10:05 | |
| *** jamesmcarthur has quit IRC | 10:09 | |
| *** annp has quit IRC | 10:10 | |
| *** sshnaidm|off has quit IRC | 10:10 | |
| *** shrasool has quit IRC | 10:11 | |
| *** shrasool has joined #openstack-infra | 10:14 | |
| *** sshnaidm has joined #openstack-infra | 10:14 | |
| eumel8 | ok | 10:16 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Use yaml2ical 0.9.0 to get monthly events https://review.openstack.org/614892 | 10:16 |
| shrasool | the website is also “SilverStripe Framework requires a $databaseConfig defined.” | 10:17 |
| *** quiquell is now known as quiquell|brb | 10:17 | |
| *** sshnaidm is now known as sshnaidm|off | 10:24 | |
| *** yamamoto has quit IRC | 10:24 | |
| *** EmilienM is now known as EvilienM | 10:25 | |
| *** yamamoto has joined #openstack-infra | 10:28 | |
| *** owalsh_ is now known as owalsh | 10:30 | |
| *** alexchadin has joined #openstack-infra | 10:36 | |
| *** alexchadin has quit IRC | 10:37 | |
| *** alexchadin has joined #openstack-infra | 10:37 | |
| *** alexchadin has quit IRC | 10:37 | |
| *** alexchadin has joined #openstack-infra | 10:38 | |
| *** alexchadin has quit IRC | 10:38 | |
| *** alexchadin has joined #openstack-infra | 10:39 | |
| *** agopi|off has quit IRC | 10:39 | |
| *** alexchadin has quit IRC | 10:39 | |
| *** agopi|off has joined #openstack-infra | 10:39 | |
| *** udesale has quit IRC | 10:40 | |
| *** mgoddard has joined #openstack-infra | 10:41 | |
| mgoddard | morning, could someone add me to the tenks-core group in Gerrit? | 10:41 |
| *** agopi|off has quit IRC | 10:43 | |
| *** stephenfin is now known as finucannot | 10:44 | |
| finucannot | I'd love it if someone could finish reviewing https://review.openstack.org/#/q/topic:story/1130330+(status:open+OR+status:merged) so we could get it in a release. It's killing me downstream, where our bug reports naturally have a different number | 10:46 |
| finucannot | clarkb: Any progress on nominating new cores for git-review? ^_^ | 10:47 |
| *** quiquell|brb is now known as quiquell | 10:48 | |
| *** longkb has quit IRC | 10:50 | |
| *** shrasool has quit IRC | 10:59 | |
| *** electrofelix has joined #openstack-infra | 11:00 | |
| *** shrasool has joined #openstack-infra | 11:01 | |
| *** erlon has joined #openstack-infra | 11:06 | |
| frickler | mgoddard: I cannot seem to find that group, do you have a reference to a patch that should have caused it to be created? | 11:08 |
| mgoddard | frickler: it's a new project, added to project-config via https://review.openstack.org/#/c/600397/2. I can see the group in the gerrit group list but it currently has no members | 11:09 |
| frickler | ttx: do you have some feedback from the foundation regarding www or will we have to wait for US folks to be awake? I'm wondering whether we should push a status notice | 11:09 |
| frickler | mgoddard: ah, right, I shouldn't look for it under "My groups" ;) ... added you as a member | 11:12 |
| mgoddard | frickler: heh, that caught me out first time around. Thanks | 11:12 |
| *** shrasool has quit IRC | 11:13 | |
| *** shrasool has joined #openstack-infra | 11:16 | |
| *** alexchadin has joined #openstack-infra | 11:17 | |
| ttx | frickler: haven't got a response yet. We have a group in South America that is usually awake earlier but I suspect this is a holiday where they are | 11:18 |
| ttx | I don't think the infra team should put out a notice for an outage on a system they don't operate :) | 11:19 |
| *** alexchadin has quit IRC | 11:22 | |
| *** lpetrut has joined #openstack-infra | 11:32 | |
| *** yamamoto has quit IRC | 11:33 | |
| eumel8 | ttx: it seems it's working again | 11:34 |
| ttx | likely a lost connection to the DB instance | 11:35 |
| *** yamamoto has joined #openstack-infra | 11:39 | |
| *** kjackal has quit IRC | 11:43 | |
| *** yamamoto has quit IRC | 11:48 | |
| openstackgerrit | Vieri proposed openstack-infra/irc-meetings master: Update min tox version to 2.0 https://review.openstack.org/615139 | 11:48 |
| eumel8 | ttx: would be good to know how we can report such kind of issues in the future without disturbing the infra team :) | 11:49 |
| openstackgerrit | Vieri proposed openstack-infra/os-loganalyze master: Update min tox version to 2.0 https://review.openstack.org/615142 | 11:51 |
| openstackgerrit | Vieri proposed openstack-infra/openstackid master: Update min tox version to 2.0 https://review.openstack.org/615143 | 11:52 |
| openstackgerrit | Vieri proposed openstack-infra/gear master: Update min tox version to 2.0 https://review.openstack.org/615144 | 11:52 |
| *** alexchadin has joined #openstack-infra | 11:53 | |
| openstackgerrit | Vieri proposed openstack-infra/storyboard-webclient master: Update min tox version to 2.0 https://review.openstack.org/615145 | 11:53 |
| openstackgerrit | Vieri proposed openstack-infra/germqtt master: Update min tox version to 2.0 https://review.openstack.org/615146 | 11:54 |
| openstackgerrit | Vieri proposed openstack-infra/lpmqtt master: Update min tox version to 2.0 https://review.openstack.org/615147 | 11:55 |
| openstackgerrit | Vieri proposed openstack-infra/zuul-base-jobs master: Update min tox version to 2.0 https://review.openstack.org/615148 | 11:56 |
| openstackgerrit | Vieri proposed openstack-infra/python-storyboardclient master: Update min tox version to 2.0 https://review.openstack.org/615149 | 11:56 |
| openstackgerrit | Vieri proposed openstack-infra/glean master: Update min tox version to 2.0 https://review.openstack.org/615151 | 12:00 |
| *** alexchadin has quit IRC | 12:00 | |
| openstackgerrit | Vieri proposed openstack-infra/puppet-openstackci master: Update min tox version to 2.0 https://review.openstack.org/615152 | 12:01 |
| openstackgerrit | Vieri proposed openstack-infra/ansible-role-puppet master: Update min tox version to 2.0 https://review.openstack.org/615153 | 12:02 |
| *** shrasool has quit IRC | 12:02 | |
| *** shrasool has joined #openstack-infra | 12:02 | |
| openstackgerrit | Vieri proposed openstack-infra/infra-specs master: Update min tox version to 2.0 https://review.openstack.org/615154 | 12:03 |
| strigazi | Hello, I'm trying to encrypt a secret with tools/encrypt_secret.py from openstack-infra/zuul but I can't find the zuul server url. It is not zuul.openstack.org | 12:03 |
| openstackgerrit | Vieri proposed openstack-infra/system-config master: Update min tox version to 2.0 https://review.openstack.org/615155 | 12:04 |
| openstackgerrit | Vieri proposed openstack-infra/project-config master: Update min tox version to 2.0 https://review.openstack.org/615156 | 12:05 |
| openstackgerrit | Vieri proposed openstack-infra/subunit2sql master: Update min tox version to 2.0 https://review.openstack.org/615157 | 12:06 |
| *** zul has quit IRC | 12:06 | |
| openstackgerrit | Vieri proposed openstack-infra/elastic-recheck master: Update min tox version to 2.0 https://review.openstack.org/615158 | 12:06 |
| eumel8 | strigazi: maybe ask in #zuul | 12:07 |
| strigazi | eumel8: I was looking for #openstack-zuul thanks | 12:07 |
| eumel8 | yeah, it's only #zuul :) | 12:08 |
| openstackgerrit | Vieri proposed openstack-infra/openstackid-resources master: Update min tox version to 2.0 https://review.openstack.org/615160 | 12:09 |
| *** pbourke has quit IRC | 12:10 | |
| *** pbourke has joined #openstack-infra | 12:10 | |
| *** kjackal has joined #openstack-infra | 12:10 | |
| openstackgerrit | Vieri proposed openstack-infra/infra-manual master: Update min tox version to 2.0 https://review.openstack.org/615161 | 12:10 |
| strigazi | I think it is an infra question, since I look for the openstack.org zuul deployment. Thanks anyway. | 12:11 |
| openstackgerrit | Vieri proposed openstack-infra/zuul-sphinx master: Update min tox version to 2.0 https://review.openstack.org/615162 | 12:11 |
| *** jcoufal has joined #openstack-infra | 12:12 | |
| openstackgerrit | Vieri proposed openstack-infra/gerritlib master: Update min tox version to 2.0 https://review.openstack.org/615163 | 12:13 |
| *** lpetrut has quit IRC | 12:13 | |
| openstackgerrit | Vieri proposed openstack-infra/zuul master: Update min tox version to 2.0 https://review.openstack.org/615164 | 12:13 |
| openstackgerrit | Vieri proposed openstack-infra/openstack-zuul-roles master: Update min tox version to 2.0 https://review.openstack.org/615165 | 12:14 |
| openstackgerrit | Vieri proposed openstack-infra/afsmon master: Update min tox version to 2.0 https://review.openstack.org/615166 | 12:15 |
| openstackgerrit | Vieri proposed openstack-infra/reviewstats master: Update min tox version to 2.0 https://review.openstack.org/615167 | 12:16 |
| openstackgerrit | Vieri proposed openstack-infra/openstack-zuul-jobs master: Update min tox version to 2.0 https://review.openstack.org/615168 | 12:18 |
| openstackgerrit | Vieri proposed openstack-infra/devstack-gate master: Update min tox version to 2.0 https://review.openstack.org/615169 | 12:19 |
| openstackgerrit | Vieri proposed openstack-infra/ciwatch master: Update min tox version to 2.0 https://review.openstack.org/615170 | 12:20 |
| openstackgerrit | Vieri proposed openstack-infra/grafyaml master: Update min tox version to 2.0 https://review.openstack.org/615171 | 12:20 |
| openstackgerrit | Vieri proposed openstack-infra/zuul-jobs master: Update min tox version to 2.0 https://review.openstack.org/615172 | 12:22 |
| frickler | did I miss something or is this a new batch of uncoordinated mass patches? | 12:24 |
| openstackgerrit | Vieri proposed openstack-infra/log_processor master: Update min tox version to 2.0 https://review.openstack.org/615173 | 12:24 |
| openstackgerrit | Ildiko Vancsa proposed openstack-infra/project-config master: Add StarlingX core groups https://review.openstack.org/615174 | 12:25 |
| *** trown|outtypewww is now known as trown | 12:25 | |
| openstackgerrit | Vieri proposed openstack-infra/yaml2ical master: Update min tox version to 2.0 https://review.openstack.org/615175 | 12:26 |
| openstackgerrit | Gary Perkins proposed openstack-infra/system-config master: Add Arm64 CI cloud https://review.openstack.org/602436 | 12:26 |
| openstackgerrit | Vieri proposed openstack-infra/lodgeit master: Update min tox version to 2.0 https://review.openstack.org/615176 | 12:27 |
| openstackgerrit | Ildiko Vancsa proposed openstack-infra/project-config master: Add StarlingX core groups https://review.openstack.org/615174 | 12:28 |
| aspiers | anyone interested in a better understanding of OpenStack's code review culture within Gerrit should check out this awesome talk which is finally online one year later! https://twitter.com/GerritReview/status/1058310312389197824 | 12:32 |
| aspiers | fungi, corvus, clarkb: ^^^ | 12:33 |
| *** jamesmcarthur has joined #openstack-infra | 12:34 | |
| fungi | aspiers: awesome! how was the conference overall? | 12:34 |
| aspiers | it was really good, which is why I promoted this year's event within this community. | 12:35 |
| fungi | oh, that was from last year's? | 12:35 |
| aspiers | yes | 12:35 |
| aspiers | just got published a few minutes agoo | 12:35 |
| fungi | i see that now in the quoted frame | 12:35 |
| fungi | quite the delay | 12:35 |
| *** yamamoto has joined #openstack-infra | 12:36 | |
| aspiers | enough that I got a personal apology as you can see :) | 12:36 |
| *** zul has joined #openstack-infra | 12:36 | |
| *** jamesmcarthur has quit IRC | 12:39 | |
| *** quiquell is now known as quiquell|lunch | 12:43 | |
| fungi | quiquell: https://nb01.openstack.org/images/ or if you want the arm64 images instead of amd64 then https://nb03.openstack.org/images/ (there's a self-signed https cert you'll have to okay to access those urls) | 12:44 |
| *** jamesmcarthur has joined #openstack-infra | 12:45 | |
| *** yamamoto has quit IRC | 12:47 | |
| *** yamamoto has joined #openstack-infra | 12:49 | |
| fungi | mgoddard: i've added you to tenks-core and removed the ironic-core group from being included in it for now since it looks like it ended up not becoming an official ironic project after all | 12:51 |
| *** kgiusti has joined #openstack-infra | 12:53 | |
| fungi | strigazi: eumel8: yes, this is the appropriate channel to ask questions about zuul.openstack.org, we try not to burden #zuul with support questions from users of openstack's deployment of their software (though there is a lot of overlap of the same people on both channels) | 12:53 |
| *** janki has quit IRC | 12:53 | |
| mgoddard | fungi: I think it's still TBD but correct that it's not official yet. I'll wait for that to happen before adding ironic-core | 12:53 |
| *** janki has joined #openstack-infra | 12:53 | |
| *** yamamoto has quit IRC | 12:56 | |
| fungi | strigazi: the last time i used encrypt_secret.py i'm pretty certain i passed --url=https://zuul.openstack.org/ but maybe something has changed about that recently and we missed updating documentation... what does your command-line invocation look like? | 12:57 |
| *** roman_g has joined #openstack-infra | 12:58 | |
| fungi | er, not --url= just as the first positional argument | 12:58 |
| *** bobh has joined #openstack-infra | 12:59 | |
| *** quiquell|lunch is now known as quiquell | 13:00 | |
| *** jamesmcarthur has quit IRC | 13:04 | |
| *** boden has joined #openstack-infra | 13:07 | |
| *** yamamoto has joined #openstack-infra | 13:08 | |
| *** yamamoto has quit IRC | 13:08 | |
| *** yamamoto has joined #openstack-infra | 13:08 | |
| *** ykarel_ has joined #openstack-infra | 13:10 | |
| *** panda is now known as panda|lunch | 13:10 | |
| *** ykarel has quit IRC | 13:10 | |
| frickler | infra-root: please review https://review.openstack.org/614545 so that creating venvs on bridge.o.o will work without fancy tricks | 13:12 |
| *** yamamoto has quit IRC | 13:13 | |
| frickler | infra-root: I'd also still like to discuss how to resolve access to the all-clouds.yaml. do we want to add all infra-roots to the admin group and change group ownership of /etc/openstack to that? (c.f. https://docs.openstack.org/infra/system-config/sysadmin.html#accessing-clouds) | 13:13 |
| *** ccamacho has quit IRC | 13:14 | |
| *** shrasool has quit IRC | 13:14 | |
| Shrews | fungi: strigazi: tools/encrypt_secret.py --infile file_with_secret --tenant openstack https://zuul.openstack.org openstack/kolla | 13:14 |
| Shrews | wfm | 13:14 |
| *** ccamacho has joined #openstack-infra | 13:14 | |
| fungi | yeah, looks like it still uses the parameters i remember | 13:17 |
| fungi | it's possible https://docs.openstack.org/infra/manual/zuulv3.html#secret-variables would benefit from a sample invocation for our deployment which indicates the need for --tenant=openstack | 13:18 |
| fungi | my guess is that strigazi didn't pass that option | 13:18 |
| Shrews | well, that works for me w/o --tenant too | 13:21 |
| Shrews | my guess is strigazi is using "kolla" and not "openstack/kolla" for the project | 13:22 |
| Shrews | which results in a 404 | 13:22 |
| Shrews | admittedly confusing | 13:22 |
| fungi | oh, likely so | 13:22 |
| *** janki has quit IRC | 13:27 | |
| *** mriedem has joined #openstack-infra | 13:28 | |
| *** efried is now known as fried_rice | 13:35 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack-infra/project-config master: Add os_placement role to OpenStack-Ansible https://review.openstack.org/614896 | 13:40 |
| *** d0ugal has quit IRC | 13:40 | |
| *** lpetrut has joined #openstack-infra | 13:42 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual master: Give encryption example https://review.openstack.org/615189 | 13:44 |
| AJaeger | fungi, Shrews, strigazi , is this ok? ^ | 13:44 |
| *** panda|lunch is now known as panda | 13:48 | |
| *** eharney has joined #openstack-infra | 13:50 | |
| *** ykarel_ is now known as ykarel | 13:53 | |
| *** xarses_ has quit IRC | 13:54 | |
| *** jamesmcarthur has joined #openstack-infra | 13:55 | |
| fungi | frickler: regarding access to all-clouds.yaml, i was advocating for similarly granting admin group access to the ansible inventory so we could launch servers without sudo. there seemed to be a fair amount of opposition to that for some reason, and consensus was that we should just run all commands on bridge.o.o with sudo or from a root shell | 13:56 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack-infra/project-config master: Add os_placement role to OpenStack-Ansible https://review.openstack.org/614896 | 13:56 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Configure adns1.opendev.org via ansible https://review.openstack.org/614648 | 13:57 |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 13:57 |
| fungi | frickler: those advocating for avoiding explicit unix group management on the server seemed to feel a lot more passionate about it than i did, so i dropped it | 13:57 |
| mordred | corvus, clarkb, fungi: ^^ updated to take in to account logan- review comment | 13:57 |
| fungi | mordred: thanks! i was about to +2 and suggest a followup, but that works | 13:58 |
| *** tdasilva has joined #openstack-infra | 13:58 | |
| fungi | fwiw i agree with logan- we've seen that same exact issue elsewhere though in this particular case the servers we're initially deploying to have working global ipv6 addresses so it isn't a blocker | 13:58 |
| mordred | yah | 13:59 |
| fungi | (and i can't imagine us ever wanting to have v4-only nameservers anyway) | 13:59 |
| mordred | I figure we're waiting for people to awaken anyway - so wasn't too much of a thing | 13:59 |
| *** yamamoto has joined #openstack-infra | 13:59 | |
| frickler | fungi: hmm, o.k., but I really don't like advocating to run sudo in combination with osc from a private venv. not sure how to best amend the docs, then | 14:03 |
| *** armax has joined #openstack-infra | 14:06 | |
| fungi | frickler: yeah, it's not my first choice either. maybe if we get consensus on setting group ownership of all-clouds.yaml we can come to a similar consensus on reversing the decision about the ansible inventory cache | 14:07 |
| mordred | fungi, fungi: I'd be in support of group management | 14:07 |
| fungi | it's less of a security measure to me and more avoiding gratuitous use of sudo which could lead to accidents | 14:07 |
| fungi | if i don't prefix commands with `sudo` then the files on the server i can accidentally remove or overwrite are extremely limited | 14:08 |
| frickler | I agree. o.k., let me try to come up with a patch for that | 14:09 |
| fungi | you might want to avoid making that group "admin" though. it was a particular hot-button group name which is why it got removed in the first place if memory serves | 14:10 |
| fungi | ubuntu used to grant sudo access to members of the admin group, but deprecated it in favor of the sudo group, so to some that seems to mean we shouldn't have a group named admin | 14:11 |
| *** rfolco is now known as rfolco|off | 14:11 | |
| guilhermesp | hello all! AJaeger :) could you review these please https://review.openstack.org/#/c/615187/2 https://review.openstack.org/#/c/614896/5 | 14:13 |
| *** dansmith is now known as SteelyDan | 14:13 | |
| frickler | fungi: actually I've been thinking about just re-using the sudo group instead of creating/setting up another one that would just duplicate membership there. | 14:16 |
| frickler | though for some reason we keep the default ubuntu account a member of that, maybe we need to drop that? | 14:17 |
| frickler | I could not find any code that touches/updates /etc/openstack, would it be o.k. to just change the group for that manually once? | 14:18 |
| mordred | frickler, fungi: maybe in our base.yaml playbook we should remove the ubuntu user | 14:18 |
| gary_perkins | Hi! I've redeployed our Arm cloud and fixed the outstanding issues: https://review.openstack.org/#/c/602436 Can I PM you new credentials, ianw? Would appreciate if people can review when they get a dull moment :) | 14:19 |
| fungi | mordred: is that an artifact of using cloud-provided images? | 14:19 |
| mordred | frickler: it's in playbooks/roles/configure-openstacksdk/tasks/main.yaml | 14:19 |
| mordred | fungi: yes | 14:19 |
| mordred | fungi: it's the user that has blanket sudo access - because somehow that's better than allowing remote root logins | 14:20 |
| *** shrasool has joined #openstack-infra | 14:20 | |
| fungi | gary_perkins: ianw is in au so likely already wound down for the day. i'm happy to get the updated credentials if you can get them to me | 14:20 |
| gary_perkins | fungi: Thanks :) | 14:21 |
| mordred | frickler: and, in fact, you could likely just change the default for openstacksdk_config_group in playbooks/roles/configure-openstacksdk/defaults/main.yaml | 14:21 |
| openstackgerrit | Merged openstack-infra/system-config master: Make the pip3 role really install something https://review.openstack.org/614545 | 14:21 |
| fungi | mordred: yeah, i've always found that absurd as well. it's basically to shut up orange book obsessives who insist "root" logins must be disabled for a secure system | 14:21 |
| mordred | fungi: YOU MUST FOLLOW THE ARBITRARY RULES TO BE GOOD | 14:21 |
| frickler | mordred: hmm, I was thinking only to override it for bridge.yaml, since the role seems to be used for nodepool deployments, too | 14:22 |
| mordred | frickler: ah - fair | 14:22 |
| mordred | frickler: well, luckily we're already even passing parameters to that role in playbooks/base.yaml | 14:23 |
| *** lpetrut has quit IRC | 14:26 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Adjust query for bug 1800472 https://review.openstack.org/615194 | 14:27 |
| openstack | bug 1800472 in OpenStack Compute (nova) "nova.tests.functional.test_server_group.ServerGroupTestV264.test_boot_servers_with_affinity_no_valid_host intermittently failing with "OpenStackApiNotFoundException: Item not found"" [Medium,Triaged] https://launchpad.net/bugs/1800472 | 14:27 |
| *** d0ugal has joined #openstack-infra | 14:27 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Consume rate limiting task manager from openstacksdk https://review.openstack.org/612169 | 14:27 |
| corvus | fungi, frickler, mordred: i'm not *opposed* to group management, i'm just not in favor of it. that sort of thing is always complex and takes a lot of time and maintenance for almost no gain on a server where every action is effectively a root action. i was hoping to save us a bunch of time. | 14:28 |
| *** lpetrut has joined #openstack-infra | 14:30 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/system-config master: Fix access to clouds on bridge https://review.openstack.org/615197 | 14:32 |
| frickler | corvus: fungi: mordred: that does look simple enough to me ^^ | 14:32 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/jeepyb master: Make use of Gerrit CLI to retrieve group UUID https://review.openstack.org/284843 | 14:33 |
| corvus | frickler: oh, yeah, i was thinking in context of the previous discussion where running ansible without sudo was contemplated. | 14:34 |
| fungi | corvus: yeah, but also running openstackclient without sudo | 14:34 |
| corvus | frickler: i think read access to to clouds.yaml is a great idea. that will let us do server lists, etc, easily. we won't be able to launch new nodes or run ansible though.. | 14:34 |
| frickler | corvus: that would possibly be a second step, indeed, but I'd be fine with the first one for now | 14:35 |
| corvus | frickler: the second step is where several more things on the filesystem need to be group-writeable -- and stay group-writable even though automatic processes are maintaining them. | 14:36 |
| *** ccamacho has quit IRC | 14:36 | |
| openstackgerrit | Merged openstack-infra/git-review master: Remove auto-branch name https://review.openstack.org/610573 | 14:37 |
| *** lpetrut has quit IRC | 14:37 | |
| frickler | corvus: yeah, that would be tricky | 14:38 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 14:39 |
| *** quiquell is now known as quiquell|off | 14:45 | |
| *** ccamacho has joined #openstack-infra | 14:53 | |
| *** coolsvap has quit IRC | 14:54 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Give encryption example https://review.openstack.org/615189 | 14:57 |
| frickler | wow, canonical is getting quite fancy it seems ... cosmic ... disco ... ;) | 14:58 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 14:58 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Remove start_services https://review.openstack.org/614961 | 14:59 |
| * fungi turns on the mirror ball and starts to groove | 14:59 | |
| *** diablo_rojo has joined #openstack-infra | 15:02 | |
| *** xek has joined #openstack-infra | 15:05 | |
| fungi | mordred: i'm a bit confused by the seemingly arbitrary use of globbing and explicit lists of nearly-identical hostnames in 602385. is there some reason i'm overlooking or is that just an opportunity for cleanup in the future? | 15:05 |
| *** xek__ has quit IRC | 15:06 | |
| strigazi | Shrews: fungi AJaeger this works: ./tools/encrypt_secret.py --tenant=openstack https://zuul.openstack.org/ openstack/<myproject> --infile <file> ; Thank you ; I was trying like this ./tools/encrypt_secret.py https://zuul.openstack.org <myproject> | 15:06 |
| strigazi | Shrews: fungi AJaeger And I was getting http://paste.openstack.org/show/734047/ | 15:07 |
| *** ssbarnea has quit IRC | 15:08 | |
| *** ssbarnea has joined #openstack-infra | 15:10 | |
| fungi | strigazi: yeah, in this case the confusion is that openstack/ is actually part of the name of the project | 15:11 |
| fungi | e.g., openstack/kolla is a project name as far as zuul (and gerrit, and storyboard and whatever else) is concerned | 15:12 |
| fungi | kolla by itself is not | 15:12 |
| fungi | just like openstack-infra/bindep or openstack-dev/pbr are project names | 15:13 |
| Shrews | git.openstack.org/openstack/kolla would also be valid | 15:13 |
| *** chandankumar is now known as chkumar|off | 15:14 | |
| *** munimeha1 has joined #openstack-infra | 15:14 | |
| mordred | fungi: I think it's just an opportunity for cleanup in the future | 15:16 |
| fungi | okay, just wanted to be sure i understood the change. the lack of consistency made me worry there was something i was overlooking. thanks! | 15:16 |
| strigazi | Shrews: fungi this works too "curl https://zuul.openstack.org/api/key/openstack/kolla.pub" | 15:16 |
| strigazi | Shrews: fungi thanks again for your time | 15:17 |
| mordred | fungi: I tnik it was some combination of me getting bored with typing - and also not being sure which of the approaches (globs vs lists) we liked better | 15:17 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Adjust query for bug 1800472 https://review.openstack.org/615194 | 15:18 |
| openstack | bug 1800472 in OpenStack Compute (nova) "nova.tests.functional.test_server_group.ServerGroupTestV264.test_boot_servers_with_affinity_no_valid_host intermittently failing with "OpenStackApiNotFoundException: Item not found"" [Medium,Triaged] https://launchpad.net/bugs/1800472 | 15:18 |
| fungi | mordred: great--this way we get to try both! ;) | 15:18 |
| fungi | strigazi: glad it's working for you | 15:19 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Merger: automatically add new hosts to the known_hosts file https://review.openstack.org/608453 | 15:19 |
| arxcruz | mtreinish: hey around? I'm getting some weird tempest results, hope you can help me, at some point all scenarios are being executed in one single worker while the other workers are idle | 15:20 |
| arxcruz | mtreinish: http://logs.openstack.org/33/615133/2/check/tripleo-ci-centos-7-standalone/9ac46cc/logs/stackviz/#/testrepository.subunit/timeline | 15:20 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP: support foreign required-projects https://review.openstack.org/613143 | 15:22 |
| fungi | strigazi: would https://review.openstack.org/615189 have helped avoid your confusion at all? | 15:24 |
| *** dave-mccowan has joined #openstack-infra | 15:26 | |
| *** gyee has joined #openstack-infra | 15:26 | |
| *** ramishra has quit IRC | 15:29 | |
| clarkb | finucannot: I'm sending email now to formalize the "lets keep git-review stable" and what that means (corvus had a great one liner for that). So look out for email shortly | 15:30 |
| finucannot | clarkb: Yup, got that and agreed. FWIW, I'm for _removing_ extraneous features (auto-configuration of topics) where possible | 15:34 |
| *** Swami has joined #openstack-infra | 15:34 | |
| clarkb | mordred: fungi we should watch bridge.o.o closely when https://review.openstack.org/#/c/602385/ merges | 15:35 |
| fungi | clarkb: i agree. we have testing there, but as we saw with things like misapplication of firewall rules having changes to how hostnames get mapped can result in some surprising problems | 15:36 |
| *** eharney has quit IRC | 15:36 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Move operations-guide translations to project-config https://review.openstack.org/615222 | 15:39 |
| mordred | clarkb: I've opened a root shell on bridge, just in case | 15:39 |
| fungi | clarkb: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=115&rra_id=all is making me think there's been some automated process hammering the etherpad server since ~2018-10-16/17 and someone just turned it off around 22:15z | 15:39 |
| clarkb | fungi: https://review.openstack.org/#/c/614883/ sort of | 15:40 |
| clarkb | fungi: I think ^ was the culprit all along (with the broken web server config) | 15:40 |
| clarkb | fungi: basically all our browsers had to poll etherpad and that created sadness | 15:40 |
| fungi | aha! | 15:41 |
| clarkb | fungi: it wasn't until I had asserted "we use websockets" to TheJulia that I realized I can actually check that using browser debugging tools and then saw the websocket urls were 400ing and we were doing many requests wthat had transport=poll in them that it hit me | 15:41 |
| fungi | yikes | 15:41 |
| fungi | i had already checked out by then, i think | 15:41 |
| clarkb | I manually applied that change before it merged since I had TheJulia around helping test things | 15:42 |
| clarkb | and that seemed to make it happy at that point. Good to see cacti seems to agree | 15:42 |
| clarkb | mordred: great. I'm still booting my morning but assume I'm actually here in the next half hour or so | 15:44 |
| clarkb | fungi: thank you for helping gary_perkins | 15:45 |
| openstackgerrit | Merged openstack-infra/system-config master: Add yamlgroup inventory plugin https://review.openstack.org/602385 | 15:46 |
| *** AJaeger has quit IRC | 15:46 | |
| mordred | clarkb: yes. I am here | 15:46 |
| openstackgerrit | Merged openstack-infra/system-config master: Add unittest for yamlgroup inventory plugin https://review.openstack.org/614694 | 15:47 |
| *** AJaeger has joined #openstack-infra | 15:48 | |
| *** xek_ has joined #openstack-infra | 15:48 | |
| *** eharney has joined #openstack-infra | 15:51 | |
| *** xek has quit IRC | 15:51 | |
| fungi | clarkb: yeah, no problem. i'm only just starting on resetting the passwords and getting them into our usual list | 15:51 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove puppet config for opendev nameservers https://review.openstack.org/614869 | 15:54 |
| clarkb | looks like gra1 is happy again | 15:57 |
| *** jaosorior has quit IRC | 15:58 | |
| *** bnemec is now known as beekneemech | 16:00 | |
| *** yamamoto has quit IRC | 16:03 | |
| *** yamamoto has joined #openstack-infra | 16:05 | |
| *** yamamoto has quit IRC | 16:05 | |
| *** yamamoto has joined #openstack-infra | 16:05 | |
| clarkb | I need to start putting together the infra project update talk as well as an onboarding session. For the project update I think big changes have been the ansiblification and work to use containers. We've also added a new arm64 region and are in the process of adding a third. On the zuul side of things we have been (helpful I hope) beta testers. We've also onboarded new top level projects (starlingx | 16:07 |
| clarkb | and airship) | 16:07 |
| clarkb | any other big ticket items I'm forgetting that should be called out? | 16:07 |
| mordred | clarkb: I think that's about it | 16:08 |
| clarkb | for onboarding in particular I was thinking about changing tactics a bit and targetting top level projects in partciular. I need to see what I can learn about how is attending and who can go to the onboarding session but more 1:1 type time with interested individuals might be helpful? | 16:10 |
| clarkb | dtroyer: ^ do you have any sense for whether or not that could be useful for starlingx in berlin? | 16:10 |
| *** yamamoto has quit IRC | 16:10 | |
| fungi | interesting... etherpad is working on a new default theme for the next release https://github.com/ether/etherpad-lite/issues/3441 | 16:10 |
| fungi | clarkb: opendev is a big ticket item | 16:11 |
| clarkb | fungi: ya, though I think it is still a bit in the air as far as how much we will be talking about it? maybe we should just go for it at this point | 16:11 |
| clarkb | lsell has feedback on my email too so hopefully we can get that and the message/faq sorted soonish | 16:12 |
| fungi | even though we're really just getting started on the implementation of the changes it implies, it's probably worth calling out | 16:12 |
| fungi | given people are already talking about it and using the name whether we want them to or not, so not at least mentioning it could seem a bit weird | 16:13 |
| *** AJaeger_ has joined #openstack-infra | 16:13 | |
| clarkb | fair point | 16:13 |
| AJaeger_ | clarkb: you can also point out that with the python3-first goal, most of the jobs migrated from central place to in-repo. | 16:14 |
| mordred | clarkb: like user onboarding rather than trying to onboard new admins? | 16:14 |
| fungi | oh, yep that's a fairly big change | 16:14 |
| AJaeger_ | clarkb: you could give a summary on what is placed in-repo - and what in the central repo. | 16:15 |
| *** imacdonn has quit IRC | 16:15 | |
| *** AJaeger has quit IRC | 16:15 | |
| *** imacdonn has joined #openstack-infra | 16:15 | |
| clarkb | mordred: ya | 16:16 |
| clarkb | AJaeger_: ++ thanks | 16:16 |
| corvus | fungi: if you zoom in, the document gets wider | 16:16 |
| fungi | interesting | 16:17 |
| *** pcaruana has quit IRC | 16:17 | |
| fungi | not sure i like that much better, but at least you can zoom it so the editing space is as wide as the window | 16:18 |
| corvus | fungi: so basically, it's a fixed number of characters wide, as compared to the current system which lets you change size and line length independently. | 16:18 |
| fungi | i'll miss being able to alter the display size of the text independently of the width of the editing box | 16:18 |
| fungi | yes, that | 16:18 |
| clarkb | it looks more google docs | 16:18 |
| dtroyer | clarkb: that would be useful, there will be a small number of stx devs in Berlin that don't have much, if any, OpenStack background. | 16:18 |
| fungi | clarkb: yes, i don't consider that a plus | 16:19 |
| dtroyer | and some of that do could probably make sure we are up-to-date :) | 16:19 |
| AJaeger_ | dtroyer: could you review https://review.openstack.org/615174 , please? | 16:19 |
| *** AJaeger_ is now known as AJaeger | 16:20 | |
| fungi | AJaeger_: if we resume work on https://review.openstack.org/578557 would you still expect the same set of jobs to be kept in project-config? | 16:20 |
| dtroyer | AJaeger_: sure… I'm getting a slow start today, didn't realize ildiko had that up already | 16:20 |
| fungi | AJaeger: once that's no longer a necessary workaround for matching tag-triggered jobs against specific branches? | 16:20 |
| AJaeger | fungi: I think in that case we can move publish-to-pypi etc. in-repo | 16:20 |
| AJaeger | fungi: agreed. | 16:21 |
| fungi | got it. wasn't sure if there were additional reasons for centralizing some of those | 16:21 |
| AJaeger | fungi: other reason was that release team had no way to query that proper release jobs are set up | 16:21 |
| AJaeger | fungi: zuul is working on an API - once that is in, the publish-to-pypi jobs can move ... | 16:22 |
| AJaeger | (and it needs a change to release tools - it might be that this change is already done) | 16:22 |
| AJaeger | fungi, I suggest to rename publish-to-pypi-python3 to publish-to-pypi - I don't think we need both jobs and tmplates. dhellmann, do you agree? | 16:23 |
| AJaeger | fungi: so, I would do that first before we move these in-repo | 16:23 |
| clarkb | dtroyer: ok let me try to formalize this a bit more but if I get an etherpad together maybe you can circulate that with stx folks and I'll reach out to airhsip and kata et al too | 16:23 |
| dtroyer | clarkb: that sounds good, will do | 16:24 |
| mordred | clarkb: the yamlgroup update on bridge did not go well - we missed a sequence somewhere | 16:25 |
| clarkb | mordred: ok, did it just fail to run at all? | 16:25 |
| mordred | clarkb: basically, the new groups.yaml file got put down before the new ansible.cfg telling ansible to load the yamlgroup plugin | 16:25 |
| mordred | yeah - totally faile to run - looking to see how much is in place | 16:25 |
| mordred | clarkb: OH - duh | 16:27 |
| mordred | clarkb: so - the issue is that we serve our inventory files directly from /opt/system-config/inventory/groups.yaml | 16:28 |
| mordred | so when system-config got updated by the git pull, the new groups.yaml content was immediately in place | 16:28 |
| mordred | but ansible had not yet run to update ansible.cfg or copy the plugin in to place | 16:29 |
| mordred | clarkb: do you think we should revert the yamlgroup patch and re-add it in a sequence that will run - or just fix in place for now and learn for next time? | 16:29 |
| clarkb | mordred: probably fix in place? | 16:30 |
| clarkb | that seems less error prone | 16:30 |
| mordred | yeah. and this isn't a thing we do very often | 16:30 |
| *** fried_rice is now known as fried_rolls | 16:30 | |
| fungi | seems like an okay fix to me for now | 16:31 |
| mordred | ok. I copied the yamlgroup.py and ansible.cfg files into place by hand | 16:32 |
| mordred | you know - for the future - we could change the execution of the bridge.yaml playbook to use its own ansible.cfg that doesn't load the dynamic inventory at all | 16:33 |
| *** shrasool has quit IRC | 16:33 | |
| *** e0ne has quit IRC | 16:33 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use python3 for release https://review.openstack.org/615236 | 16:34 |
| mordred | AJaeger: your commit message doesn't seem to match the content of the patch | 16:36 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Sync publish-to-pypi templates https://review.openstack.org/615237 | 16:38 |
| AJaeger | mordred: let me expand the commit message... | 16:38 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use python3 for release https://review.openstack.org/615236 | 16:39 |
| AJaeger | mordred: updated ^ | 16:39 |
| AJaeger | config-core, could you review https://review.openstack.org/615222 and https://review.openstack.org/615174 , please | 16:41 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use publish-to-pypi everywhere https://review.openstack.org/615239 | 16:43 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove publish-to-pypi-python3 https://review.openstack.org/615241 | 16:44 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use publish-to-pypi everywhere https://review.openstack.org/615239 | 16:46 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove python3 release jobs https://review.openstack.org/615242 | 16:46 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove publish-to-pypi-python3 https://review.openstack.org/615241 | 16:46 |
| AJaeger | mordred, dhellmann, smcginnis, could you review the stack starting at https://review.openstack.org/615236, please? This is a bit back and forth between the two repos... | 16:47 |
| smcginnis | Sure, I'll take a look. | 16:47 |
| *** ykarel is now known as ykarel|away | 16:48 | |
| AJaeger | smcginnis: I mainly need feedback on whether using the python3 variant is fine for *all* repos. | 16:48 |
| AJaeger | The essence of the change is "mv publish-to-pypi-python3 publish-to-pypi", so making publish-to-pypi using the set up of publish-to-pypi-python3 - and then removing publish-to-pypi-python3 | 16:49 |
| smcginnis | I think it was needed for transition to make sure everything was working right under python3, but now that we've established that fact, I don't see any reason why we can't just use one updated job everywhere. | 16:49 |
| AJaeger | agreed | 16:50 |
| smcginnis | AJaeger: That first one says it's changing release-openstack-python, but looks like it is only adding test-release-openstack. Was that missed, or refactored after the commit message was written to break things up? | 16:51 |
| * AJaeger checks | 16:52 | |
| smcginnis | AJaeger: I don't see release-openstack-python being updated anywhere to have release_python: python3. | 16:52 |
| AJaeger | smcginnis: indeed - thanks | 16:52 |
| * AJaeger fixes | 16:53 | |
| *** zul has quit IRC | 16:53 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Sync publish-to-pypi templates https://review.openstack.org/615237 | 16:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove publish-to-pypi-python3 https://review.openstack.org/615241 | 16:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use python3 for release https://review.openstack.org/615236 | 16:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use publish-to-pypi everywhere https://review.openstack.org/615239 | 16:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove python3 release jobs https://review.openstack.org/615242 | 16:55 |
| AJaeger | smcginnis: updated and rebased | 16:56 |
| AJaeger | thanks, smcginnis | 16:56 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: add tc meetings https://review.openstack.org/608682 | 16:58 |
| *** yamamoto has joined #openstack-infra | 16:59 | |
| *** Swami has quit IRC | 17:01 | |
| *** yamamoto has quit IRC | 17:07 | |
| *** ykarel_ has joined #openstack-infra | 17:10 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add os_placement role to OpenStack-Ansible https://review.openstack.org/614896 | 17:11 |
| finucannot | mordred: Would you do me the honor? https://review.openstack.org/#/c/610988/ | 17:12 |
| clarkb | is etherpad unresponsive for anyone else? | 17:13 |
| clarkb | I can hit it via ssh and apache and nodejs are running | 17:13 |
| clarkb | but firefox says it cannot connect | 17:13 |
| *** ykarel|away has quit IRC | 17:13 | |
| clarkb | aha | 17:13 |
| clarkb | we broke iptables | 17:13 |
| clarkb | mordred: ^ is this group fallout | 17:13 |
| clarkb | ya I see the bug | 17:14 |
| mwhahaha | breaking stuff on a friday :( i assume :8080 mirror failures are related? | 17:16 |
| fungi | mwhahaha: what are you doing working on a friday? sheesh! ;) | 17:16 |
| mwhahaha | i know right? | 17:16 |
| clarkb | mwhahaha: maybe? | 17:16 |
| mwhahaha | no one else does | 17:16 |
| fungi | and no, i don't think we knew about mirror problems yet, so thanks! | 17:17 |
| clarkb | can someone else look at the mirrors I'm sorting out etherpad | 17:17 |
| fungi | yeah, i'm looking now | 17:17 |
| fungi | yep, we're blocking 8080 with the iptables rules on the mirror servers | 17:17 |
| fungi | now to figure out why | 17:18 |
| clarkb | fungi: likely the groups change | 17:18 |
| fungi | right, that much i'm almost sure of, just looking to figure out what it did wrong | 17:19 |
| *** dtantsur is now known as dtantsur|afk | 17:19 | |
| fungi | playbooks/group_vars/mirror.yaml has 80, 8080, 8081 and 8082 i its iptables_extra_public_tcp_ports list | 17:20 |
| fungi | guessing these hosts didn't match the host pattern for the mirror group | 17:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Update etherpad group membership https://review.openstack.org/615250 | 17:20 |
| clarkb | fungi: that is a good guess ^ essentially the same problem with etherpad | 17:21 |
| *** trown is now known as trown|lunch | 17:21 | |
| fungi | yeah, i was just pulling it up to see what you fixed ;) | 17:21 |
| clarkb | infra-root ^ can we get reviews on that please | 17:21 |
| fungi | are these globs or regular expressions? | 17:21 |
| clarkb | fungi: globs | 17:21 |
| fungi | so so why didn't that match? | 17:22 |
| fungi | if they're globs then what you patched would just be a subset of what was there, right? | 17:22 |
| *** jpich has quit IRC | 17:22 | |
| clarkb | fungi: not under the webservers group | 17:23 |
| fungi | oh! | 17:23 |
| clarkb | fungi: and the webservers group is what writes out the port 80 and 443 iptables rules | 17:23 |
| fungi | yep, i didn't look far enough down | 17:23 |
| clarkb | the first change in the diff is to rpevent etherpad-dev from using etherpad's cert and database | 17:23 |
| clarkb | (I'm going to try not to think about what that would break if it happened) | 17:23 |
| clarkb | (I actually think we still have hostvars in place which take precedence over the group vars so don't think that would've actually happened) | 17:24 |
| clarkb | mwhahaha: have a specific example? I'm reading our groups and the mirrors look right to me | 17:25 |
| mwhahaha | http://logs.openstack.org/02/610102/19/check/tripleo-ci-centos-7-containers-multinode-pike/2e37e64/job-output.txt.gz#_2018-11-02_17_00_08_476384 | 17:25 |
| mwhahaha | it's failing at the end, if i curl it it's giving me a No route to host | 17:25 |
| mwhahaha | which i assume is iptables reject | 17:25 |
| mwhahaha | i noticed it in vexxhost and there was aa different failure in inap | 17:26 |
| mwhahaha | so i don't think it's regions pecific | 17:26 |
| fungi | clarkb: here's the openstack-INPUT chain from the limestone mirror for example: http://paste.openstack.org/show/734048/ | 17:26 |
| clarkb | ya its definitely not allowing 8080 | 17:26 |
| mwhahaha | and it just started | 17:26 |
| fungi | we have duplicate rules for 22/tcp (strangely) as well as rules for 80/tcp and 443/tcp but not 8080,8081,8082 | 17:27 |
| fungi | it's like it didn't actually apply the mirror group | 17:27 |
| *** ccamacho has quit IRC | 17:27 | |
| clarkb | fungi: ya | 17:28 |
| fungi | we have mirror[0-9]*.*.*.openstack.org in the webservers group | 17:28 |
| fungi | which i think explains the 80/443 rules | 17:28 |
| *** shrasool has joined #openstack-infra | 17:28 | |
| clarkb | and specific names in the mirror group | 17:28 |
| fungi | mordred: ^ extra eyes would be helpful here, we're breaking a *lot* of ci jobs right now | 17:29 |
| fungi | i confirmed the limestone entry in the mirror group, for example, matches the hostname | 17:29 |
| clarkb | fnmatch.fnmatch('mirror01.bhs1.ovh.openstack.org', 'mirror01.bhs1.ovh.openstack.org') returns True which is what I think the yamlgroup inventory plugin will run | 17:29 |
| fungi | we could try switching to the glob used for applying afs-client and webserver to them | 17:30 |
| clarkb | ya | 17:30 |
| *** mriedem has quit IRC | 17:30 | |
| clarkb | I'd like to udnerstand that though to better understand what else might be broken | 17:31 |
| fungi | i concur | 17:31 |
| clarkb | the zookeeper group for example has all of its iptables rules and is also explicitly listed | 17:31 |
| fungi | if i `sudo ansible --list-hosts mirror` i get a list of the mirror servers i expect too | 17:33 |
| fungi | [WARNING]: Unable to parse /etc/ansible/hosts/emergency.yaml as an inventory | 17:33 |
| fungi | source | 17:33 |
| fungi | i do see that | 17:33 |
| clarkb | uh | 17:33 |
| clarkb | I'm guessing that means we are going to try and run stuff on disabled hosts? | 17:34 |
| fungi | and /etc/ansible/hosts/emergency.yaml is indeed an empty file | 17:34 |
| fungi | er, nonexistent file | 17:34 |
| fungi | but /etc/ansible/hosts/emergency is still there | 17:34 |
| fungi | so it didn't get renamed | 17:35 |
| *** mriedem has joined #openstack-infra | 17:35 | |
| clarkb | ah ok if we read the old emergency file then maybe we are ok | 17:35 |
| *** ralonsoh has quit IRC | 17:35 | |
| clarkb | fungi: can you try a /opt/system-config/tools/kick.sh mirror01.bhs1.ovh.openstack.org | 17:35 |
| clarkb | that might give us more insight into why it isn't applying the rules despite being in the right group | 17:35 |
| fungi | yeah, `sudo ansible --list-hosts disabled` does still show the stuff from the old file at least | 17:35 |
| *** panda is now known as panda|off | 17:36 | |
| fungi | running in a root screen session now | 17:36 |
| fungi | no errors from the iptables playbook tasks | 17:37 |
| clarkb | and yet no port 8080 rule | 17:37 |
| clarkb | fungi: also maybe we should disable teh ansible puppet cron until we understand this? | 17:39 |
| fungi | i don't know what that buys us at this point since whatever it was going to do it's already done | 17:39 |
| clarkb | zuul-executor group also defines iptables rules in playbooks/group_vars. These rules are applied to ze01. This group is also defined by listing all of the nodes | 17:41 |
| clarkb | I'm stumped on what makes the mirror nodes different | 17:42 |
| clarkb | maybe the asnible cache stuff will tell us | 17:42 |
| clarkb | "name": "mirror01.bhs1.ovh.openstack.org", looks correct to me | 17:44 |
| fungi | is /etc/puppet/hieradata/production/group_vars still where the group vars files are written out on the servers now? | 17:47 |
| clarkb | fungi: no, /etc/ansible/hosts/group_vars is the location | 17:48 |
| clarkb | oh sorry ^ is the source. the path you have is the right destiation | 17:48 |
| fungi | yeah, there's no /etc/ansible on the servers | 17:48 |
| fungi | thinking we should probably roll back that stack of changes if we can't find a smoking gun shortly | 17:50 |
| clarkb | agreed. I'm not really getting anywhere trying to debug this mirror group listing | 17:51 |
| fungi | here's an extra bit of crazy for you... | 17:51 |
| clarkb | all of the ansible data looks as I expect it and if you feed that to fnmatch you get a match which should put the hsots in the mirror group which should apply the 8080 port firewall rules | 17:52 |
| fungi | /etc/iptables/rules has the old rules in it. not a symlink to /etc/iptables/rules.v4 (which has the new rules) | 17:52 |
| fungi | it's definitely not copying the group_vars/mirror.yaml to the server | 17:53 |
| fungi | but ansible itself says the servers are members of the mirror group | 17:53 |
| fungi | though it's also not putting the webservers.yaml in there yet we're getting webserver ports opened | 17:54 |
| clarkb | fungi: I don't think group_vars/mirror.yaml is a valid hieradata file | 17:54 |
| clarkb | fungi: we only copy the private data that way | 17:54 |
| fungi | ahh | 17:54 |
| clarkb | so new theory it is in the group properly but we are misapplying the files? | 17:54 |
| fungi | as in not applying the group_vars file for the mirror group in particular? | 17:55 |
| clarkb | or applying it wrong? rules.v4 is the file we write and it looks like we did update it on bhs1 | 17:55 |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/system-config master: Fix access to clouds on bridge https://review.openstack.org/615197 | 17:55 |
| clarkb | oh v4 lacks the 8080 rule, now I understand | 17:56 |
| fungi | yeah, i think /etc/iptables/rules became a transitional symlink between precise and trusty and we could just delete that file off our servers at this point | 17:57 |
| fungi | i don't think it's the problem | 17:57 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/yaml2ical master: update the versions of python 3 claimed https://review.openstack.org/615266 | 17:57 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/yaml2ical master: add base class for recurrence https://review.openstack.org/615267 | 17:57 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/yaml2ical master: add day_specifier to recurrence https://review.openstack.org/615268 | 17:57 |
| fungi | ansible is updating /etc/iptables/rules.v4 (and .v6) and that's what iptables-persistent is applying | 17:57 |
| clarkb | ya | 17:58 |
| fungi | is it possible ansible is overriding iptables_extra_public_tcp_ports from mirror with the one from webservers rather than merging them? | 17:59 |
| clarkb | fungi: ya that is what I'm wondering, we combine things as part of the all group | 17:59 |
| clarkb | my current hunch is that that combination is what fails | 17:59 |
| clarkb | possibly due to what you describe | 17:59 |
| fungi | if it's going alphabetically, then zuul is overriding webservers for the executors you were looking at | 18:00 |
| clarkb | ya though exectuors don't run a webserver ? | 18:00 |
| fungi | oh, right | 18:00 |
| fungi | the zuul-scheduler group respecifies 80 and 443 in its iptables_extra_public_tcp_ports list | 18:01 |
| clarkb | webservers: inventory_hostname is match('(grafana\d*|health\d*|graphite\d*|groups\d*|groups-dev\d*|eavesdrop\d*|paste\d*|ethercalc\d+|etherpad\d*|etherpad-dev\d*|files\d*|refstack\d*|static\d*|status\d*|survey\d+|nodepool|nl\d+|nb\d+|zm\d+|ask|ask-staging|translate.*|codesearch\d*|cacti\d+|wiki.*|storyboard.*|openstackid-dev|planet\d*)\.openstack\.org|openstackid.org') is the old webservers list | 18:01 |
| clarkb | I think you are right. webservers is meant to be mutually exclusive to any other iptables modifications | 18:02 |
| clarkb | mirrors were not listed in ^ previously | 18:02 |
| *** derekh has quit IRC | 18:02 | |
| fungi | patch on the way | 18:02 |
| clarkb | I believe the fix is to remove mirrors from webservers | 18:02 |
| clarkb | and ya mirror lists port 80 too so doesn't need to be in webservers | 18:03 |
| clarkb | fungi: you should probably put your change under mine as we need working mirrors to test my change? | 18:03 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/irc-meetings master: use python 3 to build the site https://review.openstack.org/615269 | 18:04 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/irc-meetings master: add day_specifier from recurrence https://review.openstack.org/615270 | 18:04 |
| clarkb | oh its only the non standard ports that are affected | 18:04 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Remove mirrors from the webservers group https://review.openstack.org/615271 | 18:04 |
| clarkb | so no need to rebase | 18:04 |
| fungi | we should likely force merge those both | 18:04 |
| clarkb | also how does this look #status notice OpenStack infra's mirror nodes stopped accepting connections on ports 8080, 8081, and 8082. We will notify when this is fixed and jobs can be rechecked if they failed to communicate with a mirror on these ports. | 18:05 |
| fungi | lgtm | 18:06 |
| fungi | or we could use status alert | 18:06 |
| fungi | and then status ok | 18:06 |
| AJaeger | dhellmann: could you review https://review.openstack.org/615236 , please? | 18:06 |
| dhellmann | AJaeger : looking | 18:06 |
| clarkb | fungi: I've approved your fix. I agree you should probably force merge or direct enqueue to the gate | 18:06 |
| fungi | clarkb: i've emergency approved yours as well | 18:06 |
| AJaeger | thanks, dhellmann. Backscroll has some more details... | 18:06 |
| corvus | i've +2d both after double checking against the original yamlgroup change | 18:07 |
| fungi | i'll bypass gating for both and then we can rerun against the mirror servers ahead of schedule | 18:07 |
| dhellmann | AJaeger : why not just delete the old one and rename the new one? I'll read scrollback | 18:07 |
| fungi | thanks corvus! | 18:07 |
| AJaeger | dhellmann: we cannot - as long as a job is used, zuul does not allow to remove it... | 18:07 |
| clarkb | #status notice OpenStack infra's mirror nodes stopped accepting connections on ports 8080, 8081, and 8082. We will notify when this is fixed and jobs can be rechecked if they failed to communicate with a mirror on these ports. | 18:07 |
| openstackstatus | clarkb: sending notice | 18:07 |
| AJaeger | dhellmann: I hope I didn't complicate things... | 18:07 |
| dhellmann | AJaeger : ok, that's probably why I just added the new one in the first place | 18:08 |
| *** betherly has quit IRC | 18:08 | |
| AJaeger | dhellmann: part was also to test that it didn't break anything | 18:08 |
| clarkb | fungi: the steps for running that on bridge will be to update /opt/system-config to the merged state then kick.sh. I'm happy to help with that | 18:08 |
| dhellmann | I only updated the official projects, and didn't include any infra projects IIRC | 18:08 |
| -openstackstatus- NOTICE: OpenStack infra's mirror nodes stopped accepting connections on ports 8080, 8081, and 8082. We will notify when this is fixed and jobs can be rechecked if they failed to communicate with a mirror on these ports. | 18:09 | |
| AJaeger | dhellmann: I know - the end result of that series is that publish-to-pypi-python3 gets renamed to publish-to-pypi and all repos use it. | 18:09 |
| dhellmann | yeah, I'm still reading | 18:09 |
| dhellmann | I think this is probably ok, but I wasn't sure of that when we started | 18:10 |
| *** abishop has joined #openstack-infra | 18:10 | |
| openstackstatus | clarkb: finished sending notice | 18:11 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove mirrors from the webservers group https://review.openstack.org/615271 | 18:11 |
| clarkb | after lunch I'm thinking I'll go back through the conversion and possibly propose more simplifications. Like we shouldn't need to explicitly list all of the mirrors that way (whcih will add an additional step for anyone replacing mirrors) | 18:11 |
| openstackgerrit | Merged openstack-infra/system-config master: Update etherpad group membership https://review.openstack.org/615250 | 18:12 |
| dhellmann | AJaeger : wow, zuul left a lot of repeated comments on https://review.openstack.org/#/c/615241/3 | 18:12 |
| clarkb | corvus: related to ^ any idea why zuul mergers are a web server? | 18:13 |
| clarkb | corvus: hold over from zuulv2 maybe? | 18:13 |
| AJaeger | dhellmann: yeah - it does not do speculatoin if project-config is involved ;( So, will need a couple of rechecks... | 18:13 |
| fungi | clarkb: i've reset master to origin/master so the new commits are there | 18:13 |
| fungi | can i pass kick.sh a group name? | 18:14 |
| clarkb | fungi: agreed on git being up to date | 18:14 |
| clarkb | fungi: I think so? I've done it as foo1:foo2:foo3 before | 18:14 |
| corvus | clarkb: probably so; feel free to drop | 18:14 |
| *** florianf is now known as florianf|afk | 18:14 | |
| fungi | trying with `/opt/system-config/tools/kick.sh mirror` | 18:14 |
| clarkb | fungi: I expect that will also work | 18:14 |
| clarkb | if it does can you do etherpad-dev and etherpad after? | 18:14 |
| fungi | gladly! | 18:15 |
| dhellmann | AJaeger : other than needing that patch to the releases repo, the changes all look good | 18:15 |
| fungi | clarkb: corvus: yeah, looks like it's doing the hosts we wanted | 18:15 |
| openstackgerrit | Ryan Beisner proposed openstack-infra/project-config master: Retire the unmaintained PLUMgrid Charms projects https://review.openstack.org/615273 | 18:15 |
| AJaeger | dhellmann: thanks, will work on the releases changes now... | 18:16 |
| clarkb | fungi: I've not seen bhs1 update yet | 18:17 |
| clarkb | I guess with many more nodes to process it may take longer for any one of them to finish | 18:17 |
| fungi | yep | 18:18 |
| fungi | it hasn't gotten to the iptables tasks yet | 18:19 |
| mriedem | i assume etherpad being down is a known issue? | 18:21 |
| fungi | mriedem: yeah, we'll have it back up in just a sec | 18:21 |
| fungi | growing pains :/ | 18:21 |
| mriedem | cool | 18:21 |
| fungi | clarkb: i've started a kick.sh run for etherpad-dev:etherpad in a second screen window | 18:22 |
| fungi | so that we won't have to wait for the mirror servers to finish | 18:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 18:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 18:23 |
| clarkb | fungi: thanks | 18:23 |
| *** jamesmcarthur has quit IRC | 18:23 | |
| mordred | clarkb: just got back from grabbing a sandwich - looks like I missed the fun | 18:23 |
| mriedem | thar she blar | 18:24 |
| *** electrofelix has quit IRC | 18:24 | |
| clarkb | fungi: still no port 8080 on bhs1 mirror | 18:25 |
| fungi | it's still going | 18:26 |
| *** shrasool has quit IRC | 18:27 | |
| fungi | mordred: if you get a moment, might be a good idea to re-audit the yamlgroup inventory plugin change to make sure we didn't inadvertently add any other hosts to groups they weren't in previously | 18:27 |
| mordred | ++ | 18:27 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure adns1.opendev.org via ansible https://review.openstack.org/614648 | 18:27 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 18:27 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 18:27 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Reload nameservers when config changes https://review.openstack.org/615278 | 18:27 |
| *** pcaruana has joined #openstack-infra | 18:27 | |
| corvus | um. hrm. that sort of looks like a git-review rebase gone awry. | 18:27 |
| mordred | corvus: did you catch that I updated https://review.openstack.org/614648 with logan's review included? | 18:28 |
| corvus | mordred: yes, i've been building on top of it | 18:28 |
| mordred | kk. then I don't know what git-review did :) | 18:28 |
| corvus | i am just surprised to see the adns1 change updated. | 18:28 |
| clarkb | mordred: I think the next thing we need to look at is how disabled vs emergency vs emergency.yaml is working | 18:28 |
| clarkb | mordred: because well fungi reports it may not be working 100% as expected. | 18:28 |
| corvus | i believe its rebase was aborted and it left my tree in the rebased state rather than resetting to the previous state | 18:29 |
| clarkb | mordred: 17:33:44* fungi | [WARNING]: Unable to parse /etc/ansible/hosts/emergency.yaml as an inventory | 18:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 18:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 18:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Reload nameservers when config changes https://review.openstack.org/615278 | 18:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 18:30 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 18:30 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Reload nameservers when config changes https://review.openstack.org/615278 | 18:30 |
| corvus | ugh | 18:30 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure adns1.opendev.org server via ansible https://review.openstack.org/614648 | 18:31 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 18:31 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 18:31 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Reload nameservers when config changes https://review.openstack.org/615278 | 18:31 |
| corvus | okay, that should be back to the way it was, with the addition of a word in the commit message. we'll never get back the lost testing time though :( | 18:32 |
| *** trown|lunch is now known as trown | 18:33 | |
| corvus | git-review had its rebase aborted because the cwd was created in a patch involved in the rebase | 18:33 |
| clarkb | fungi: bhs1 looks correct now | 18:33 |
| corvus | that could probably be improved (if it hasn't already been in a later version than what i'm running) by cding out of the cwd when performing the rebase | 18:33 |
| *** e0ne has joined #openstack-infra | 18:34 | |
| corvus | oh, nope still wrong. one more revision. | 18:34 |
| fungi | corvus: i concur, a story about that would be awesome. also, git-review ought to refuse to push commits while a rebase is in progress (not finalized)? | 18:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Cleanup zuul-mergers in groups.yaml https://review.openstack.org/615279 | 18:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Cleanup zuul-executors in groups.yaml https://review.openstack.org/615280 | 18:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simpligy logstash nodes in groups.yaml https://review.openstack.org/615281 | 18:35 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify mirror node listings in groups.yaml https://review.openstack.org/615282 | 18:35 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 18:35 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 18:35 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Reload nameservers when config changes https://review.openstack.org/615278 | 18:35 |
| mordred | fungi: I actually frequently git review halfway up a stack | 18:35 |
| mordred | on purpose | 18:35 |
| mordred | but I'm weird | 18:35 |
| clarkb | I'm breaking up those cleanups by logical groups of services so that it is easier for us to monitor them and be happy with their running state as the changes go in | 18:36 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure adns1.opendev.org server via ansible https://review.openstack.org/614648 | 18:36 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 18:36 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Move start_services to all.yaml https://review.openstack.org/614959 | 18:36 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Reload nameservers when config changes https://review.openstack.org/615278 | 18:36 |
| *** ykarel_ is now known as ykarel | 18:37 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify nodepool groups.yaml membership https://review.openstack.org/615284 | 18:38 |
| fungi | clarkb: judging from file timestamps, i'm guessing the mirrors were rejecting traffic to those ports between ~17:00-18:30z | 18:38 |
| AJaeger | clarkb, fungi, mordred, once you're done with firefighting and have time for a review I would appreciate if you could look at my stack starting at https://review.openstack.org/615236 - it copies the content of the newish publish-to-pypi-python3 to publish-to-pypi - and then removes publish-to-pypi-python3. We really don't need both. | 18:38 |
| clarkb | fungi: ya it was 16:50 somethign when the rules.v4 file was written on bhs1 before | 18:38 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/yaml2ical master: add base class for recurrence https://review.openstack.org/615267 | 18:39 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/yaml2ical master: add day_specifier to recurrence https://review.openstack.org/615268 | 18:39 |
| fungi | clarkb: 16:58 is what i saw | 18:39 |
| fungi | according to my shell buffer | 18:39 |
| clarkb | fungi: same here | 18:40 |
| fungi | and you confirmed it corrected at 18:33 | 18:40 |
| mordred | clarkb: I love Simpligigation | 18:40 |
| fungi | okay, the mirror ansible run finally completed | 18:40 |
| fungi | just moments ago now | 18:40 |
| *** zul has joined #openstack-infra | 18:41 | |
| fungi | the etherpad kicks are also done | 18:41 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify zookeeper groups.yaml membership https://review.openstack.org/615288 | 18:41 |
| mordred | fungi: woot | 18:42 |
| clarkb | fungi: mordred that leaves us mostly with cleanup and simplification and the emergency file situation? | 18:42 |
| fungi | i thnik so | 18:42 |
| clarkb | fwiw I'm going to do a few more simplification changes then go find lunch | 18:42 |
| *** shrasool has joined #openstack-infra | 18:42 | |
| mordred | clarkb, fungi: what's the emergency file situtation? | 18:43 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify git backend server groups.yaml membership https://review.openstack.org/615290 | 18:43 |
| clarkb | mordred: 17:33:44* fungi | [WARNING]: Unable to parse /etc/ansible/hosts/emergency.yaml as an inventory | 18:43 |
| fungi | mordred: ansible is looking for the emergency disable file with a .yaml extension and not finding it | 18:43 |
| mordred | ahhh. yeah. kk. patch coming | 18:44 |
| fungi | it also seems to be finding (and using) the one without the .yaml suffix for the moment | 18:44 |
| fungi | so i don't think we ran against anything we had expressly disabled, at least | 18:44 |
| mordred | fungi: ok. I made an emergency.yaml that's got the content from the original emergency file | 18:46 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify elasticsearch cluster groups.yaml membership https://review.openstack.org/615293 | 18:46 |
| *** fried_rolls is now known as fried_rice | 18:46 | |
| *** jamesmcarthur has joined #openstack-infra | 18:47 | |
| fungi | yeah, i figured that was all that was required | 18:48 |
| mordred | fungi: I'm going to move the old file out of the way and run 'ansible disabled --list-hosts' just to make sure | 18:49 |
| mordred | fungi: looks like a sane list | 18:49 |
| * clarkb finds food | 18:50 | |
| clarkb | oh before I do that. fungi you think we are good to send a ntoice that jobs can be recheckedn ow? | 18:51 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove publish-to-pypi-python3 https://review.openstack.org/615241 | 18:51 |
| mordred | fungi: hah. all of the hosts in the emergency file are in the main disabled list | 18:51 |
| mordred | fungi: I'm going to remove them from emergency | 18:51 |
| clarkb | how about #status notice The firewall situation with ports 8080, 8081, and 8082 on mirror nodes has been resolved. You can recheck jobs that have failed to communicate to the mirrors on those ports now. | 18:51 |
| mordred | ++ | 18:52 |
| clarkb | mwhahaha: ^ fyi | 18:52 |
| mwhahaha | gracias | 18:52 |
| clarkb | #status notice The firewall situation with ports 8080, 8081, and 8082 on mirror nodes has been resolved. You can recheck jobs that have failed to communicate to the mirrors on those ports now. | 18:52 |
| openstackstatus | clarkb: sending notice | 18:52 |
| *** EvilienM is now known as EmilienM | 18:53 | |
| fungi | clarkb: yeah, we're all clear i think | 18:53 |
| fungi | sorry, just trying to figure out these new cloud accounts | 18:53 |
| fungi | i'm getting "The request you have made requires authentication." from openstackclient | 18:53 |
| corvus | fungi: for which cloud(s)? | 18:53 |
| -openstackstatus- NOTICE: The firewall situation with ports 8080, 8081, and 8082 on mirror nodes has been resolved. You can recheck jobs that have failed to communicate to the mirrors on those ports now. | 18:54 | |
| fungi | corvus: new credentials gary_perkins provided me earlier today | 18:54 |
| fungi | more arm resources i think | 18:54 |
| clarkb | ya this is the arm arm cloud aiui | 18:55 |
| fungi | "armci" | 18:55 |
| *** ykarel is now known as ykarel|away | 18:55 | |
| fungi | hosted in a packethost facility in japan | 18:55 |
| openstackstatus | clarkb: finished sending notice | 18:56 |
| fungi | ooh, progress. i was missing the project_domain_name and user_domain_name | 18:57 |
| mordred | those are important for keystone v3 | 18:58 |
| fungi | now i just need to work out the self-signed cert | 18:58 |
| fungi | mordred: well, to be more specific, i had them set to "default" but there's an actual non-default domain for them | 18:58 |
| mordred | you should be able to provide a ca bundle - I think we do that for one of the other clouds already | 18:58 |
| fungi | yup | 18:58 |
| mordred | fungi: ah - yes - it's even MORE important if there is an actual domain :) | 18:58 |
| fungi | i'm copying that now, i just need to grab the cert via openssl s_client | 18:59 |
| corvus | mordred: clarkb has a few more groups.yaml changes if you want to +3 those too | 18:59 |
| corvus | mordred: 615288 and children | 18:59 |
| *** ykarel|away has quit IRC | 19:01 | |
| fungi | mordred: oh! this might be a keystone catalog problem actually? | 19:02 |
| fungi | the cert is letsencrypt signed | 19:02 |
| fungi | getting this from osc: | 19:02 |
| fungi | SSL exception connecting to https://136.144.53.18:8774/v2.1/63ff25c687f3458cbf98a53936ca9bbf/servers/detail: HTTPSConnectionPool(host='136.144.53.18', port=8774): Max retries exceeded with url: /v2.1/63ff25c687f3458cbf98a53936ca9bbf/servers/detail (Caused by SSLError(CertificateError("hostname '136.144.53.18' doesn't match 'arm64ci.cloud'",),)) | 19:03 |
| mordred | fungi: yah. looks like that catalog is listing ips | 19:03 |
| mordred | fungi: is this in a clouds.yaml on bridge? or just locally? | 19:03 |
| fungi | mordred: in ~fungi/armci.yaml on bridge.o.o | 19:04 |
| *** bobh has quit IRC | 19:04 | |
| fungi | i was trying to get it working well enough with the temporary passwords so i could use the keystone api to set some new passwords before recording them in our usual places | 19:04 |
| mordred | fungi: import openstack ; c=openstack.connect(cloud='foo') ; c.pprint(c.service_catalog) | 19:04 |
| mordred | is a nice way to look at the catalog | 19:05 |
| *** e0ne has quit IRC | 19:05 | |
| mordred | fungi: and yes - they are returning ips not hostnames in their catalog | 19:06 |
| fungi | i suppose that's fairly trivially configurable? | 19:07 |
| fungi | it looks like it's the same ip address that the cn resolves to in dns | 19:07 |
| mordred | fungi: it's data entries in the keystone database | 19:07 |
| *** pcaruana has quit IRC | 19:08 | |
| mordred | so they either need to delete and re-add all of the services in via the api - or do an sql query to update things | 19:08 |
| *** jamesmcarthur has quit IRC | 19:08 | |
| fungi | got it | 19:08 |
| fungi | thanks! | 19:08 |
| mordred | sure nuff - for now, you can work around by adding "insecure: true" | 19:08 |
| mordred | which will completely ignore all of the security provided by ssl | 19:09 |
| clarkb | it will still ssl right? just verify nothing? | 19:10 |
| clarkb | fyi logstash.o.o is not going to currently manage indexing jobs beacuse the firewall is broken there. But one of my approved cleanups to the inventory groups.yaml file should fix that(just pointing it out if anyone notices) | 19:12 |
| mordred | clarkb: yes, that's right | 19:13 |
| corvus | clarkb: test failures are coming back on the groups cleanup patches | 19:14 |
| clarkb | corvus: bah | 19:14 |
| clarkb | let me look | 19:14 |
| fungi | thanks mordred, i'll pass the catalog info along to gary_perkins | 19:14 |
| clarkb | corvus: its a side effect of our fix for mirrors. I'll rebase the fix for that under my stack | 19:14 |
| *** betherly has joined #openstack-infra | 19:16 | |
| corvus | clarkb: it looks like the unit test for the groups has test fixtures which more or less assert the current state | 19:16 |
| *** gtmanfred has quit IRC | 19:17 | |
| clarkb | corvus: ya the issue is results.yaml in the test-fixtures dir wants mirror02.something in webservers group which we took the mirrors out of | 19:17 |
| clarkb | I'm rerunning tox.ini locally to make sure that is the only fix that is needed | 19:17 |
| corvus | clarkb: what's with the groups.yaml fixture? | 19:17 |
| clarkb | corvus: that is a symlink to our actual groups.yaml file | 19:18 |
| clarkb | corvus: I think its in the test-fixtures dir for ease of finding the path to it relative to the test script | 19:18 |
| corvus | oh... that's an interesting choice :) | 19:18 |
| corvus | ok then yeah, i agree this should be a straightforward fix | 19:18 |
| fungi | figures bypassing ci on changes would wedge our tests ;) | 19:19 |
| fungi | _every_time_ | 19:19 |
| corvus | fungi: it has *never* failed to do so :) | 19:19 |
| corvus | i'm going to get a sandwich | 19:19 |
| fungi | that sounds like a capital idea | 19:19 |
| *** gtmanfred has joined #openstack-infra | 19:20 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Cleanup zuul-mergers in groups.yaml https://review.openstack.org/615279 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Cleanup zuul-executors in groups.yaml https://review.openstack.org/615280 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simpligy logstash nodes in groups.yaml https://review.openstack.org/615281 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify mirror node listings in groups.yaml https://review.openstack.org/615282 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify nodepool groups.yaml membership https://review.openstack.org/615284 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify zookeeper groups.yaml membership https://review.openstack.org/615288 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify git backend server groups.yaml membership https://review.openstack.org/615290 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Simplify elasticsearch cluster groups.yaml membership https://review.openstack.org/615293 | 19:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Remove mirrors from webservers in groups test https://review.openstack.org/615300 | 19:20 |
| clarkb | I think ^ fixes it | 19:20 |
| fungi | mordred: thanks again, the insecure: true option does indeed seem to have worked around it for me in the meantime | 19:20 |
| *** betherly has quit IRC | 19:21 | |
| clarkb | fungi: isn't it great how universal that law of testing is? | 19:22 |
| *** apetrich has quit IRC | 19:24 | |
| *** eharney has quit IRC | 19:24 | |
| *** bobh has joined #openstack-infra | 19:25 | |
| *** shrasool has quit IRC | 19:27 | |
| *** jamesmcarthur has joined #openstack-infra | 19:28 | |
| fungi | yeah, i continue to be amazed by it. even something as simple as a one-line change | 19:28 |
| *** shrasool has joined #openstack-infra | 19:32 | |
| *** lpetrut has joined #openstack-infra | 19:34 | |
| *** finucannot is now known as stephenfin | 19:35 | |
| *** jistr has quit IRC | 19:35 | |
| *** jistr has joined #openstack-infra | 19:37 | |
| *** apetrich has joined #openstack-infra | 19:38 | |
| *** shrasool has quit IRC | 19:41 | |
| clarkb | its looking like that fix for the test will fix things | 19:42 |
| AJaeger | great! | 19:44 |
| *** betherly has joined #openstack-infra | 19:46 | |
| clarkb | AJaeger: question about the openstack python release thing. Are we using that anywhere for infra. I am not sure all of our things are python3 safe yet (gerritbot for example) | 19:50 |
| *** betherly has quit IRC | 19:50 | |
| clarkb | what do you know its got the python3 job on it now | 19:51 |
| clarkb | so maybe it does and if it doesn't I guess we fix that later | 19:51 |
| AJaeger | I can test on gerritbot... | 19:51 |
| clarkb | AJaeger: I +2'd since we've already transitioned the jobs to python3. If we really need to you can do the inverse and add a pytho2n specific template later | 19:52 |
| AJaeger | Yes, we use publish-to-pypi in many places - but the job is very simple, it's just "python3 setup.py sdist bdist_wheel " | 19:52 |
| clarkb | (I do think having defaults be python3 then explicitly opt into python2 instead of what we've doing is best) | 19:52 |
| AJaeger | clarkb: yeah, that is one option... | 19:52 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/gerritbot master: DNM: Testing publish-to-pypi-python3 https://review.openstack.org/615304 | 19:55 |
| AJaeger | clarkb: gerritbot test ^ | 19:55 |
| AJaeger | clarkb's stack passes - any infra-root to +2 the bottom of it? https://review.openstack.org/#/c/615300/ | 19:58 |
| *** kgiusti has left #openstack-infra | 19:59 | |
| AJaeger | clarkb: https://review.openstack.org/615304 is passing | 20:03 |
| clarkb | AJaeger: cool so probably a non issue then | 20:03 |
| corvus | mordred: 615288 and children can use a +3 | 20:06 |
| *** jamesmcarthur has quit IRC | 20:07 | |
| AJaeger | corvus, mordred first 615300 needs a +3... | 20:08 |
| *** eharney has joined #openstack-infra | 20:15 | |
| AJaeger | mordred: children are 615290 and 615293 - if you like to merge full stack... | 20:16 |
| mordred | AJaeger: done | 20:18 |
| AJaeger | thanks | 20:19 |
| AJaeger | mordred: what do you think of the stack starting at https://review.openstack.org/#/c/615236/ ? You looked earlier at it, now it should be ready... | 20:20 |
| AJaeger | thanks, mordred | 20:22 |
| mordred | \o/ | 20:22 |
| AJaeger | clarkb, mordred, want to review the rest of the stack as well, please? https://review.openstack.org/#/q/topic:publish-pypi+(status:open+OR+status:merged) - I'll recheck once bits are merged... | 20:22 |
| clarkb | AJaeger: https://review.openstack.org/#/c/615237/2 probably deserves email when it merges | 20:26 |
| AJaeger | clarkb: I'll send one now... | 20:26 |
| *** Swami has joined #openstack-infra | 20:27 | |
| *** jamesmcarthur has joined #openstack-infra | 20:28 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use python3 for release https://review.openstack.org/615236 | 20:33 |
| clarkb | ok where was I? project update slide brainstorming | 20:33 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove mirrors from webservers in groups test https://review.openstack.org/615300 | 20:33 |
| *** lpetrut has quit IRC | 20:35 | |
| *** betherly has joined #openstack-infra | 20:35 | |
| openstackgerrit | Merged openstack-infra/system-config master: Cleanup zuul-mergers in groups.yaml https://review.openstack.org/615279 | 20:36 |
| openstackgerrit | Merged openstack-infra/system-config master: Cleanup zuul-executors in groups.yaml https://review.openstack.org/615280 | 20:38 |
| openstackgerrit | Merged openstack-infra/system-config master: Simpligy logstash nodes in groups.yaml https://review.openstack.org/615281 | 20:38 |
| openstackgerrit | Merged openstack-infra/system-config master: Simplify mirror node listings in groups.yaml https://review.openstack.org/615282 | 20:40 |
| openstackgerrit | Merged openstack-infra/system-config master: Simplify nodepool groups.yaml membership https://review.openstack.org/615284 | 20:40 |
| *** betherly has quit IRC | 20:40 | |
| openstackgerrit | Merged openstack-infra/system-config master: Simplify zookeeper groups.yaml membership https://review.openstack.org/615288 | 20:43 |
| openstackgerrit | Merged openstack-infra/system-config master: Simplify git backend server groups.yaml membership https://review.openstack.org/615290 | 20:43 |
| openstackgerrit | Merged openstack-infra/system-config master: Simplify elasticsearch cluster groups.yaml membership https://review.openstack.org/615293 | 20:43 |
| AJaeger | \o/ | 20:44 |
| *** eharney has quit IRC | 20:46 | |
| AJaeger | mordred: could you review https://review.openstack.org/615237 and https://review.openstack.org/615239 as well, please? | 20:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure adns1.opendev.org server via ansible https://review.openstack.org/614648 | 20:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Configure opendev nameservers using ansible https://review.openstack.org/614870 | 20:50 |
| fungi | gonna go grab some dinner, back later | 20:55 |
| *** bobh has quit IRC | 20:59 | |
| *** abishop has quit IRC | 21:02 | |
| mordred | AJaeger: done | 21:03 |
| *** bobh has joined #openstack-infra | 21:03 | |
| AJaeger | thanks, mordred - can you review pbrx as well, please? https://review.openstack.org/615296 | 21:03 |
| *** jamesmcarthur has quit IRC | 21:04 | |
| mordred | all done | 21:04 |
| *** yamamoto has joined #openstack-infra | 21:05 | |
| *** jamesmcarthur has joined #openstack-infra | 21:05 | |
| AJaeger | thanks | 21:05 |
| *** jamesmcarthur has quit IRC | 21:08 | |
| *** bobh has quit IRC | 21:09 | |
| *** yamamoto has quit IRC | 21:09 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Sync publish-to-pypi templates https://review.openstack.org/615237 | 21:09 |
| *** jamesmcarthur has joined #openstack-infra | 21:13 | |
| corvus | mordred, clarkb, fungi: the opendev nameserver changes https://review.openstack.org/614648 and https://review.openstack.org/614870 pass tests now; i'd appreciate a +3 and we should be able to get those in production next week | 21:15 |
| clarkb | corvus: great I'll take a look shortly | 21:18 |
| *** shrasool has joined #openstack-infra | 21:23 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use publish-to-pypi everywhere https://review.openstack.org/615239 | 21:25 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual master: Fix link to publish-to-pypi https://review.openstack.org/615329 | 21:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: DNM: extra debugging https://review.openstack.org/615330 | 21:30 |
| AJaeger | clarkb, mordred, fungi, small update for the publish-to-pypi change for infra-manual in https://review.openstack.org/615329 | 21:31 |
| clarkb | looks like logstash.o.o is working again | 21:31 |
| *** dave-mccowan has quit IRC | 21:35 | |
| clarkb | corvus: https://review.openstack.org/#/c/614648/11..16/playbooks/roles/master-nameserver/tasks/main.yaml you stopped checking if the service should be started or not, but the test remains in place to check if the service is started. Does bind9 start automatically when the package is installed? | 21:35 |
| *** betherly has joined #openstack-infra | 21:37 | |
| *** jamesmcarthur has quit IRC | 21:38 | |
| *** rlandy has quit IRC | 21:41 | |
| *** betherly has quit IRC | 21:41 | |
| *** jamesmcarthur has joined #openstack-infra | 21:42 | |
| corvus | clarkb: yes, they both do, so i simplified | 21:42 |
| corvus | clarkb: with nsd, we needed to write the config before installing the package anyway (because it starts on install), so i figured we could just rely on that for now. | 21:43 |
| corvus | (the default config for nsd breaks for us because it listens on 0.0.0.0) | 21:43 |
| clarkb | ya I got to nsd and realized that must be what is happening | 21:43 |
| clarkb | fungi: re etherpad, maybe we want to redeploy it without the hwe kernel after the summit? I worry about changing much on it now that we've appeared to get it stable and happy | 21:44 |
| *** jamesmcarthur has quit IRC | 21:47 | |
| *** jamesmcarthur has joined #openstack-infra | 21:48 | |
| *** boden has quit IRC | 21:51 | |
| *** jamesmcarthur has quit IRC | 21:52 | |
| *** jamesmcarthur has joined #openstack-infra | 22:06 | |
| *** munimeha1 has quit IRC | 22:06 | |
| *** jamesmcarthur has quit IRC | 22:10 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: DNM: extra debugging https://review.openstack.org/615330 | 22:19 |
| *** rossella_s has quit IRC | 22:21 | |
| fungi | clarkb: yeah, i haven't seen evidence that the hwe kernel made any observable difference in performance there | 22:22 |
| *** shrasool has quit IRC | 22:24 | |
| fungi | clarkb: we *can* also just swap back to the non-hwe kernel and reboot. no need to redeploy | 22:25 |
| fungi | both are installed and available | 22:25 |
| clarkb | ya the kernel tends to be pretty good about forward and backward compatibility | 22:26 |
| fungi | they're parallel packages, it's not as if there's any overlap | 22:26 |
| *** shrasool has joined #openstack-infra | 22:27 | |
| clarkb | mostly I think its happy now so don't want to keep changing it this close to summit :) | 22:28 |
| *** shrasool has quit IRC | 22:28 | |
| fungi | k | 22:28 |
| corvus | pfft. summit isn't until november. | 22:30 |
| clarkb | corvus: don't look at a calendar | 22:30 |
| clarkb | I looked this morning and got scared. I need to put head down and get slides/talk/forum stuff done next week | 22:30 |
| fungi | time is an illusion | 22:32 |
| corvus | lunchtime doubly so | 22:32 |
| fungi | i continue to use "missing, presumed fed" as my lunchtime /away message | 22:33 |
| clarkb | also for people not in Oregon Tuesday probably involves physically going to a polling station. I get to drop my ballot off tonight | 22:33 |
| fungi | clarkb: north carolina hasn't officially announced that they burn early voting ballots in a big pile yet, so i cast mine a couple weeks ago | 22:34 |
| corvus | mine weighs like 10 pounds | 22:34 |
| fungi | was encouraged to discover that sometime in the past few months nc also decided to completely get rid of electronic voting machines in favor of good old-fashioned paper ballots and a pen | 22:35 |
| clarkb | fungi: oh nice. In oregon we get a paper ballot with scantron bubbles you fill in then send back in the mail (or if you are like me wait too long for USPS to deliver it on time so have to drop it off in a collection box) | 22:35 |
| corvus | fungi: that is good and surprising news | 22:35 |
| logan- | fungi: in texas we use electronic voting machines that autocorrect your ballot for you | 22:35 |
| clarkb | logan-: the computers know best | 22:36 |
| fungi | i suppose the scantrons could still get hacked, but that seems like it would take a much more concerted effort | 22:36 |
| corvus | fungi: and you can always recount scantrons manually | 22:36 |
| clarkb | washington has the best voting process I've run into yet. Its like Oregon's in most ways except that your ballot counts if post marked on election day | 22:36 |
| clarkb | this does mean it usually takes washington a few extra days to certify results though | 22:37 |
| corvus | (or run them through different scantrons) | 22:37 |
| fungi | corvus: well, someone can anyway. those people can also be hacked of course | 22:37 |
| corvus | fungi: i think you just came up with an election-themed slasher movie | 22:37 |
| fungi | or a new season of ghost in the shell | 22:37 |
| clarkb | the other thing Oregon did recentlyish was automagic voter registration through the dmv | 22:38 |
| clarkb | I wonder what that will do for turnout | 22:39 |
| fungi | our dmv gives you the option to register to vote when you get/renew your license (and update your registration if you file an address change for your driver's license). is oregon going further? opt-out? | 22:40 |
| clarkb | fungi: ya opt out | 22:40 |
| clarkb | brainstorming of berlin infra onboarding https://etherpad.openstack.org/p/openstack-infra-berlin-onboarding | 22:41 |
| clarkb | dtroyer: ^ happy for feedback on what you think would be valuable for starlingx | 22:41 |
| *** erlon has quit IRC | 22:43 | |
| *** bobh has joined #openstack-infra | 22:44 | |
| *** jamesmcarthur has joined #openstack-infra | 22:47 | |
| *** bobh has quit IRC | 22:50 | |
| *** jamesmcarthur has quit IRC | 22:51 | |
| *** jcoufal has quit IRC | 22:54 | |
| *** ianychoi has quit IRC | 23:02 | |
| *** owalsh has quit IRC | 23:16 | |
| *** owalsh has joined #openstack-infra | 23:17 | |
| *** shardy_ has quit IRC | 23:18 | |
| *** shardy has quit IRC | 23:18 | |
| *** mriedem has quit IRC | 23:18 | |
| *** shrasool has joined #openstack-infra | 23:19 | |
| *** roman_g has quit IRC | 23:23 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP: Set relative priority of node requests https://review.openstack.org/615356 | 23:24 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Merger: automatically add new hosts to the known_hosts file https://review.openstack.org/608453 | 23:30 |
| *** jtomasek has quit IRC | 23:31 | |
| *** betherly has joined #openstack-infra | 23:39 | |
| *** markvoelker has joined #openstack-infra | 23:41 | |
| openstackgerrit | Merged openstack-infra/system-config master: Configure adns1.opendev.org server via ansible https://review.openstack.org/614648 | 23:41 |
| *** dhill_ has quit IRC | 23:43 | |
| *** betherly has quit IRC | 23:43 | |
| *** Swami has quit IRC | 23:44 | |
| *** markvoelker has quit IRC | 23:46 | |
| *** gyee has quit IRC | 23:48 | |
| *** ianychoi has joined #openstack-infra | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!