| *** Keitaro has quit IRC | 00:18 | |
| *** dingyichen has joined #openstack-infra | 00:18 | |
| *** jesusaur has joined #openstack-infra | 00:21 | |
| *** Keitaro has joined #openstack-infra | 00:27 | |
| *** longkb has joined #openstack-infra | 00:44 | |
| *** hongbin has joined #openstack-infra | 01:03 | |
| *** Qiming has joined #openstack-infra | 01:18 | |
| *** jamesmcarthur has joined #openstack-infra | 01:20 | |
| openstackgerrit | Merged openstack-infra/system-config master: Cleanup docs formatting https://review.openstack.org/596457 | 01:22 |
|---|---|---|
| *** jamesmcarthur has quit IRC | 01:24 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add kerberos-client role https://review.openstack.org/593486 | 01:39 |
| openstackgerrit | Merged openstack-infra/system-config master: Add openafs-client role https://review.openstack.org/593487 | 01:39 |
| *** jamesmcarthur has joined #openstack-infra | 01:40 | |
| *** jamesmcarthur has quit IRC | 01:41 | |
| *** ramishra has joined #openstack-infra | 02:09 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] create an ansible group for bridge.o.o https://review.openstack.org/596630 | 02:10 |
| openstackgerrit | Nguyen Hai proposed openstack-infra/project-config master: remove job settings for solum repositories https://review.openstack.org/592923 | 02:20 |
| *** adrianreza has joined #openstack-infra | 02:30 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] create an ansible group for bridge.o.o https://review.openstack.org/596630 | 02:37 |
| ianw | tonyb: ^^^ should we try switching the wheel build jobs? | 02:38 |
| *** psachin has joined #openstack-infra | 02:46 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] create an ansible group for bridge.o.o https://review.openstack.org/596630 | 02:51 |
| *** egonzalez has quit IRC | 02:52 | |
| *** egonzalez has joined #openstack-infra | 02:53 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:55 | |
| tonyb | ianw: what does 'switching' look like? | 02:55 |
| tonyb | ianw: isn't 590636 going to do the switch? | 02:58 |
| *** ramishra has quit IRC | 03:00 | |
| ianw | tonyb: well, yeah, that and a bunch of baby-sitting i'd imagine :) | 03:02 |
| tonyb | :) | 03:02 |
| ianw | actually just realised the o-z-j change won't go in, so the deps are broken there | 03:02 |
| tonyb | ianw: :( | 03:02 |
| ianw | ahh, i guess i'll fix that, and we can wait for corvus to look at the integration jobs in system-config and see if we agree on that path | 03:02 |
| tonyb | ianw: that saves me asking the next question ;p | 03:03 |
| *** ramishra has joined #openstack-infra | 03:08 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] create an ansible group for bridge.o.o https://review.openstack.org/596630 | 03:14 |
| *** jamesmcarthur has joined #openstack-infra | 03:22 | |
| *** jamesmcarthur has quit IRC | 03:23 | |
| *** udesale has joined #openstack-infra | 03:27 | |
| *** jamesmcarthur has joined #openstack-infra | 03:39 | |
| *** jamesmcarthur has quit IRC | 03:44 | |
| *** viks__ has joined #openstack-infra | 03:52 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Create an ansible group for bridge.o.o https://review.openstack.org/596630 | 04:02 |
| ianw | fungi / pabelanger: ^ response to your suggestions over the "admin" group. I think it works out better, and we have a targeted group only on bridge.o.o | 04:05 |
| ianw | and it's tested. i'm struggling to think of any other project I could not only have insights on this sort of thing as a regular user, but contribute changes and have them actually tested and deployed. it's pretty cool | 04:06 |
| AJaeger | ianw, frickler, please review https://review.openstack.org/596523 and https://review.openstack.org/596524 | 04:27 |
| ianw | what is pypi-mirror? | 04:32 |
| AJaeger | "PyPI mirror builder" - might be dead? | 04:39 |
| AJaeger | mmh, http://git.openstack.org/cgit/openstack-infra/system-config/tree/doc/source/jeepyb.rst#n25 | 04:40 |
| AJaeger | ianw: this goes back to http://lists.openstack.org/pipermail/openstack-infra/2018-August/006058.html | 04:41 |
| AJaeger | ianw: I still suggest to merge as is - it keeps the status quo. | 04:41 |
| *** lathiat has quit IRC | 04:49 | |
| *** ykarel has joined #openstack-infra | 04:54 | |
| *** lathiat has joined #openstack-infra | 04:58 | |
| *** lathiat has quit IRC | 04:58 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add role integration jobs https://review.openstack.org/596018 | 05:00 |
| *** shabeeb has joined #openstack-infra | 05:07 | |
| *** lathiat has joined #openstack-infra | 05:24 | |
| *** hongbin has quit IRC | 05:27 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Remove legacy-install-afs-with-puppet role https://review.openstack.org/590636 | 05:32 |
| *** e0ne has joined #openstack-infra | 05:34 | |
| *** shabeeb has quit IRC | 05:38 | |
| *** shabeeb has joined #openstack-infra | 05:39 | |
| *** srp has joined #openstack-infra | 05:41 | |
| *** srp has quit IRC | 05:43 | |
| *** gagehugo has joined #openstack-infra | 05:50 | |
| *** tinwood has quit IRC | 06:05 | |
| *** janki has joined #openstack-infra | 06:06 | |
| *** tinwood has joined #openstack-infra | 06:07 | |
| *** dpawlik has joined #openstack-infra | 06:10 | |
| *** dpawlik has quit IRC | 06:11 | |
| *** dpawlik_ has joined #openstack-infra | 06:11 | |
| *** bhavikdbavishi has quit IRC | 06:16 | |
| *** jtomasek has joined #openstack-infra | 06:16 | |
| *** e0ne has quit IRC | 06:16 | |
| *** aojea has joined #openstack-infra | 06:22 | |
| *** Dobroslaw has joined #openstack-infra | 06:24 | |
| *** Bhujay has joined #openstack-infra | 06:34 | |
| *** chandankumar is now known as chkumar|pto | 06:35 | |
| *** apetrich has joined #openstack-infra | 06:38 | |
| *** Bhujay has quit IRC | 06:40 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Cancel Tap as a service meeting https://review.openstack.org/578328 | 06:43 |
| *** annp has joined #openstack-infra | 06:46 | |
| *** pcaruana has joined #openstack-infra | 06:49 | |
| *** Bhujay has joined #openstack-infra | 06:49 | |
| *** ccamacho has joined #openstack-infra | 06:50 | |
| openstackgerrit | Nguyen Hai proposed openstack-infra/project-config master: remove job settings for zun repositories https://review.openstack.org/595221 | 06:50 |
| AJaeger | config-core, https://review.openstack.org/#/c/592923/ and https://review.openstack.org/595221 are ready to merge - python3-first changes... | 06:55 |
| *** dmellado has quit IRC | 07:00 | |
| *** dmellado has joined #openstack-infra | 07:02 | |
| *** ginopc has joined #openstack-infra | 07:04 | |
| *** Bhujay has quit IRC | 07:10 | |
| *** eumel8 has joined #openstack-infra | 07:16 | |
| *** Bhujay has joined #openstack-infra | 07:18 | |
| *** jpich has joined #openstack-infra | 07:19 | |
| *** hashar has joined #openstack-infra | 07:20 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Add zuul roles in system-config to linters https://review.openstack.org/596645 | 07:24 |
| *** alexchadin has joined #openstack-infra | 07:24 | |
| *** dtantsur|afk is now known as dtantsur | 07:26 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Remove legacy-install-afs-with-puppet role https://review.openstack.org/590636 | 07:26 |
| *** slaweq has quit IRC | 07:30 | |
| *** slaweq has joined #openstack-infra | 07:31 | |
| *** rcernin has quit IRC | 07:31 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add role integration jobs https://review.openstack.org/596018 | 07:36 |
| *** tosky has joined #openstack-infra | 07:36 | |
| *** jamesmcarthur has joined #openstack-infra | 07:40 | |
| *** jamesmcarthur has quit IRC | 07:44 | |
| *** jpena|off is now known as jpena | 07:45 | |
| *** graphene has joined #openstack-infra | 07:47 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/infra-specs master: Add anomaly detection in CI/CD jobs specification draft https://review.openstack.org/581214 | 07:51 |
| *** d0ugal has joined #openstack-infra | 07:54 | |
| *** florianf has joined #openstack-infra | 07:56 | |
| *** ykarel is now known as ykarel|lunch | 07:59 | |
| *** dbecker has joined #openstack-infra | 08:10 | |
| *** jtomasek has quit IRC | 08:15 | |
| *** jtomasek has joined #openstack-infra | 08:16 | |
| *** ginopc has quit IRC | 08:20 | |
| *** ginopc has joined #openstack-infra | 08:21 | |
| *** electrofelix has joined #openstack-infra | 08:22 | |
| openstackgerrit | Merged openstack-infra/project-config master: remove job settings for solum repositories https://review.openstack.org/592923 | 08:33 |
| openstackgerrit | Merged openstack-infra/project-config master: remove job settings for zun repositories https://review.openstack.org/595221 | 08:37 |
| AJaeger | frickler, could you please review https://review.openstack.org/596523 and https://review.openstack.org/596524 as well? | 08:37 |
| *** flaper87 has joined #openstack-infra | 08:38 | |
| AJaeger | frickler: and https://review.openstack.org/596542 would be nice as well. Thanks! | 08:38 |
| *** ykarel|lunch is now known as ykarel | 08:41 | |
| *** ginopc has quit IRC | 08:45 | |
| *** Diabelko has joined #openstack-infra | 08:45 | |
| *** graphene has quit IRC | 08:46 | |
| *** graphene has joined #openstack-infra | 08:47 | |
| *** e0ne has joined #openstack-infra | 08:48 | |
| *** Bhujay has quit IRC | 08:48 | |
| *** ginopc has joined #openstack-infra | 08:49 | |
| *** Emine has joined #openstack-infra | 08:53 | |
| *** graphene has quit IRC | 08:55 | |
| *** ginopc has quit IRC | 08:55 | |
| *** ginopc has joined #openstack-infra | 08:58 | |
| *** graphene has joined #openstack-infra | 08:58 | |
| *** graphene has joined #openstack-infra | 09:00 | |
| *** dingyichen has quit IRC | 09:01 | |
| *** r-mibu has joined #openstack-infra | 09:01 | |
| *** ginopc has quit IRC | 09:02 | |
| *** owalsh_ is now known as owalsh | 09:04 | |
| *** ginopc has joined #openstack-infra | 09:04 | |
| *** holser_ has joined #openstack-infra | 09:07 | |
| *** ginopc has quit IRC | 09:08 | |
| *** ginopc has joined #openstack-infra | 09:09 | |
| *** olivierb has joined #openstack-infra | 09:10 | |
| *** ginopc has quit IRC | 09:13 | |
| *** vivsoni_ has quit IRC | 09:19 | |
| *** Bhujay has joined #openstack-infra | 09:21 | |
| openstackgerrit | Merged openstack-infra/project-config master: Fix some doc publishing jobs https://review.openstack.org/596524 | 09:21 |
| *** alexchadin has quit IRC | 09:43 | |
| *** dtantsur is now known as dtantsur|brb | 09:44 | |
| *** lennyb_ has joined #openstack-infra | 09:45 | |
| *** lennyb_ is now known as lennyb | 09:45 | |
| *** vivsoni has joined #openstack-infra | 09:52 | |
| *** hashar has quit IRC | 10:15 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy-requirements-integration-dsvm* jobs https://review.openstack.org/596523 | 10:16 |
| *** hashar has joined #openstack-infra | 10:22 | |
| *** olivierb_ has joined #openstack-infra | 10:24 | |
| *** jaosorior has joined #openstack-infra | 10:25 | |
| *** olivierb has quit IRC | 10:25 | |
| *** pcaruana has quit IRC | 10:32 | |
| *** pcaruana has joined #openstack-infra | 10:32 | |
| AJaeger | config-core, could you review https://review.openstack.org/592474 and https://review.openstack.org/587768 and https://review.openstack.org/#/c/595763/ https://review.openstack.org/596449, please? | 10:34 |
| *** shabeeb has quit IRC | 10:39 | |
| *** srp has joined #openstack-infra | 10:39 | |
| *** dtantsur|brb is now known as dtantsur | 10:41 | |
| *** slagle has quit IRC | 10:42 | |
| *** olivierb has joined #openstack-infra | 10:43 | |
| *** slagle has joined #openstack-infra | 10:44 | |
| *** olivierb_ has quit IRC | 10:45 | |
| *** graphene has quit IRC | 10:49 | |
| *** graphene has joined #openstack-infra | 10:51 | |
| *** srp has quit IRC | 10:53 | |
| *** shabeeb has joined #openstack-infra | 10:54 | |
| *** slagle has quit IRC | 10:55 | |
| *** ginopc has joined #openstack-infra | 11:03 | |
| *** apetrich has quit IRC | 11:04 | |
| *** hashar has quit IRC | 11:11 | |
| *** apetrich has joined #openstack-infra | 11:13 | |
| *** nicolasbock has joined #openstack-infra | 11:13 | |
| dtantsur | hi folks! devstack (at least locally) seems broken with AssertionError: pysaml2 .dist-info directory not found | 11:21 |
| *** jpena is now known as jpena|lunch | 11:22 | |
| *** hashar has joined #openstack-infra | 11:24 | |
| dtantsur | and it seems devstack-specific, just installing it in a venv works.. | 11:24 |
| *** rh-jelabarre has joined #openstack-infra | 11:25 | |
| *** rosmaita has joined #openstack-infra | 11:30 | |
| *** udesale has quit IRC | 11:38 | |
| *** snapiri has joined #openstack-infra | 11:39 | |
| *** jamesmcarthur has joined #openstack-infra | 11:40 | |
| *** ykarel is now known as ykarel|away | 11:42 | |
| *** jroll has quit IRC | 11:43 | |
| *** rtjure has quit IRC | 11:44 | |
| *** jroll has joined #openstack-infra | 11:44 | |
| *** jamesmcarthur has quit IRC | 11:45 | |
| *** ykarel|away has quit IRC | 11:46 | |
| *** longkb has quit IRC | 11:52 | |
| openstackgerrit | Merged openstack-dev/pbr master: add lib-forward-testing-python3 test job https://review.openstack.org/591177 | 11:53 |
| AJaeger | stephenfin: please review https://review.openstack.org/#/c/571571/ as well | 11:54 |
| *** rtjure has joined #openstack-infra | 11:58 | |
| *** vivsoni has quit IRC | 12:01 | |
| *** jamesmcarthur has joined #openstack-infra | 12:03 | |
| stephenfin | AJaeger: Yup, done | 12:06 |
| *** vivsoni has joined #openstack-infra | 12:06 | |
| AJaeger | thanks, stephenfin | 12:07 |
| *** boden has joined #openstack-infra | 12:09 | |
| *** jamesmcarthur has quit IRC | 12:10 | |
| *** tpsilva has joined #openstack-infra | 12:13 | |
| *** alexchadin has joined #openstack-infra | 12:18 | |
| *** jpena|lunch is now known as jpena | 12:24 | |
| *** shabeeb has quit IRC | 12:25 | |
| *** jamesmcarthur has joined #openstack-infra | 12:26 | |
| *** jamesmcarthur has quit IRC | 12:30 | |
| *** rlandy has joined #openstack-infra | 12:30 | |
| *** ykarel|away has joined #openstack-infra | 12:31 | |
| *** ykarel|away is now known as ykarel | 12:32 | |
| *** ykarel_ has joined #openstack-infra | 12:36 | |
| *** jcoufal has joined #openstack-infra | 12:37 | |
| *** ykarel has quit IRC | 12:39 | |
| *** rtjure has quit IRC | 12:43 | |
| *** trown|outtypewww is now known as trown | 12:44 | |
| *** ramishra has quit IRC | 12:46 | |
| *** kgiusti has joined #openstack-infra | 12:47 | |
| *** _ari_|pto has quit IRC | 12:48 | |
| *** rtjure has joined #openstack-infra | 12:48 | |
| *** jamesmcarthur has joined #openstack-infra | 12:48 | |
| *** alexchadin has quit IRC | 12:51 | |
| *** ykarel_ is now known as ykarel | 12:53 | |
| *** janki has quit IRC | 12:55 | |
| *** mriedem has joined #openstack-infra | 12:55 | |
| *** ykarel has quit IRC | 13:03 | |
| *** ykarel has joined #openstack-infra | 13:03 | |
| *** alexchadin has joined #openstack-infra | 13:18 | |
| *** ansmith has joined #openstack-infra | 13:18 | |
| *** jamesmcarthur has quit IRC | 13:21 | |
| *** ginopc has quit IRC | 13:21 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: rewrite interface in react https://review.openstack.org/591604 | 13:22 |
| openstackgerrit | Nguyen Hai proposed openstack-infra/project-config master: remove job settings for horizon repositories https://review.openstack.org/594069 | 13:29 |
| *** _ari_ has joined #openstack-infra | 13:34 | |
| *** eharney has joined #openstack-infra | 13:43 | |
| *** alexchadin has quit IRC | 13:43 | |
| *** alexchadin has joined #openstack-infra | 13:44 | |
| *** jamesmcarthur has joined #openstack-infra | 13:44 | |
| *** bdodd has joined #openstack-infra | 13:49 | |
| *** psachin has quit IRC | 13:50 | |
| AJaeger | smcginnis: want to +2A https://review.openstack.org/#/c/593895 to get the operations-guide stack in? | 13:55 |
| *** Emine has quit IRC | 13:55 | |
| smcginnis | AJaeger: Done - thanks! | 13:55 |
| AJaeger | thanks, smcginnis | 13:56 |
| *** sthussey has joined #openstack-infra | 13:56 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: rewrite interface in react https://review.openstack.org/591604 | 13:59 |
| *** r-daneel has joined #openstack-infra | 14:05 | |
| openstackgerrit | Samuel Cassiba proposed openstack-infra/project-config master: Unretire the openstack/cookbook-openstack-bare-metal repo https://review.openstack.org/596743 | 14:05 |
| openstackgerrit | Samuel Cassiba proposed openstack-infra/project-config master: Unretire the openstack/cookbook-openstack-bare-metal repo https://review.openstack.org/596743 | 14:07 |
| pabelanger | ianw: left comment | 14:09 |
| AJaeger | config-core, dependencies are in - we can merge https://review.openstack.org/#/c/593899/ . please review | 14:10 |
| *** alexchadin has quit IRC | 14:11 | |
| openstackgerrit | Nguyen Hai proposed openstack-infra/project-config master: remove job settings for horizon repositories https://review.openstack.org/594069 | 14:11 |
| *** dabukalam has quit IRC | 14:13 | |
| openstackgerrit | Nguyen Hai proposed openstack-infra/project-config master: remove job settings for horizon repositories https://review.openstack.org/594069 | 14:13 |
| *** Emine has joined #openstack-infra | 14:13 | |
| openstackgerrit | Samuel Cassiba proposed openstack-infra/project-config master: Unretire the openstack/cookbook-openstack-bare-metal repo https://review.openstack.org/596743 | 14:15 |
| openstackgerrit | Nguyen Hai proposed openstack-infra/project-config master: remove job settings for horizon repositories https://review.openstack.org/594069 | 14:16 |
| frickler | infra-root: devstack master seems currently broken for local deployments, but not for gating. see my ramblings in #-qa and take action if necessary, I'll be afk now for a bit | 14:16 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: rewrite interface in react https://review.openstack.org/591604 | 14:16 |
| openstackgerrit | Samuel Cassiba proposed openstack-infra/project-config master: Unretire the openstack/cookbook-openstack-bare-metal repo https://review.openstack.org/596743 | 14:17 |
| AJaeger | frickler: not all airship repos have in-repo jobs yet... | 14:19 |
| openstackgerrit | Samuel Cassiba proposed openstack-infra/project-config master: Unretire the openstack/cookbook-openstack-bare-metal repo https://review.openstack.org/596743 | 14:21 |
| Shrews | corvus: ohai! http://logs.openstack.org/73/593973/16/check/puppet-beaker-rspec-infra-centos-7-system-config/05b2bf2/job-output.txt.gz#_2018-08-24_23_06_05_298352 | 14:21 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Add release note for fix to bug 1786306 https://review.openstack.org/596755 | 14:22 |
| openstack | bug 1786306 in PBR "pbr doesn't work when there's subdirectory in url " [Undecided,Fix released] https://launchpad.net/bugs/1786306 - Assigned to Wei T (nuaafe) | 14:22 |
| corvus | Shrews: i had been reading that as an expected failure (since the user doesn't exist) | 14:22 |
| scas | smcginnis: AJaeger: appreciate the eyes. i haven't had coffee or tea since thursday | 14:22 |
| corvus | Shrews: i think http://logs.openstack.org/73/593973/16/check/puppet-beaker-rspec-infra-centos-7-system-config/05b2bf2/job-output.txt.gz#_2018-08-24_23_06_05_449974 is the real failure | 14:23 |
| fungi | though it's interesting that it thinks slukjanov should exist | 14:23 |
| Shrews | corvus: ah, i guess that makes sense (especially since it says 23 examples, 1 failure | 14:24 |
| Shrews | ) | 14:24 |
| corvus | Shrews: but i believe that as soon as we merge https://review.openstack.org/596503 or https://review.openstack.org/596504 things will be fixed | 14:24 |
| *** r-daneel_ has joined #openstack-infra | 14:24 | |
| *** r-daneel has quit IRC | 14:24 | |
| *** r-daneel_ is now known as r-daneel | 14:24 | |
| fungi | i can get behind 596503 if people think that's safer. 596504 seemed nice from the perspective of being able to use consustent local variables in a service handler | 14:25 |
| corvus | Shrews: you want to have a look at those 2 changes and let me know if you have a preference? | 14:25 |
| fungi | er, consistent too | 14:25 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove operations-guide jobs https://review.openstack.org/593899 | 14:26 |
| corvus | fungi: yeah; i was kind of digging it because it felt really self-contained, until pabelanger mentioned the fact that we'd be flushing all handlers, which suddently reversed that feeling for me, as it meant that we might suddenly be interfering with other roles after all. | 14:26 |
| Shrews | corvus: looking | 14:26 |
| AJaeger | infra-root, could you review this devstack-gate job to import jobs, please? https://review.openstack.org/596542 | 14:27 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/puppet-ptgbot master: Push configuration for Denver PTG4 https://review.openstack.org/596765 | 14:29 |
| *** ramishra has joined #openstack-infra | 14:30 | |
| *** rosmaita has quit IRC | 14:31 | |
| AJaeger | config-core, next python3-first repo ready: https://review.openstack.org/#/c/594069/ please review! | 14:31 |
| Shrews | corvus: i prefer the scoped var name | 14:32 |
| Shrews | and i'm ready to approve that, unless there are objections | 14:32 |
| corvus | Shrews, fungi: then i think we have a winner | 14:32 |
| corvus | Shrews: go for it | 14:32 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Add release note for fix to bug 1786306 https://review.openstack.org/596755 | 14:33 |
| openstack | bug 1786306 in PBR "pbr doesn't work when there's subdirectory in url " [Undecided,Fix released] https://launchpad.net/bugs/1786306 - Assigned to Wei T (nuaafe) | 14:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add iptables role https://review.openstack.org/593973 | 14:33 |
| *** shabeeb has joined #openstack-infra | 14:34 | |
| corvus | Shrews: ^ i added a depends-on to that; it should pass tests and be ready to merge now | 14:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add NTP service to base-server https://review.openstack.org/595312 | 14:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add timezone role https://review.openstack.org/594542 | 14:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add testinfra test for ntp https://review.openstack.org/596382 | 14:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add snmpd role and add it to base https://review.openstack.org/593969 | 14:34 |
| corvus | there are its friends, rebased | 14:34 |
| *** jamesmcarthur has quit IRC | 14:35 | |
| Shrews | corvus: should i add tests for ntp and tz in those reviews ^^ ? | 14:35 |
| corvus | Shrews: 382 adds an ntp test... i wasn't sure how to test tz; if you find a way i think that would be great :) | 14:36 |
| *** ianychoi has quit IRC | 14:36 | |
| Shrews | ah i didn't notice that one | 14:37 |
| Shrews | i think i can come up with something for tz | 14:38 |
| corvus | cool | 14:38 |
| openstackgerrit | Merged openstack-dev/pbr master: Move pbr-installation jobs in-tree https://review.openstack.org/571571 | 14:39 |
| openstackgerrit | Chuck Short proposed openstack/diskimage-builder master: Replace assertRaisesRegexp with assertRaisesRegex https://review.openstack.org/596774 | 14:40 |
| corvus | Shrews: i'd like to try to get all the base playbook stuff wrapped up this week, as well as the pbrx image publishing jobs. do you have changes for that i should start reviewing? | 14:41 |
| *** ianychoi has joined #openstack-infra | 14:42 | |
| Shrews | corvus: https://review.openstack.org/593125 | 14:43 |
| Shrews | and this tz test i am working up | 14:43 |
| Shrews | that is all | 14:43 |
| *** jcoufal has quit IRC | 14:44 | |
| *** jcoufal has joined #openstack-infra | 14:45 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Add testinfra test for timezone https://review.openstack.org/596778 | 14:45 |
| *** dpawlik_ has quit IRC | 14:46 | |
| corvus | Shrews: i think i have a way we can move that out of project-config and into zuul... should i suggest that now or should we proceed on the existing path for now? | 14:47 |
| Shrews | corvus: i originally had it in zuul but mordred suggested placing it in project-config so others can use the job | 14:48 |
| corvus | Shrews: hrm, but "others" shouldn't use the openstackzuul credentials | 14:48 |
| corvus | oh wait | 14:48 |
| Shrews | nodepool would | 14:48 |
| *** hashar is now known as hasharAway | 14:49 | |
| *** alexchadin has joined #openstack-infra | 14:49 | |
| *** jcoufal_ has joined #openstack-infra | 14:49 | |
| corvus | Shrews: yes, i don't consider "nodepool" others -- but what do the credentials mean? do they determine the publishing url (like, do they cause things to get published to "/zuul") or are they like pypi and they can publish to anything that gives them access? | 14:50 |
| *** jcoufal has quit IRC | 14:50 | |
| *** shabeeb has quit IRC | 14:50 | |
| AJaeger | config-core, a couple of cleanups: https://review.openstack.org/596449 and https://review.openstack.org/596550 - could you review, please? | 14:51 |
| corvus | Shrews: oh, does pbrx_prefix determine the location? | 14:51 |
| Shrews | corvus: the credentials are for anything that wants to publish to the openstackzuul account, but could be to different "orgs" owned by that account (zuul, nodepool, etc) | 14:51 |
| Shrews | corvus: yes, under that account | 14:51 |
| Shrews | pbrx_prefix is the org | 14:52 |
| Shrews | section | 14:52 |
| Shrews | whatever you want to call it | 14:52 |
| *** felipemonteiro has joined #openstack-infra | 14:52 | |
| *** jcoufal_ has quit IRC | 14:52 | |
| openstackgerrit | Merged openstack-infra/system-config master: Scope exim service name variable https://review.openstack.org/596503 | 14:53 |
| Shrews | dockerhub calls it "organizations" or "teams" | 14:53 |
| *** rosmaita has joined #openstack-infra | 14:53 | |
| *** jamesmcarthur has joined #openstack-infra | 14:54 | |
| *** eharney has quit IRC | 14:54 | |
| corvus | Shrews: ok got it. then i think the status quo makes sense. i think we may want to think more about security at some point (is it possible for, say, the nova repo to go rogue and push up zuul images?) but i haven't wrapped my head around that yet. | 14:54 |
| Shrews | corvus: fyi, the convo that led to that: http://paste.openstack.org/show/728828/ | 14:54 |
| Shrews | corvus: that is a good question that we should consider | 14:55 |
| *** eharney has joined #openstack-infra | 14:55 | |
| *** markvoelker has joined #openstack-infra | 14:55 | |
| Shrews | b/c i think the answer is "yes", which leads to "how do we prevent/police that" | 14:55 |
| *** jamesmcarthur has quit IRC | 14:55 | |
| corvus | Shrews: for the moment, we can put "allowed-projects" on the job and whitelist them | 14:56 |
| *** jamesmcarthur_ has joined #openstack-infra | 14:56 | |
| *** gcb_ has joined #openstack-infra | 14:56 | |
| corvus | Shrews: for the doc publishing jobs, we wrote them to enforce the repo name as a prefix, so it's not possible for gertty to publish docs to nova. we could probably do something similar here to take care of the typical case, but then create special jobs for groups of repos (like zuul, nodepool, and friends) which should all publish to the same prefix. | 14:58 |
| Shrews | corvus: we will also want to consider pushing tagged releases (creating tagged containers with the version). pbrx needs a little more support for that first | 14:58 |
| Shrews | corvus: sounds reasonable | 14:59 |
| *** Bhujay has quit IRC | 15:02 | |
| * Shrews kicks gerrit in the rear | 15:02 | |
| AJaeger | corvus: for doc publishing we have already a special publish jobs for the zuul repos (sorry, missing context) | 15:02 |
| Shrews | corvus: how is var scope handled when we have multiple group_vars defining the same var name? example, https://review.openstack.org/#/c/593973/17/playbooks/group_vars/mirror.yaml and https://review.openstack.org/#/c/593973/17/playbooks/group_vars/pbx.yaml ? | 15:04 |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config master: Add openstack/os-ken to neutron https://review.openstack.org/588355 | 15:05 |
| Shrews | my gut says that could be problematic | 15:05 |
| *** bobh has joined #openstack-infra | 15:05 | |
| * Shrews reads https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#variable-scopes | 15:06 | |
| corvus | Shrews: those are for different groups, so a host would never get both of those files. a mirror host would get mirror.yaml, and pbx.openstack.org would get pbx.yaml | 15:08 |
| Shrews | but a host could be in different groups. i don't think we have that situation here, but even if we did, I think the var just gets redefined (last one read wins). something to keep in mind i guess | 15:09 |
| *** jcoufal has joined #openstack-infra | 15:09 | |
| corvus | Shrews: yep | 15:10 |
| Shrews | "Within any section, redefining a var will overwrite the previous instance. If multiple groups have the same variable, the last one loaded wins. If you define a variable twice in a play’s vars: section, the second one wins." | 15:10 |
| Shrews | unless we change hash_behaviour setting | 15:10 |
| *** bdodd_ has joined #openstack-infra | 15:11 | |
| *** bdodd has quit IRC | 15:14 | |
| openstackgerrit | wes hayutin proposed openstack-infra/system-config master: allow cr repo mirror for early testing https://review.openstack.org/596799 | 15:14 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Rename pbr jobs https://review.openstack.org/596535 | 15:15 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: remove job settings for Oslo repositories https://review.openstack.org/588842 | 15:15 |
| Shrews | hrm, my tz test is not happy | 15:15 |
| openstackgerrit | wes hayutin proposed openstack-infra/system-config master: allow cr repo mirror for early testing https://review.openstack.org/596799 | 15:16 |
| AJaeger | config-core, can you +2A https://review.openstack.org/#/c/588842/ to follow jobs moved in-repo, please? | 15:16 |
| *** r-daneel_ has joined #openstack-infra | 15:17 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Add testinfra test for timezone https://review.openstack.org/596778 | 15:17 |
| *** r-daneel has quit IRC | 15:17 | |
| *** weshay|rover has joined #openstack-infra | 15:17 | |
| *** r-daneel_ is now known as r-daneel | 15:17 | |
| weshay|rover | fyi https://review.openstack.org/#/c/596799/ | 15:18 |
| *** e0ne has quit IRC | 15:19 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Add testinfra test for timezone https://review.openstack.org/596778 | 15:19 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Add testinfra test for timezone https://review.openstack.org/596778 | 15:21 |
| fungi | popping out for lunch and some errands, but should be back in an hour or so | 15:22 |
| mnaser | infra-root: anyone has time to work on standing up a new region for nodepool? | 15:24 |
| mnaser | we'll probably just need to deploy a mirror and that should be it | 15:25 |
| openstackgerrit | wes hayutin proposed openstack-infra/system-config master: allow cr repo mirror for early testing https://review.openstack.org/596799 | 15:25 |
| *** r-mibu has quit IRC | 15:27 | |
| Shrews | mnaser: you can submit a change to http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nl03.openstack.org.yaml | 15:29 |
| *** pcaruana has quit IRC | 15:29 | |
| Shrews | assuming it's for vexxhost | 15:29 |
| mnaser | Shrews: yeah, don't we need to add a mirror manually before adding it to nodepool? | 15:29 |
| mnaser | the region is pretty far away physically | 15:29 |
| Shrews | mnaser: oh, that i'm unsure of. pabelanger and clarkb would know best | 15:30 |
| *** dpawlik has joined #openstack-infra | 15:30 | |
| mnaser | Shrews: cool, i wasn't sure about the ideal next steps to come up with that | 15:31 |
| clarkb | ya we add the mirror first, but need to knoe the region name for that. First step is adding the new region to our cloud bootstrap | 15:31 |
| clarkb | then boot mirror then update nodepool | 15:31 |
| mnaser | clarkb: sjc1, it's already available in the keystone catalog | 15:31 |
| mnaser | https://review.openstack.org/#/c/596617/ i tried here but i wasnt sure what pabelanger meant about the bridge.o.o clouds.yaml | 15:31 |
| corvus | mnaser: i think if you grep for ca-ymq-1 in system-config, you'll find all the spots to add the new one | 15:32 |
| *** alexchadin has quit IRC | 15:32 | |
| mnaser | corvus: hm, good idea in retrospect. let me do that | 15:32 |
| corvus | clarkb: welcome back! :) | 15:33 |
| clarkb | corvus: morning! | 15:33 |
| clarkb | looks like vexxhost isnt in the clouds launcher | 15:33 |
| clarkb | we should add it for the ssh key management at the very least | 15:33 |
| *** dpawlik has quit IRC | 15:34 | |
| openstackgerrit | Mohammed Naser proposed openstack-infra/system-config master: Adding SJC1 for VEXXHOST https://review.openstack.org/596617 | 15:35 |
| mnaser | clarkb: maybe that's why i wasn't able to find it | 15:35 |
| mnaser | okay let me add that in a follow up patch too i guess | 15:36 |
| *** markvoelker has quit IRC | 15:36 | |
| mnaser | hmm | 15:37 |
| mnaser | i'm trying to see how to make this change without breaking nodepool | 15:37 |
| pabelanger | mnaser: sorry, missed your review comments. You got the right files | 15:37 |
| clarkb | mnaser: https://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/clouds_layouts.yml is the cloud launcher | 15:37 |
| mnaser | i guess we'll have to rename openstackci-vexxhost to openstackci-vexxhost-mtl1 or something | 15:37 |
| mnaser | at some point | 15:37 |
| *** efried has quit IRC | 15:37 | |
| mnaser | i'll avoid that to avoid breaking stuff for now | 15:37 |
| corvus | mnaser: what object are you referring to there? | 15:38 |
| *** Bhujay has joined #openstack-infra | 15:38 | |
| corvus | (what thing is called openstackci-vexxhost that you say needs to be renamed?) | 15:38 |
| mnaser | corvus: the refernces here? https://review.openstack.org/#/c/596617/ | 15:38 |
| *** graphene has quit IRC | 15:38 | |
| mnaser | okay | 15:39 |
| mnaser | https://github.com/openstack-infra/project-config/blob/master/nodepool/nl03.openstack.org.yaml#L113 i'm thinking whatever references that | 15:40 |
| mnaser | but i guess it's not tied into region name there so it should be ok | 15:40 |
| *** graphene has joined #openstack-infra | 15:40 | |
| corvus | mnaser: that references nodepool/clouds.yaml.erb... and right | 15:40 |
| corvus | mnaser: clouds.yaml understands multiple regions per cloud. nodepool requires that we "explode" cloud*region combinations into "providers". https://github.com/openstack-infra/project-config/blob/master/nodepool/nl03.openstack.org.yaml#L113 is a provider. we'll just make a second one and change 'region-name' in it. everything else can stay the same, and 596617 is correct for doing that. | 15:41 |
| corvus | mnaser: but you're right 596617 first, then launch the mirror, then make the nl03 change | 15:42 |
| *** ykarel has quit IRC | 15:42 | |
| mnaser | corvus: awesome, sounds good, we can probably move that kata stuff too because the new region does nested virt everywhere from day 1 :) | 15:42 |
| *** promethe1nfire has joined #openstack-infra | 15:42 | |
| corvus | er, well, actually, we should probably change the nodepool builder config firest | 15:42 |
| mnaser | yeah, so you can get an image to launch stuff out of i guess? | 15:42 |
| mnaser | so maybe add it with max-servers: 0 which should get us images only | 15:42 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/system-config master: Add VEXXHOST to Ansible cloud layouts https://review.openstack.org/596823 | 15:42 |
| mnaser | thats the change for ansible ^ | 15:43 |
| *** efried has joined #openstack-infra | 15:43 | |
| mnaser | should i make a nodepool change with `max-servers: 0` so nodepool publishes images? | 15:43 |
| *** ykarel has joined #openstack-infra | 15:43 | |
| corvus | mnaser: so order is 596617, then add to project-config/nodepool/nodepool.yaml (for builder), then boot mirror, then add to nl03.openstack.org.yaml | 15:43 |
| corvus | mnaser: if you look at nodepool.yaml, you'll see how we've set up the other providers for image building without any servers | 15:44 |
| clarkb | corvus: mnaser don't forget the update to cloud launcher to configure root ssh keys and possibly security groups depending on network setup | 15:44 |
| mnaser | clarkb: 596823 does that | 15:44 |
| *** prometheanfire has quit IRC | 15:44 | |
| corvus | clarkb: that's https://review.openstack.org/596823 i think | 15:44 |
| mnaser | i will push up the nodepool change right now too, and then at that point it'll be out of my hands till we have a mirror up | 15:45 |
| mnaser | fwiw: this is a fresh region and it's running rocky, so it might be a... rocky start. | 15:45 |
| mnaser | so we might have to slowly ease into it, just in case. | 15:45 |
| corvus | okay, so revised order: 596617, 596823, nodepool.yaml, boot mirror, nl03.openstack.org.yaml :) | 15:45 |
| mnaser | aaah nodepool.yaml is the builder stuff only, okay i was confused about how to make sure it doesn't start using the nodes | 15:46 |
| corvus | yep | 15:46 |
| corvus | fungi, Shrews: i'm going to +3 the uptate-cfg-mgmt stack now | 15:48 |
| clarkb | mnaser: comment on the cloud launcher config (minor yaml formatting thing) | 15:48 |
| *** gcb_ has quit IRC | 15:48 | |
| corvus | clarkb: fyi ^ we're moving iptables, snmpd, timezone, and ntp into ansible base playbook today | 15:48 |
| clarkb | corvus: exciting | 15:48 |
| Shrews | corvus: awesome. my stomach has just +3d a lunch break | 15:48 |
| clarkb | I'm goign to try and read through email and probably dig into those changes AJaeger linked earlier, but let me know if I can help with ansible things or new vexxhost region | 15:49 |
| pabelanger | odd, how did we get infra-root keys into vexxhost if not already cloud launcher? | 15:49 |
| *** nicolasbock has quit IRC | 15:49 | |
| mnaser | clarkb: oops amateur hour | 15:49 |
| mnaser | pabelanger: the keys are probably not in the new region but in the old one by some legacy way? | 15:49 |
| *** Bhujay has quit IRC | 15:49 | |
| clarkb | pabelanger: must've been manual? | 15:49 |
| corvus | clarkb: we have a new job called 'system-config-run-base' which has a node of every type we use, and runs base.yaml for real on it, then uses testinfra to validate it worked. | 15:49 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/system-config master: Add VEXXHOST to Ansible cloud layouts https://review.openstack.org/596823 | 15:50 |
| mnaser | clarkb: addressed ^ and corvus if you don't mind i'll borrow that +2 again | 15:50 |
| pabelanger | mnaser: clarkb: must have been | 15:50 |
| * AJaeger recently gave wrong link - I wanted to ask for review of https://review.openstack.org/#/c/596535 | 15:50 | |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Start building images for vexxhost-sjc1 https://review.openstack.org/596826 | 15:50 |
| openstackgerrit | Sam Yaple proposed openstack-infra/irc-meetings master: Update meeting time and location for LOCI https://review.openstack.org/596827 | 15:51 |
| mnaser | i'm going to push up a change enabling it in nodepool but i will -W it until we're ready | 15:51 |
| corvus | clarkb: cool, i think this round is good to go, i'll watch it go in. i'm halfway through a change to implement unbound, and Shrews is looking into unattended upgrades. i think those are the last things we need to get into base.yaml. i'll loop you into that when they're ready. | 15:51 |
| corvus | mnaser: sounds good | 15:51 |
| mnaser | given that this is a newer cloud, do we feel like trying boot-from-volume again? | 15:52 |
| corvus | clarkb: ^ | 15:52 |
| mnaser | and work through fixing (if any) leaks? or not a lot of time on infra side and i should get a custom flavor? | 15:52 |
| mnaser | im open to both, but probably favouring using bfv | 15:52 |
| *** promethe1nfire is now known as prometheanfire | 15:53 | |
| pabelanger | I'd be open to fixing leaks for bfv, if we get them | 15:53 |
| openstackgerrit | Sam Yaple proposed openstack-infra/irc-meetings master: Update meeting time and location for LOCI https://review.openstack.org/596827 | 15:53 |
| clarkb | mnaser: we can certainly give it a try, boot from volume breaking is harder on the cloud than on us I think | 15:54 |
| clarkb | (we just stop booting new instances while the cloud has a sad) | 15:54 |
| mnaser | this has really beefy controllers so i think it should be much easier to handle | 15:54 |
| mnaser | as long as we don't get qcow2 images again :) | 15:54 |
| openstackgerrit | Sam Yaple proposed openstack-infra/irc-meetings master: Update meeting time and location for LOCI https://review.openstack.org/596827 | 15:54 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Start building images for vexxhost-sjc1 https://review.openstack.org/596826 | 15:58 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Start booting instances at vexxhost-sjc1 https://review.openstack.org/596830 | 15:58 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Start building images for vexxhost-sjc1 https://review.openstack.org/596826 | 15:59 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Start booting instances at vexxhost-sjc1 https://review.openstack.org/596830 | 15:59 |
| mnaser | sorry for spam, addressed comment AJaeger ^ | 15:59 |
| *** Emine has quit IRC | 16:00 | |
| corvus | ftr the quotes are not needed; i have no idea why they're there :) | 16:01 |
| mnaser | yeah, i totally understand the consistency pov but agreed :p | 16:04 |
| *** aojea has quit IRC | 16:06 | |
| *** jpena is now known as jpena|off | 16:09 | |
| *** ccamacho is now known as ccamacho|brb | 16:10 | |
| AJaeger | corvus: Ah - wasn't sure. Next time... | 16:13 |
| mnaser | hmm, it would be nice if the config validate nodepool job would get upstreamed up to zuul-jobs | 16:15 |
| efried | Hey folks, how does a new core team come into being? Sounds like it would be a patch in some governancey repo somewhere...? | 16:15 |
| mnaser | seems like something that's much more useful and very generic enough | 16:15 |
| clarkb | efried: governance repos handles the TC officialness of the thing. From Gerrit's perspective we just add the new group to the acl config file and our automated project creation tool will create the new group if it does not exist yet | 16:15 |
| AJaeger | efried: new core team as part of a project? | 16:15 |
| AJaeger | efried: basically follow http://docs.openstack.org/infra/manual/creators.html | 16:16 |
| clarkb | AJaeger: re https://review.openstack.org/#/c/596550/2 doesn't devstack-gate multinode use that config too? | 16:17 |
| efried | clarkb, AJaeger: ack. I'm trying to figure out what's required to create placement-core (which would have control over openstack/placement and openstack/placement-specs, of whose status I'm not actually sure at this point; and would be a derivative of nova-core). | 16:17 |
| clarkb | AJaeger: it is possible that we don't have anymore of those jobs, but the commit message only calls out tripleo | 16:17 |
| *** eharney_ has joined #openstack-infra | 16:18 | |
| AJaeger | clarkb: according to codesearch - nobody uses it. | 16:18 |
| *** eharney has quit IRC | 16:18 | |
| *** eharney_ is now known as eharney | 16:18 | |
| AJaeger | efried: just follow the guide and mention these ACLs | 16:19 |
| mnaser | efried: when you create the new repo in gerrit as a project, it will have a set of cores assigned | 16:19 |
| openstackgerrit | Merged openstack-infra/system-config master: Add iptables role https://review.openstack.org/593973 | 16:19 |
| openstackgerrit | Merged openstack-infra/system-config master: Add NTP service to base-server https://review.openstack.org/595312 | 16:19 |
| AJaeger | clarkb: but please double check | 16:19 |
| openstackgerrit | Merged openstack-infra/system-config master: Add timezone role https://review.openstack.org/594542 | 16:19 |
| openstackgerrit | Merged openstack-infra/system-config master: Add testinfra test for ntp https://review.openstack.org/596382 | 16:19 |
| openstackgerrit | Merged openstack-infra/system-config master: Add snmpd role and add it to base https://review.openstack.org/593969 | 16:19 |
| openstackgerrit | Merged openstack-infra/system-config master: Add testinfra test for timezone https://review.openstack.org/596778 | 16:19 |
| mnaser | if it stays under nova, then an infra root can add nova-core to that group | 16:19 |
| mnaser | and then nova-core adds whoever at that point after | 16:19 |
| AJaeger | clarkb: http://codesearch.openstack.org/?q=nodepool-provider&i=nope&files=&repos= | 16:20 |
| clarkb | AJaeger: ya looks like I was mistaken we use the mirror info script in devstack land instead | 16:21 |
| clarkb | which already processes the provider and region info | 16:21 |
| AJaeger | clarkb: double checking these is important ;) | 16:21 |
| AJaeger | mnaser: time for a quick review, please? https://review.openstack.org/#/c/596535/ follow rename/in-repo move ob jobs | 16:22 |
| openstackgerrit | Merged openstack-infra/system-config master: Adding SJC1 for VEXXHOST https://review.openstack.org/596617 | 16:22 |
| clarkb | AJaeger: and for https://review.openstack.org/#/c/594069/7 we can assume the parent change to add the jobs to horizon has merged already? (I don't see a depends on) | 16:22 |
| *** rkukura_ has joined #openstack-infra | 16:23 | |
| *** rkukura has quit IRC | 16:23 | |
| *** rkukura_ is now known as rkukura | 16:23 | |
| AJaeger | thanks, mnaser | 16:23 |
| mnaser | no problem | 16:24 |
| AJaeger | clarkb: yes, we assume that - there was an email to openstack-infra about that... | 16:24 |
| AJaeger | clarkb: https://review.openstack.org/#/q/topic:python3-first+is:open+projects:openstack/horizon | 16:24 |
| clarkb | AJaeger: ok it just says the patches were approved in the email which is maybe good enough | 16:25 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove unneeded noop-jobs from airship https://review.openstack.org/596449 | 16:26 |
| AJaeger | clarkb: so, quick sanity check like the above says it'S fine - and dhellmann gave his +1 ;) | 16:26 |
| *** ykarel is now known as ykarel|dinner | 16:26 | |
| clarkb | ya I approved it a the search above looks good | 16:26 |
| AJaeger | thanks, clarkb | 16:26 |
| *** pcaruana has joined #openstack-infra | 16:26 | |
| dhellmann | if you want to wait for a +1 from me on those I can review them all | 16:26 |
| dhellmann | I don't have to, though | 16:27 |
| Shrews | corvus: clarkb: I really didn't get too far into figuring out what we do with unattended-upgrades before I diverted to other things. If someone more knowledgeable of that domain wants in on the ansible fun, I will gladly yield. Otherwise, I'll start looking at it now. | 16:27 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove obsolete playbooks/legacy/nodepool-provider https://review.openstack.org/596550 | 16:27 |
| clarkb | dhellmann: we probably don't need to do that, and if we jump on one early it won't be the end fo the world | 16:27 |
| corvus | Shrews: i yield (i'm monitoring bridge and working on unbound) | 16:27 |
| dhellmann | clarkb : wfm | 16:27 |
| dhellmann | thanks | 16:27 |
| Shrews | corvus: but whitewashing this fence is so fun! | 16:28 |
| *** trown is now known as trown|lunch | 16:29 | |
| openstackgerrit | Merged openstack-infra/project-config master: Rename pbr jobs https://review.openstack.org/596535 | 16:31 |
| *** jpich has quit IRC | 16:32 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Update meeting time and location for LOCI https://review.openstack.org/596827 | 16:32 |
| *** ykarel|dinner has quit IRC | 16:33 | |
| openstackgerrit | Merged openstack-infra/project-config master: remove job settings for horizon repositories https://review.openstack.org/594069 | 16:33 |
| AJaeger | config-core, a few more reviews if anybody has time, please? https://review.openstack.org/596537 , https://review.openstack.org/594539 https://review.openstack.org/587768 https://review.openstack.org/592474 https://review.openstack.org/594070 | 16:33 |
| *** bobh has quit IRC | 16:36 | |
| *** rkukura has quit IRC | 16:38 | |
| AJaeger | thanks, mnaser ! | 16:38 |
| openstackgerrit | Merged openstack-infra/system-config master: Add VEXXHOST to Ansible cloud layouts https://review.openstack.org/596823 | 16:38 |
| *** rkukura has joined #openstack-infra | 16:38 | |
| mnaser | AJaeger: no problem, i did the ones that i could do on hand | 16:38 |
| mnaser | gotta do a few other things so the others are a bit more in depth :X | 16:38 |
| *** electrofelix has quit IRC | 16:38 | |
| mnaser | infra-root: i think we're ready to launch mirror01.sjc.vexxhost.openstack.org | 16:39 |
| mnaser | er | 16:39 |
| mnaser | mirror01.sjc1.vexxhost.openstack.org | 16:39 |
| *** jlviva-viva is now known as jlvillal | 16:40 | |
| *** e0ne has joined #openstack-infra | 16:41 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: move legacy requirements jobs to the requirements repository https://review.openstack.org/594539 | 16:43 |
| clarkb | mnaser: if no one beats me to it I can make an attempt at booting that on the new bridge server using the new launch node scripts in a bit | 16:44 |
| clarkb | corvus: ^ I'm assuming that at this point we shouldn't use puppetmaster or the old launch script | 16:44 |
| corvus | clarkb: i believe that's the case | 16:45 |
| openstackgerrit | Merged openstack-infra/project-config master: Use noop-jobs for astara https://review.openstack.org/596537 | 16:46 |
| AJaeger | pabelanger: can we merge https://review.openstack.org/#/c/591457/ ? Has two +2s already, just wanted to double check with you | 16:48 |
| openstackgerrit | Merged openstack-infra/project-config master: announce patches proposed to oslo stable branches https://review.openstack.org/592474 | 16:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Add translation job to zuul.d/projects.yaml https://review.openstack.org/594070 | 16:50 |
| pabelanger | AJaeger: think it is ready, yes. Maybe make sure other infra-root are onboard, since it will affect all jobs | 16:50 |
| pabelanger | I've done my best to ensure it won't break things | 16:51 |
| AJaeger | infra-root, shall we land https://review.openstack.org/#/c/591457 to change test-headers? | 16:51 |
| AJaeger | config-core, https://review.openstack.org/#/c/596538/ is now ready - cleanup to remove obsolete jobs. please review | 16:56 |
| corvus | clarkb, Shrews, fungi: i believe we're about to restart iptables on all our hosts | 16:59 |
| *** Bhujay has joined #openstack-infra | 17:00 | |
| corvus | i've spot-checked several hosts, and the files it wrote out look correct | 17:00 |
| openstackgerrit | Santiago Palenque proposed openstack-infra/openstackid master: Change text on logout screen https://review.openstack.org/596842 | 17:01 |
| clarkb | corvus: was also integration tested prior looks like | 17:01 |
| clarkb | or do we run them role by role, in any case we tested it too | 17:02 |
| corvus | ok all the new stuff (iptables, timezone, ntp, snmpd) is done now | 17:04 |
| corvus | i don't see any brokenness | 17:04 |
| *** openstackgerrit has quit IRC | 17:04 | |
| pabelanger | great work! | 17:05 |
| clarkb | Gerrit and zuul are still talking to me | 17:05 |
| *** Bhujay has quit IRC | 17:05 | |
| *** openstackgerrit has joined #openstack-infra | 17:06 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: add-fileserver: fileserver_leading_path isn't from the secret https://review.openstack.org/587768 | 17:06 |
| *** jamesmcarthur_ has quit IRC | 17:07 | |
| Shrews | corvus: w00t | 17:08 |
| AJaeger | \o/ | 17:09 |
| openstackgerrit | Julia Kreger proposed openstack-infra/project-config master: Add virtualbmc-core group https://review.openstack.org/596844 | 17:09 |
| *** bobh has joined #openstack-infra | 17:11 | |
| TheJulia | infra folk, ^^^ should be fairly strait forward, but it has been a very very very very long time since I've put in such a change. | 17:15 |
| clarkb | TheJulia: the commit message says we should add ironic-core to the new group, if that is the case can we remove the ironic-core group from that acl file? | 17:17 |
| clarkb | those existing perms will persist through the new group membership | 17:18 |
| openstackgerrit | Merged openstack-infra/zuul master: Add container push job https://review.openstack.org/593125 | 17:18 |
| *** ykarel has joined #openstack-infra | 17:20 | |
| *** dayou has quit IRC | 17:21 | |
| *** dayou has joined #openstack-infra | 17:22 | |
| TheJulia | clarkb: that is viable I think, I can go ahead and remove the ironic-core lines if you guys will add ironic-core to virutalbmc-core, and then I should be abel to add to virtualbmc core | 17:23 |
| *** dtantsur is now known as dtantsur|afk | 17:24 | |
| mnaser | can we get +2 on https://review.openstack.org/#/c/596826/3 so images can get uploaded (assuming you use those to boot up mirrors?) | 17:25 |
| clarkb | mnaser: we don't use those images to boot the mirrors, but we can upload images before mirror is up | 17:25 |
| mnaser | clarkb: ok cool, we only have bionic and centos right now i think, i can get a xenial image if needed | 17:25 |
| clarkb | mnaser: I think we may need that, we also upload the stock ubuntu xenial image if we need to | 17:26 |
| mnaser | i'm sure others will find it useful | 17:26 |
| * TheJulia steps away for a littl ebit | 17:26 | |
| *** trown|lunch is now known as trown | 17:26 | |
| mnaser | though i have noticed most customers just upload their own images | 17:26 |
| *** bobh has quit IRC | 17:29 | |
| *** bobh has joined #openstack-infra | 17:29 | |
| *** ykarel_ has joined #openstack-infra | 17:29 | |
| *** ykarel has quit IRC | 17:31 | |
| mnaser | clarkb: cool, 16.04 image up | 17:31 |
| clarkb | thanks | 17:32 |
| openstackgerrit | Merged openstack-infra/project-config master: Start building images for vexxhost-sjc1 https://review.openstack.org/596826 | 17:36 |
| Shrews | i need a centos host | 17:37 |
| Shrews | anyone know one off hand? | 17:37 |
| pabelanger | git farm is centos | 17:37 |
| pabelanger | otherwise, maybe run a test jobs in gate on centos-7 nodeset | 17:38 |
| Shrews | pabelanger: perfect, thx | 17:38 |
| fungi | corvus: thanks! i'm back now too in case firewall rules go sideways | 17:38 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 17:41 |
| Shrews | if that test actually works, and this was that simple, i may do a little dance | 17:43 |
| *** dpawlik has joined #openstack-infra | 17:47 | |
| openstackgerrit | Ben Nemec proposed openstack-dev/pbr master: Add release note for fix to bug 1786306 https://review.openstack.org/596755 | 17:51 |
| openstack | bug 1786306 in PBR "pbr doesn't work when there's subdirectory in url " [Undecided,Fix released] https://launchpad.net/bugs/1786306 - Assigned to Wei T (nuaafe) | 17:51 |
| *** dpawlik has quit IRC | 17:51 | |
| *** jamesmcarthur has joined #openstack-infra | 17:52 | |
| clarkb | I'm noticing that bridge.o.o doesn't appear to have openstack client or virtualenv installed | 17:56 |
| *** holser_ has quit IRC | 17:56 | |
| *** jamesmcarthur has quit IRC | 17:58 | |
| clarkb | hrm python3-virtualenv should be installed /me looks more | 17:58 |
| clarkb | /usr/lib/python3/dist-packages/virtualenv.py is what that installs, I'm confused | 17:59 |
| *** jamesmcarthur_ has joined #openstack-infra | 18:00 | |
| fungi | you don't actually need virtualenv with python3 | 18:01 |
| clarkb | `python3 /usr/lib/python3/dist-packages/virtualenv.py --python=python3 venv` is how you create a virtualenv if anyone is wondering | 18:01 |
| clarkb | fungi: there are some corner cases with the built in stdlib thing but ya | 18:01 |
| fungi | `python3 -m venv path/to/my/new/emv` just works in all cases i've cared about | 18:02 |
| clarkb | oh looks like 3.4 addressed the big one (installing pip) | 18:03 |
| clarkb | bridge.o.o doesn't have permissive permissions on /etc/openstack/clouds.yaml | 18:03 |
| *** r-daneel has quit IRC | 18:04 | |
| *** graphene has quit IRC | 18:04 | |
| clarkb | I'm not seeing where we install that with ansible though | 18:04 |
| clarkb | which also means that the new sjc1 region addition is missing | 18:05 |
| clarkb | corvus: mordred ^ was that added manually ? | 18:05 |
| *** graphene has joined #openstack-infra | 18:05 | |
| clarkb | I'm going to make a copy of the clouds.yaml, add it to my homedir, and edit it to keep moving | 18:06 |
| clarkb | oh except that launch node requires the ansible loop see the node now to run puppet on it | 18:07 |
| clarkb | so even if I boot a new instance it won't do anything useful until we mange clouds.yaml properly :/ | 18:07 |
| pabelanger | I don't think I have seen any patches for clouds.yaml in ansible yet | 18:08 |
| pabelanger | so, I'd guess it was manually copied from puppetmaster | 18:08 |
| pabelanger | and, I am unsure how we are getting hiera data into ansible right now | 18:09 |
| clarkb | https://review.openstack.org/#/c/593029/ is the closest thing I see, but the followup to write teh clouds.yaml files is missing | 18:09 |
| clarkb | any objection to me manually editing the global file once I test and confirm my local edits work? | 18:09 |
| pabelanger | I guess puppetmaster is no longer getting updated? | 18:10 |
| clarkb | ya we turned off updates there before I went on vacation. I think it won't get the updates we need either | 18:10 |
| fungi | i think puppetmaster is a hair's breath away from being ready for deletion. afaik it's no longer used for anything and is just there in case we missed copying something | 18:11 |
| *** r-daneel has joined #openstack-infra | 18:12 | |
| fungi | s/breath/breadth/ | 18:12 |
| pabelanger | will differ to other infra-root, I haven't had a chance to help too much with ansiblify. However, given we have a base testing in place now, I don't believe adding clouds.yaml would be the hard part. I'm unsure the plan to move secrets into ansible right now | 18:12 |
| clarkb | pabelanger: the secrets are shared between pupept and ansible now so both can access what was the hieradata, the biggest issue is not wanting to step on mordreds change above that seems to be a prereq for the actual change? that said I'm not sure I follow why the two have to be coupled like that | 18:13 |
| pabelanger | ah, so ansible can read the hieradata? | 18:14 |
| clarkb | (we should be able to have ansible write the clouds.yaml and pupept write the ca file) | 18:14 |
| clarkb | pabelanger: yup | 18:14 |
| pabelanger | neat, didn't realize you could do that | 18:14 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 18:14 |
| clarkb | pabelanger: hieradata now lives in /path/to/hiera/host_vars and /path/to/hiera/group_vars | 18:14 |
| clarkb | and that path is the ansible path :) | 18:14 |
| *** florianf has quit IRC | 18:14 | |
| pabelanger | in that case, yah. Adding clouds.yaml should just work then | 18:16 |
| pabelanger | I can try and do that, to learn more about the new system | 18:16 |
| clarkb | looks like the ca-ymq-1 mirror is booted from volume, but I'm not seeing how we specify the volume size in launch-node (so I'm digging into how that all works now) | 18:19 |
| clarkb | maybe we precreate the volume, then set volume and boot from volume? | 18:19 |
| corvus | reading backscroll | 18:20 |
| *** bdodd_ has quit IRC | 18:22 | |
| *** bdodd has joined #openstack-infra | 18:23 | |
| clarkb | no reading launch node I don't think that will do what we want, shade does take a volume_size argument to create server that I think we want to support in launch node. I'll locally add that and use this to test it | 18:23 |
| corvus | clarkb: i agree, we lack the ansible to update clouds.yaml on bridge, and manually keeping it in sync with what's in system-config is probably what we want now. | 18:24 |
| corvus | clarkb: because i think 593029 should be reworked as a per-host playbook | 18:25 |
| *** ykarel_ is now known as ykarel | 18:25 | |
| clarkb | corvus: ok I'll update the global value shortly | 18:25 |
| *** gyee has joined #openstack-infra | 18:27 | |
| *** gyee has quit IRC | 18:28 | |
| *** gyee has joined #openstack-infra | 18:28 | |
| clarkb | I have something booting, the global clouds.yaml was updated to add sjc1 region to vexxhost | 18:35 |
| *** ykarel is now known as ykarel|away | 18:38 | |
| *** eernst has joined #openstack-infra | 18:45 | |
| pabelanger | clarkb: https://review.openstack.org/554313/ | 18:46 |
| pabelanger | bfv for mirror server | 18:46 |
| clarkb | mnaser: ok I've managed to boot a server but ssh is failing to talk. Double checking the security groups those look good, I booted from volume using flavor v2-standard-2 and image 'Ubuntu 16.04 LTS (x86_64) [2018-08-24]' according to the nova api this seems to have worked, but I do have Keystone catalog entry not found (service_type=volume,service_name=Noneinterface=public,region_name=sjc1) (maybe | 18:46 |
| clarkb | you only have volume2 there?) | 18:46 |
| mnaser | i think we have only volumev2 because volume is not longer a thing? | 18:46 |
| mnaser | this is where mordred comes out screaming but let me check | 18:47 |
| *** olivierb_ has joined #openstack-infra | 18:47 | |
| notmyname | were there any zuul jobs lost in the last 3 days? something restarted or anythign else requireing re-enqueuing jobs? | 18:47 |
| clarkb | mnaser: is boot from volume expected to work with that flavor? it had a disk size of 0 so I assumed that it was I should do | 18:47 |
| mnaser | clarkb: yeah, we do bfv only | 18:47 |
| mnaser | in montreal we just did a custom flavor because of the leak problems | 18:47 |
| mnaser | clarkb: i dont see that server? | 18:48 |
| clarkb | mnaser: gotcha, looks like it finally timed out and deleted teh instance | 18:48 |
| mnaser | ah | 18:48 |
| clarkb | mnaser: I can rerun it and have it keep the server | 18:48 |
| clarkb | I'll do that now | 18:48 |
| pabelanger | notmyname: no, I don't believe we have restarted zuul services in the last 3 days | 18:49 |
| *** olivierb has quit IRC | 18:49 | |
| notmyname | pabelanger: ok thanks. I'm trying to figure out why https://review.openstack.org/#/c/596208/ never made it to the gate queue | 18:49 |
| pabelanger | I think we did for nodepool to pick up a fix for limestone on friday | 18:49 |
| clarkb | pabelanger: thanks! that looks like what I was testing, should have someone else review and approve it :) | 18:49 |
| pabelanger | notmyname: I think https://review.openstack.org/596206/ needs to be first approved, they seem to be a stack | 18:50 |
| clarkb | notmyname: the parent change is not approved, https://review.openstack.org/#/c/596206/1 | 18:50 |
| AJaeger | infra-root, can we merge pabelanger's change" Promote test-emit-job-header change", https://review.openstack.org/591457 ? | 18:50 |
| *** eernst has quit IRC | 18:50 | |
| clarkb | mnaser: d0326e27-fc99-4f10-97ba-7c3dc743bd36 is new attempt and should be held even if we timeout | 18:50 |
| notmyname | ah. I really miss the old gerrit UI where I could clearly see the parent of a proposed patch | 18:50 |
| * mnaser checks | 18:51 | |
| pabelanger | clarkb: agree, lets ask corvus to review https://review.openstack.org/554313 and approve | 18:51 |
| mnaser | looks like it booted up and got an ip according to console log | 18:51 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Use python3 and modern ansible for launch node https://review.openstack.org/596859 | 18:51 |
| clarkb | pabelanger: ^ is realted too, python3 and ansible updates for launch node there | 18:51 |
| fungi | notmyname: if it helps (though i expect it doesn't) gertty displays the change's parent dependency like the old gerrit change screen used to | 18:51 |
| corvus | clarkb: is 50G what you used? | 18:52 |
| clarkb | corvus: no 50GB is the shade/opensatck default, I am overriding with 200 for the mirror nodes | 18:53 |
| clarkb | mnaser: ya console log seems to show a happy node, maybe the security groups aren't as happy as I thought? | 18:53 |
| weshay|rover | any love to enable the centos cr repo for mirrors https://review.openstack.org/#/c/596799/ | 18:53 |
| corvus | okay, i guess that's a sane default then, +3 | 18:53 |
| mnaser | clarkb: security groups dont have ingress | 18:53 |
| mnaser | no rule to accept traffic from 0.0.0.0/0 | 18:54 |
| mnaser | maybe you were checking the mtl region secgroup? | 18:54 |
| mnaser | (or maybe the cloud launcher hasn't ran yet) | 18:54 |
| clarkb | oh the ingress is only for the group rules | 18:54 |
| clarkb | ya that explains it | 18:54 |
| clarkb | mnaser: ya we don't have clouds.yaml on bridge.openstack.org set up yet or the cloud launcher stuff either looks like | 18:54 |
| mnaser | i pushed up a change for that | 18:55 |
| mnaser | maybe it didnt land or get pushed out or synced | 18:55 |
| clarkb | mnaser: yup, the problem is related to the transition to bridge.openstack.org from puppetmaster.openstack.org | 18:55 |
| clarkb | I can run the cloud launcher manually from bridge.o.o | 18:55 |
| mnaser | i figured it wasnt the best of times to start making changes :p | 18:55 |
| mnaser | i guess it's just a matter of running it only | 18:56 |
| *** eharney has quit IRC | 18:57 | |
| clarkb | I'll edit the .yml file to only have vexxhost sjc1 in it then run it manually while we continue to bootstrap bridge.openstack.org | 18:58 |
| openstackgerrit | Mohammed Naser proposed openstack/ansible-role-cloud-launcher master: Add functional tests with DevStack https://review.openstack.org/588594 | 19:00 |
| openstackgerrit | Mohammed Naser proposed openstack/ansible-role-cloud-launcher master: Add extra_specs to flavor https://review.openstack.org/588332 | 19:00 |
| openstackgerrit | Mohammed Naser proposed openstack/ansible-role-cloud-launcher master: Add os_project_access https://review.openstack.org/588335 | 19:00 |
| openstackgerrit | Mohammed Naser proposed openstack/ansible-role-cloud-launcher master: Add is_public to flavor https://review.openstack.org/589574 | 19:00 |
| mnaser | i forgot about those, i had them sitting in my queue ^ | 19:00 |
| clarkb | todo list for launch node related things on bridge: ansible clouds.yaml, ansible cloud launcher, update launch node for boot from volume sizing, make ansible and openstack clouds.yaml work as non root user | 19:02 |
| *** jamesmcarthur_ has quit IRC | 19:03 | |
| pabelanger | clarkb: last one is at https://review.openstack.org/596630/ but left comments with ianw about ansible group | 19:04 |
| *** jamesmcarthur_ has joined #openstack-infra | 19:04 | |
| clarkb | ok I think taht worked (it ended up failing beacuse we don't have a clouds.yaml with the jenkins account in it available on bridge yet) | 19:05 |
| clarkb | but seems to have worked on the ci account | 19:05 |
| *** jamesmcarthur_ has quit IRC | 19:06 | |
| *** jamesmcarthur_ has joined #openstack-infra | 19:06 | |
| clarkb | mnaser: it looks like deleting a server booted from volume doesn't implicitly delete the volume fwiw, there are two 200GB volumes left over in sjc1 now | 19:08 |
| clarkb | (I think this will make nodepool unhappy) | 19:08 |
| mnaser | clarkb: thats because you booted without selecting delete on terminate | 19:08 |
| clarkb | I thought that wasw the default | 19:08 |
| clarkb | at least in shade /me double checks | 19:08 |
| clarkb | wow no the default is False | 19:09 |
| clarkb | that seems broken | 19:09 |
| clarkb | I'll manually delete those and ponder if that default works for us | 19:09 |
| mnaser | yep default is not to delete on terminate | 19:11 |
| mnaser | seems a bit counterintuitive but im sure people behind that decision | 19:11 |
| mnaser | debated a long time | 19:11 |
| clarkb | ok next boot failed due to the permissions denied on ansible caches | 19:13 |
| *** pcaruana has quit IRC | 19:13 | |
| *** eernst_ has joined #openstack-infra | 19:13 | |
| clarkb | I'll redelete server and volume then boot again with sudo this time | 19:13 |
| *** diablo_rojo has joined #openstack-infra | 19:17 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add volume-size flag to launch-node https://review.openstack.org/554313 | 19:17 |
| clarkb | heh running with sudo means the virtualenv isn't used by default (even if sourced) which means it can't find os_client_config | 19:19 |
| * clarkb uses more explicit paths | 19:19 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 19:20 |
| *** ykarel|away has quit IRC | 19:20 | |
| mnaser | clarkb: sudo -E will pass env variables :) | 19:21 |
| mnaser | ..i tihnk | 19:21 |
| clarkb | mnaser: ya it also works to just give it the path to ypthon in the venv | 19:21 |
| clarkb | because imports will be relative to that path | 19:21 |
| clarkb | pabelanger: I'm not sure I understand your comment on ianw's group change | 19:21 |
| clarkb | pabelanger: I might be overthinking it but the becomes: true isn't why we are in the sudo group | 19:22 |
| pabelanger | clarkb: I was trying to say, if we are just adding a new group to share data with root users, why not just use sudo, since we are already all in that group | 19:23 |
| *** jamesmcarthur_ has quit IRC | 19:24 | |
| fungi | clarkb: hopefully https://review.openstack.org/596630 will mean we don't need to use sudo | 19:24 |
| *** olivierb has joined #openstack-infra | 19:25 | |
| fungi | oh, you were just discussing it | 19:25 |
| *** jamesmcarthur_ has joined #openstack-infra | 19:25 | |
| clarkb | mnaser: current attempt seems to be having trouble bringing up networking based on the console log | 19:26 |
| clarkb | mnaser: that might interest you if its a sad hypervisor or similar | 19:26 |
| mnaser | it's very possible | 19:26 |
| *** olivierb_ has quit IRC | 19:26 | |
| mnaser | we had 1 bad one, we fixed that one already, but this is the usefulness of this exercise i guss | 19:27 |
| *** e0ne has quit IRC | 19:27 | |
| clarkb | pabelanger: got it, ya may not be necessary to have a non sudo group that we are all in | 19:27 |
| mnaser | ah i think i know whats up | 19:28 |
| *** e0ne has joined #openstack-infra | 19:29 | |
| mnaser | clarkb: ha, landed on last hypervisor in rack which didn't have the port configured right | 19:29 |
| clarkb | mnaser: yay we are being helpful | 19:29 |
| mnaser | i see the vm up now and it looks like it was fixed intime so dhcp recovered | 19:30 |
| clarkb | and found a new bug in launch node | 19:31 |
| fungi | mnaser: speaking of bugs, is the password change option in vexxhost horizon supposed to work? i get "Error: Unable to update the user password." when trying it, which is not remarkably helpful | 19:34 |
| fungi | kudos on making usernames be uuids though ;) | 19:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Unlink proper path to ansible cache https://review.openstack.org/596873 | 19:35 |
| mnaser | fungi: yeah, i wish we can just auth on user id only and thats it | 19:35 |
| clarkb | infra-root ^ that fixes a bug I just ran into with launch node on bridge.o.o | 19:35 |
| mnaser | fungi: but i think you should be able to, are you able via the CLI? | 19:35 |
| clarkb | or at least I presume it fixes it, I haven't tested it yet | 19:35 |
| mnaser | fwiw vexxhost horizon is upstream horizon | 19:35 |
| fungi | mnaser: haven't tried the cli yet | 19:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Use ansible git task so we can set umask https://review.openstack.org/596874 | 19:35 |
| mnaser | just trying to figure out if its a horizon thing or a keystone thing | 19:36 |
| fungi | will give it a shot in a sec | 19:36 |
| pabelanger | clarkb: AJaeger: corvus: ^ should fix our 0777 git repos in the ~/src directory for nodes. | 19:36 |
| fungi | i hadn't set up my clouds.yaml for it yet | 19:36 |
| corvus | pabelanger: any idea why we're getting 0777 with git under command? | 19:36 |
| mnaser | fungi: thanks to mordred you can download one via horizon :) | 19:36 |
| pabelanger | corvus: my best guess, is command doesn't have a shell, and think umask is bashisum for debuntu? | 19:37 |
| fungi | mnaser: indeed, i forgot about that feature! | 19:37 |
| pabelanger | so we just get 0777 from git client | 19:37 |
| corvus | pabelanger: if we write "command: 'mkdir /home/zuul/foo'" does that end up 0777? | 19:37 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul master: Need SSH client for Ansible https://review.openstack.org/596875 | 19:37 |
| clarkb | mnaser: can you let me know when it is safe to boot another instance? | 19:38 |
| *** ramishra has quit IRC | 19:38 | |
| clarkb | (or should I just try again and assume I won't get that same hypervisor?) | 19:38 |
| mnaser | clarkb: oh it should be and mirror01 was up fine | 19:38 |
| mnaser | i fix it in time so dhcp was just waiting | 19:38 |
| clarkb | mnaser: ah, my local launch had timed out and failed, I will be rebuilding it :) | 19:38 |
| mnaser | consoel logs shows it booted ok because it was fixed quick enough for it to still be polling dhcp | 19:38 |
| clarkb | also to test my fix for inventory cache cleanups above | 19:39 |
| clarkb | I want to make sure things are largely working on our end too :) | 19:39 |
| pabelanger | corvus: testing | 19:40 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 19:41 |
| clarkb | 38.108.68.37 is booted and working this time. I'm going to add it to dns and accept the ssh key from bridge.o.o | 19:44 |
| clarkb | this should allow puppet to run on it and turn it into a mirror node | 19:44 |
| openstackgerrit | Merged openstack-dev/pbr master: Add release note for fix to bug 1786306 https://review.openstack.org/596755 | 19:44 |
| openstack | bug 1786306 in PBR "pbr doesn't work when there's subdirectory in url " [Undecided,Fix released] https://launchpad.net/bugs/1786306 - Assigned to Wei T (nuaafe) | 19:44 |
| clarkb | oh it already accepted the key | 19:44 |
| *** e0ne has quit IRC | 19:44 | |
| clarkb | maybe I wait for puppet to happen first before updating dns | 19:45 |
| clarkb | mnaser: not a major issue but notice that there is no ipv6 in sjc1? | 19:45 |
| mnaser | clarkb: no, we are holding off, there is another transit provider being added dsoon | 19:45 |
| mnaser | and once that one goes up we will have reliable ipv6 | 19:45 |
| mnaser | but for now, we don't have reliable ipv6, so it's not there, but it'll go up evenutally and ill give a heads up | 19:45 |
| clarkb | sounds good | 19:46 |
| pabelanger | corvus: mkdir as the proper umask settings | 19:47 |
| pabelanger | http://logs.openstack.org/21/596421/20/check/test2/5f0c823/job-output.txt.gz#_2018-08-27_19_46_20_958040 | 19:47 |
| corvus | pabelanger: okay, so it seems like it's specifically the git command when run under ansible ? | 19:48 |
| pabelanger | it seems so, I cannot reproduce the issue locally, mind you I have a newer version of git on fedora | 19:49 |
| clarkb | pabelanger: once I get this mirror into dns I'm going to see what I can do to help wtih clouds.yaml and cloud launcher stuff. Let me know if I can help with that | 19:49 |
| dmsimard | Do we not carry job runtime metrics ? i.e, how long each job took ? | 19:49 |
| clarkb | we'll need that before we can turn on nodepool so that we can have the security groups and ssh keys set on the test node account | 19:49 |
| dmsimard | I see metrics in Zuul for success/failure but not duration | 19:49 |
| clarkb | dmsimard: we do in graphite | 19:49 |
| pabelanger | clarkb: sure, I'll likely start looking into clouds.yaml later this afternoon | 19:50 |
| clarkb | that zuul pushes via statsd | 19:50 |
| dmsimard | clarkb: I thought so too, just can't find them :/ | 19:50 |
| *** holser_ has joined #openstack-infra | 19:50 | |
| corvus | dmsimard: https://zuul-ci.org/docs/zuul/admin/monitoring.html#stat-zuul.tenant.%3Ctenant%3E.pipeline.%3Cpipeline%20name%3E.project.%3Ccanonical_hostname%3E.%3Cproject%3E.%3Cbranch%3E.job.%3Cjobname%3E.%3Cresult%3E | 19:51 |
| clarkb | mnaser: Aug 27 19:49:34 mirror01 ntpd[1069]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized I don't think that is a huge concern for our mirror node but may indicate unhappy hypervisor too (this is syslog on the current mirror node) | 19:51 |
| dmsimard | corvus: yeah, that's a counter for job success or failure, not the duration of the job | 19:51 |
| corvus | dmsimard: " If the result is SUCCESS or FAILURE, Zuul will additionally report the duration of the build as a timer." | 19:51 |
| * mnaser clarkb: i think it might just be that it was out of sync on first install or something.. ntp is sync on all those machines | 19:52 | |
| dmsimard | corvus: wait so it's under the same metric ? | 19:52 |
| clarkb | mnaser: ah could be | 19:52 |
| corvus | dmsimard: yes, it's reported as a counter and a timer [we denote that in the docs too with "(counter, timer)" in the type field] | 19:52 |
| mnaser | clarkb: i guess we can investigate here in the source code how this affected this user https://stackoverflow.com/questions/47596525/python-time-kernel-reports-time-error-0x41-clock-unsynchronized | 19:52 |
| mnaser | /s | 19:52 |
| dmsimard | corvus: hmmm, I'll try and see how I can use it that way -- thanks | 19:52 |
| *** olivierb_ has joined #openstack-infra | 19:53 | |
| corvus | dmsimard: statsd has different roots for the different metric types | 19:53 |
| corvus | dmsimard: stats, stats_counters, stats_timers, etc | 19:53 |
| corvus | dmsimard: if you still can't find it, give what you have found and i'll dig it up for you | 19:54 |
| *** olivierb has quit IRC | 19:55 | |
| pabelanger | corvus: it actually might be safer to move use-cache-repos into zuul-jobs first, currently it is trusted context. That also allows rdoproject to start consuming the role downstream too, today they are using perpare-workspace for 3pci testing. | 19:57 |
| dmsimard | corvus: well, I can find the "SUCCESS" and "FAILURE" counts for a job easily.. it's the duration I'm trying to find, I'll let you know if I can't figure it out. Thanks :D | 19:57 |
| corvus | pabelanger: yeah, i think as long as it works without the cache present, that should be fine. we can probably just replace 'prepare-workspace' with it. | 19:58 |
| *** eharney has joined #openstack-infra | 19:59 | |
| fungi | hrm, anybody happen to know, is `openstack user set --password-prompt ...` an admin-only command? | 19:59 |
| *** e0ne has joined #openstack-infra | 20:00 | |
| Shrews | corvus: fyi, you still have an autohold for system-config set that you might no longer need | 20:00 |
| corvus | Shrews: ack thx i'll remove it in a bit | 20:00 |
| clarkb | fungi: I think user password set isn't | 20:00 |
| clarkb | fungi: I've definitely used it on other clouds to change our passwd after we have received donated credentials | 20:00 |
| fungi | clarkb: aha, thanks | 20:01 |
| clarkb | as a note the lag between boot insatnce with ansible and wait for ansible to run puppet on there is quite large | 20:01 |
| clarkb | looks like our ansible puppet loop isn't all that fast | 20:02 |
| fungi | for some reason `openstack user --help` doesn't say there's a password subcommand | 20:02 |
| clarkb | fungi: it does for me, but I just insatleld opensatck client on bridge in my homedir so maybe you need a newer version? | 20:03 |
| clarkb | I'm going to grab lunch while I wait for puppet to run on new mirror before setting up dns on it | 20:04 |
| fungi | --version says openstack 3.16.0 | 20:04 |
| jungleboyj | fungi: You around? | 20:05 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add unbound role https://review.openstack.org/596884 | 20:05 |
| clarkb | fungi: huh same here | 20:05 |
| clarkb | fungi: weird | 20:05 |
| jungleboyj | fungi: I see this when I used the code you sent: | 20:07 |
| jungleboyj | https://usercontent.irccloud-cdn.com/file/oJ8Qk28e/image.png | 20:07 |
| jungleboyj | As I proceed through it still looks like it wants money at checkout. | 20:07 |
| clarkb | jungleboyj: that is the discounted pass which is $300 iirc. YOu get that as an ATC. If you attend the PTG there is a code sent out for full discount the week after the PTG | 20:08 |
| fungi | jungleboyj: yeah, you want to wait for a ptg attendee code | 20:09 |
| fungi | i guess you're not a speaker this time? | 20:09 |
| clarkb | fungi: ^ we may need to be more verbose about that distinction? | 20:09 |
| jungleboyj | fungi: Oooooohhhh. | 20:09 |
| jungleboyj | fungi: No, didn't get anything accepted. Ok. I will wait until after the PTG. Thank you. :-) | 20:09 |
| fungi | jungleboyj: that e-mail is written by the events team and might benefit from a tl;dr | 20:09 |
| jungleboyj | fungi: Thanks for explaining what I was doing wrong. | 20:10 |
| fungi | but there is a paragraph in there talking about ptg attendee discounts coming in september | 20:10 |
| clarkb | ok really grabbing lunch now, back later to check on puppeting of the new mirror | 20:10 |
| mnaser | clarkb: i see images were uploaded though which is good :D | 20:10 |
| mnaser | (nodepool images that is) | 20:10 |
| fungi | clarkb: aha! i never realized `openstack [...] --help` output differs if you don't tell it a cloud/config to use | 20:12 |
| openstackgerrit | Samuel Cassiba proposed openstack-infra/project-config master: Unretire the openstack/cookbook-openstack-bare-metal repo https://review.openstack.org/596743 | 20:12 |
| fungi | if i do `OS_CLIENT_CONFIG_FILE=./clouds.yaml openstack --os-cloud vexxhost user --help` i get different options than just running `openstack user --help` | 20:13 |
| fungi | lesson learned | 20:13 |
| corvus | yeah, for a ptg/summit attendee, that email reads like "PANIC! register soon!....actually don't panic you're ok" | 20:17 |
| fungi | mnaser: so it seems i can reset my password successfully via openstackclient, so it may just be a problem at the horizon end of things | 20:17 |
| mnaser | fungi: lets look into it | 20:17 |
| corvus | so here's a fun thing -- now that we're sucessfully executing the iptables update for real in the system-config-run-base job, we lose some logging visibility halfway through the run | 20:20 |
| fungi | mnaser: ahh, might be because i was going to identity->users->change password instead of user setting->schange password | 20:20 |
| mnaser | fungi: that is exactly what i was going to say | 20:20 |
| mnaser | https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/identity/users/forms.py is the path you took | 20:20 |
| fungi | my fault for not knowing the difference between the two | 20:21 |
| fungi | yup | 20:21 |
| mnaser | i would say a bug for horizon letting you do that change | 20:21 |
| *** kgiusti has left #openstack-infra | 20:21 | |
| fungi | yeah, agreed | 20:21 |
| mnaser | https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/identity/users/tables.py#L57-L65 | 20:22 |
| mnaser | i guess it thinks you can edit user | 20:22 |
| mnaser | https://github.com/openstack/horizon/blob/master/openstack_dashboard/api/keystone.py#L938-L940 | 20:23 |
| mnaser | which is just a horizon on/off thing, not just if you have the rights to edit users in terms of policy | 20:23 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 20:23 |
| fungi | yep, that's not particularly helpful for random users to stumble over (and i'm not even _that_ random of a user) | 20:25 |
| mnaser | https://github.com/openstack/horizon/commit/117ec5ddc286d5894368da20d5c43af6df99807b | 20:25 |
| mnaser | and the commit that broke it | 20:25 |
| fungi | wow, you ran that down fast! | 20:26 |
| mnaser | i've done this once or twice or a zillion times | 20:27 |
| fungi | it shows :/ | 20:27 |
| mnaser | time to ask keystone team if there's a reasonable alternative of policy | 20:27 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Add unbound role https://review.openstack.org/596884 | 20:29 |
| corvus | clarkb, fungi, Shrews: ^ that passed tests except for linters (PS2 is a fix for that) so should be ready for review | 20:29 |
| *** nicolasbock has joined #openstack-infra | 20:30 | |
| *** rlandy is now known as rlandy|brb | 20:31 | |
| corvus | Shrews: i'm going to manually enqueue the most recent zuul ref to exercise the push job | 20:32 |
| corvus | Shrews: oh, we can't change the variable on that job because it's final: http://paste.openstack.org/show/728872/ | 20:35 |
| corvus | Shrews: i think in the current configuration, that doesn't get us anything, so we should just drop 'final' | 20:36 |
| mnaser | sheesh horizon's repo is 306 MB? | 20:36 |
| *** holser_ has quit IRC | 20:36 | |
| fungi | mnaser: i'm guessing someone committed something they shouldn't have at some point in its history | 20:37 |
| fungi | openstack-manuals is immense for similar reasons | 20:37 |
| *** ansmith has quit IRC | 20:37 | |
| *** e0ne has quit IRC | 20:37 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: pbrx-upload: remove final flag https://review.openstack.org/596889 | 20:37 |
| corvus | Shrews: ^ | 20:38 |
| Shrews | corvus: hmm, ok | 20:39 |
| Shrews | corvus: also, https://review.openstack.org/596849 for unattended-upgrades passes now | 20:40 |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Import legacy-dg-hooks-dsvm job https://review.openstack.org/596542 | 20:41 |
| clarkb | 2018-08-27 20:34:24,327 p=15374 u=root | fatal: [mirror01.sjc1.vexxhost.openstack.org]: FAILED! => {"changed": false, "module_stderr": "/bin/sh: 1: /usr/bin/python: not found\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 127} | 20:42 |
| pabelanger | bionic? | 20:42 |
| clarkb | no xenial | 20:42 |
| clarkb | I am guessing we aren't running ansible on the remote nodes under python3 afterall | 20:42 |
| fungi | heh, indeed that does seem to be the case | 20:42 |
| *** trown is now known as trown|outtypewww | 20:43 | |
| clarkb | we must be running it underpython3 on bridge itself beacuse there is no other python | 20:43 |
| mnaser | fungi: https://review.openstack.org/596890 :> | 20:43 |
| fungi | thanks mnaser! | 20:43 |
| * mnaser goes back to massive pre-OSS list of things to do | 20:43 | |
| clarkb | corvus: pabelanger any idea how our ansible setup is expecting to determine which version of python to use? | 20:44 |
| pabelanger | I am unsure which version of python we are using on remote nodes now | 20:45 |
| corvus | clarkb: i believe it should use python2 unless we specify otherwise, and we only specify otherwise for bridge | 20:45 |
| corvus | clarkb: that error looks like python2 isn't installed | 20:45 |
| clarkb | corvus: yup that is the error I think | 20:45 |
| clarkb | where do we tell it to use python3 for bridge? I am not seeing that in /etc/ansible/hosts | 20:45 |
| corvus | clarkb: and it looks like launch-node specifies python3, which is why it worked for bootstrapping? | 20:45 |
| clarkb | corvus: ya | 20:45 |
| corvus | clarkb: that seems like maybe launch-node was customized to bootstrap bridge.o.o just a little too much :) | 20:45 |
| clarkb | corvus: I think its fine to assume python3 everywhere but centos fwiw | 20:46 |
| corvus | clarkb: in system-config: playbooks/host_vars/bridge.openstack.org.yaml:ansible_python_interpreter: python3 | 20:46 |
| clarkb | aha | 20:46 |
| corvus | clarkb: then i think we can add that to the all host group | 20:46 |
| corvus | clarkb: because we also have this: playbooks/group_vars/git-server.yaml:ansible_python_interpreter: python2 | 20:46 |
| clarkb | corvus: ++ | 20:46 |
| * clarkb double checks on trusty | 20:47 | |
| clarkb | etherpad.o.o is trusty and has a python3 | 20:47 |
| fungi | yeah, that seems like it should do the trick | 20:47 |
| *** studarus has joined #openstack-infra | 20:48 | |
| corvus | Shrews: re 596849 -- what causes yum-cron to be installed? | 20:49 |
| Shrews | corvus: distro-packages var | 20:49 |
| corvus | Shrews: and it's already there? | 20:49 |
| Shrews | yeah | 20:49 |
| Shrews | base-server/vars | 20:49 |
| Shrews | base-server/vars/RedHat.yaml | 20:49 |
| corvus | Shrews: i want to say that the unattended upgrades config file is changed on our servers... maybe to have it send mail? | 20:50 |
| *** eernst_ has quit IRC | 20:50 | |
| Shrews | corvus: orly? i couldn't find that | 20:50 |
| Shrews | ze01: Unattended-Upgrade::Mail "root"; | 20:51 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Default ansible to python3 https://review.openstack.org/596894 | 20:51 |
| corvus | Shrews: yeah that one... i *think* that's not in there by default | 20:51 |
| clarkb | corvus: fungi ^ that should do it, I have no idea how safe that is in practice (my expectation is that python>=3.3 should be supported by ansible and trusty has 3.4 additionally if there is no python3 then they should just fail like my server did above with python2 which should be a noop) | 20:52 |
| *** rlandy|brb is now known as rlandy | 20:53 | |
| Shrews | corvus: oh, there is a puppet-unattended_upgrades repo we appear to use in project-config | 20:53 |
| Shrews | corvus: ok, looks like there is more work to do then. i'm EOD so i'll pick it up tomorrow | 20:54 |
| pabelanger | +1, I added a comment poiting to it too | 20:55 |
| pabelanger | pointing* | 20:55 |
| clarkb | in general I think it is a reasonable thing for us to default to python3 as part of moving away from python2. Basically python2 should become the exception ratherthan the other way around | 20:55 |
| dhellmann | corvus , AJaeger : https://review.openstack.org/596895 should help us avoid moving the integrated gate settings | 20:55 |
| *** r-daneel_ has joined #openstack-infra | 20:57 | |
| corvus | Shrews: ack, thanks! | 20:57 |
| Shrews | corvus: pabelanger: if i'm not mistaken, that repo only works on debian based system | 20:58 |
| corvus | dhellmann: thanks! lgtm | 20:58 |
| corvus | Shrews: yes; i'm not certain if the centos systems email us, or if that's an option | 20:59 |
| *** r-daneel has quit IRC | 20:59 | |
| *** r-daneel_ is now known as r-daneel | 20:59 | |
| Shrews | corvus: pabelanger: where is the class that repo provides used??? | 21:00 |
| pabelanger | Shrews: yah, see: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/automatic_upgrades.pp | 21:00 |
| Shrews | oh, nm | 21:00 |
| Shrews | yeah, i thought automatic_upgrades.pp was referencing the puppetlabs version\ | 21:00 |
| Shrews | which is how i missed this | 21:01 |
| pabelanger | Oh.. | 21:01 |
| Shrews | *sigh* | 21:01 |
| pabelanger | ah, I see | 21:01 |
| clarkb | corvus: for the iptables stuff in ansible how are we handling puppet adding service specific rules like for port 80/443 etc | 21:03 |
| *** markvoelker has joined #openstack-infra | 21:04 | |
| *** markvoelker has quit IRC | 21:13 | |
| corvus | clarkb: you can set host_vars for that | 21:15 |
| clarkb | corvus: gotcha so its all in ansible now then (makes sense) | 21:15 |
| corvus | clarkb: http://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/group_vars/gerrit.yaml | 21:16 |
| corvus | clarkb: yeah, it all got migrated to ansible | 21:16 |
| clarkb | I'm quickly learning if I don't understand something right away check system-config/playbooks/*_vars | 21:16 |
| clarkb | completely unrelated to the ansible work, any idea if the freenode spam is dying down and we can remove the +r? | 21:16 |
| corvus | clarkb: yah. tbh, i think if that holds, that would be awesome. cause boy does that file look pretty self-explanatory to me :) | 21:17 |
| ianw | corvus: can i get your thoughts on adding role integration jobs to system-config with https://review.openstack.org/#/c/596018/. i think this is what you meant about moving things out of o-z-j. i'm not sure how else to test the system-config roles that should be also zuul roles | 21:17 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: sign artifacts built with the python3 publishing job https://review.openstack.org/596901 | 21:17 |
| ianw | also, if i can get some eyes on https://review.openstack.org/#/c/590636/ that will update the wheel jobs to be all ansible. i think tonyb and myself will have to watch that carefully, but i'm hopeful it will work | 21:18 |
| fungi | config-core: it would be great to get ^ (596901) in quickly, before any more projects release without signed artifacts from the new python3 jobs | 21:19 |
| * tonyb is up for watching that stuff and trying to debug it as I'd be great to have that nailed down | 21:20 | |
| mnaser | fungi: checking | 21:20 |
| corvus | ianw: lgtm -- do you want to add a "files: roles/" matcher to that? | 21:20 |
| mnaser | fungi: done | 21:21 |
| fungi | thanks! | 21:21 |
| ianw | corvus: can do | 21:21 |
| clarkb | ianw: pabelanger has comments on https://review.openstack.org/#/c/596630/5 whihc would be good to consider (but I'd like to get that change in asap once we are happy with it as I'm finding needing to be root all the time to launch new nodes is painful) | 21:21 |
| corvus | clarkb: can you +3 https://review.openstack.org/596889 when you have a sec? | 21:27 |
| clarkb | corvus: what makes it unsafe for anyone to run? it would allow you to upload to other projects? | 21:28 |
| ianw | pabelanger: responded, i think it's better to stick to one-group-one-thing if we can; if i've understood correctly | 21:30 |
| ianw | fungi: i think probably setgid'ing the directory is a good idea. there's stuff in those directories and i'm not sure if ansible recreates it | 21:31 |
| corvus | clarkb: yeah, you can upload something to the 'zuul' project just by setting the prefix variable to zuul | 21:32 |
| clarkb | bah python3 by default breaks on centos and trusty testing | 21:36 |
| clarkb | when we run the tests we do so against localhost iirc | 21:37 |
| clarkb | so the group and host vars won't apply there and all.yaml applies | 21:37 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Create an ansible group for bridge.o.o https://review.openstack.org/596630 | 21:37 |
| openstackgerrit | Merged openstack-infra/project-config master: sign artifacts built with the python3 publishing job https://review.openstack.org/596901 | 21:38 |
| corvus | clarkb: i think i have a solution; 1 sec | 21:39 |
| clarkb | it points out that trusty doesn't have python3 by default, I should double check our other trusty nodes for python3 | 21:39 |
| clarkb | (I think we installed that as base openstack library + python + pip installation) | 21:40 |
| clarkb | but our tests nodes don't necessarily have it | 21:40 |
| pabelanger | ianw: replied, but given clarkb request to fix sudo usage on bridge.o.o, I don't have a strong opioning to -1 | 21:41 |
| *** studarus has quit IRC | 21:41 | |
| pabelanger | opinion* | 21:41 |
| fungi | pabelanger: i'm still confused by your comments | 21:42 |
| mnaser | cant we get python3 | 21:42 |
| mnaser | via scl | 21:42 |
| mnaser | https://www.softwarecollections.org/en/scls/rhscl/rh-python36/ | 21:42 |
| clarkb | mnaser: everyone i've talked to about using software collections says its a bad idea | 21:42 |
| mnaser | oh | 21:42 |
| mnaser | okay | 21:42 |
| mnaser | lol | 21:42 |
| clarkb | I'm not too worried about it though we should be able to python2 on centos and then default to 3 elsewhere. Now its just a matter of figuring out how to express that on our tests | 21:42 |
| fungi | pabelanger: the point isn't to allow users in the ansible group to effectively sudo by running ansible, it's to allow ansible when run by users in the ansible group to refresh the inventory cache so that they don't have to run ansible as root for tasks like launching new nodes | 21:43 |
| mnaser | clarkb: group_by + vars file ? | 21:43 |
| mnaser | s/vars/group_vars/ | 21:43 |
| pabelanger | fungi: right, I guess I am saying we don't need an specific ansible group for that. We could maybe use adm or existing sudo group | 21:44 |
| clarkb | mnaser: ya group_vars is how we do it today in production, the problem is tests apply to localhost I think so the all.yaml group var applies globally even to the centos nodes | 21:44 |
| pabelanger | we could add users to adm, then /var/log we get read too | 21:45 |
| clarkb | mnaser: can we make a fact specific group_vers file and have it set to python2 if centos7 is the node type? | 21:45 |
| mnaser | clarkb: https://docs.ansible.com/ansible/2.6/modules/group_by_module.html | 21:45 |
| fungi | pabelanger: this is actually a preservation of pre-ansible behavior we used to have with puppet on puppetmaster, where users of the puppet group were allowed to write to the directories puppet wanted to write to on the puppetmaster. i agree we _could_ just revert the earlier change and continue using the admin group (i don't know why we had to remove it just because ubuntu removed it), or we could | 21:45 |
| fungi | reuse the sudo group to achieve the same thing and just ignore that we're using it for more than explicitly sudo access | 21:45 |
| mnaser | clarkb: group_by: key={{ansible_os_family}} | 21:46 |
| mnaser | and now you have both a redhat and debian group | 21:46 |
| mnaser | ugh that won't work though | 21:46 |
| mnaser | because it still has to gather facts | 21:46 |
| pabelanger | fungi: the reason we removed admin group, is it was replaced by sudo group, atleast how I understood the commit message. If we update the original patch to use sudo over admin, this should restore previous functionality | 21:46 |
| mnaser | and it cant gather facts (what os it is) if it can't run python | 21:46 |
| clarkb | mnaser: ya I think this is a bit chicken and egg | 21:47 |
| mnaser | clarkb: very hacky but given centos is the only culprit here maybe you can use raw to run something and get the rc to determine if its redhat, and then use set_fact | 21:47 |
| pabelanger | if we are looking to create a new group, I think that is fine, but we could maybe use adm, but means adding our accounts to that group | 21:47 |
| pabelanger | which, is something I'd like, to stop using sudo to view logs | 21:48 |
| openstackgerrit | Merged openstack-infra/project-config master: pbrx-upload: remove final flag https://review.openstack.org/596889 | 21:48 |
| pabelanger | and this old patch I have: https://review.openstack.org/364268/ | 21:48 |
| mnaser | clarkb: "raw: type rpm" and then if it rc's == 0 use set_fact after? | 21:48 |
| pabelanger | but if others are okay with adding new group for ansible, I'm okay with it. I didn't -1 the patch | 21:49 |
| clarkb | mnaser: or check /etc/os-release | 21:49 |
| mnaser | yeah, something along those lines | 21:49 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Use python2 on centos in system-config-run-base https://review.openstack.org/596905 | 21:49 |
| corvus | clarkb: well, that ^ should fix the run-base test | 21:49 |
| mnaser | with a big "get rid of me when we're out of centos" | 21:49 |
| corvus | clarkb: but i see now that you also hit failures on the apply tests | 21:49 |
| corvus | clarkb: that would need a different mechanism, probably in spec-helper. | 21:50 |
| clarkb | corvus: ya and probably to trusty too | 21:50 |
| pabelanger | mnaser: yah, SCL has a minimal set of python3 packages, mostly for djanjo things | 21:50 |
| corvus | clarkb: oh? should we not add python3 to the default packages for trusty? | 21:50 |
| *** rcernin has joined #openstack-infra | 21:51 | |
| mnaser | we should wire up some stuff in zuul to get some python3 rpms | 21:51 |
| clarkb | corvus: that is another option and maybe reflects production better (just have to wait for new images to be built and uploaded by nodepool) | 21:51 |
| clarkb | or maybe thats a bindep thing? | 21:51 |
| openstackgerrit | Merged openstack-infra/system-config master: Add unbound role https://review.openstack.org/596884 | 21:51 |
| corvus | clarkb: oh, i thought the issue was the actual mirror host you booted was running on trusty but did not have py3 as expected | 21:52 |
| corvus | er wait, it was py2 that was missing | 21:52 |
| clarkb | corvus: no the trusty and centos tests failed on https://review.openstack.org/#/c/596894/ | 21:52 |
| *** mriedem has quit IRC | 21:52 | |
| corvus | clarkb: what os did you use for the mirror? | 21:53 |
| clarkb | trusty because we don't install python3 on our trusty test nodes (but do install python3 via puppet on production nodes) | 21:53 |
| clarkb | corvus: xenial | 21:53 |
| clarkb | it has python3 installed by default but no python2 | 21:53 |
| pabelanger | gues we can't use python3 on trusty | 21:53 |
| clarkb | pabelanger: we can | 21:53 |
| clarkb | pabelanger: we just haev to install it first | 21:54 |
| clarkb | (whcih we do with puppet) | 21:54 |
| corvus | clarkb: okay, so we forgot to install python2 on xenial hosts and forgot to install python3 on trusty hosts | 21:54 |
| pabelanger | clarkb: it says it is too old | 21:54 |
| pabelanger | http://logs.openstack.org/94/596894/1/check/system-config-run-base/89cd19b/job-output.txt.gz#_2018-08-27_21_00_50_181637 | 21:54 |
| clarkb | WOW | 21:54 |
| pabelanger | is there a py35 for trusty? | 21:54 |
| clarkb | ansible supports python2.6 but not python 3.4 | 21:54 |
| clarkb | I'm gonna need a drink | 21:54 |
| *** slaweq has quit IRC | 21:55 | |
| fungi | ansible probably supports python2.6 because red hat has users who want it on rhel6 | 21:55 |
| *** graphene has quit IRC | 21:55 | |
| clarkb | an alternative approach would be to install python2 on all our nodes (can do this in launch node) | 21:56 |
| *** hasharAway has quit IRC | 21:56 | |
| clarkb | also we cannot currently launch trusty nodes because python3 is too old there I Guess | 21:56 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: add signing steps to javascript publishing job https://review.openstack.org/596907 | 21:56 |
| dhellmann | fungi : ^^ in case no one else has already done that one | 21:56 |
| fungi | i don't think anyone has yet | 21:57 |
| *** graphene has joined #openstack-infra | 21:57 | |
| clarkb | corvus: would there be objection for me setting the python version in just group_vars/mirror.yaml to python3 for now? | 21:57 |
| clarkb | corvus: all our mirrors are currently xenial and will have python3 as a result (and this lets us continue moving forward on that without major haxs) | 21:57 |
| pabelanger | clarkb: https://github.com/ansible/ansible/pull/25548 | 21:58 |
| pabelanger | py34 was never supported it seems | 21:58 |
| corvus | clarkb: i think that's fine; maybe add a todo? | 21:58 |
| *** jamesmcarthur_ has quit IRC | 21:59 | |
| corvus | clarkb: i think to fix the apply tests with your change we'd need to add something here: http://git.openstack.org/cgit/openstack-infra/puppet-openstack_infra_spec_helper/tree/lib/puppet-openstack_infra_spec_helper/spec_helper_acceptance.rb#n5 | 21:59 |
| corvus | clarkb: probably some ruby to determine what the platform is, then add ansible_python_interpreter to line 7 | 22:00 |
| clarkb | corvus: ya and we'd need to python2 on trusty and centos and python3 everywhere else. But as mnaser points out its a weird chicken and egg to group_by with that as you need python to group_by | 22:00 |
| clarkb | (thinking about this outside of testing and in production but should fix testing too I think) | 22:00 |
| corvus | clarkb: yeah, i was just talking about testing. in prod, it seems like defaulting to py3 in all.yaml and adding exceptions to host/group vars should be okay -- the exceptions should decrease over time. | 22:01 |
| corvus | we'll approach a state where all we have is the setting in all.yaml. then, presumably, the default will change in ansible at some point and we can remove even that. | 22:02 |
| *** jamesmcarthur has joined #openstack-infra | 22:02 | |
| clarkb | ya and trusty will go away over the next six months or so | 22:03 |
| clarkb | keeps time frame for exceptions to a minimum | 22:03 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Default ansible to python3 https://review.openstack.org/596894 | 22:04 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Run ansible on mirror nodes with python3 https://review.openstack.org/596911 | 22:04 |
| clarkb | corvus: pabelanger ^ restacked to just start with python3 on the mirror nodes. Will look at making this work for testing now | 22:04 |
| *** bobh has quit IRC | 22:07 | |
| clarkb | going to make transitioning from trusty to xenail interesting too. I guess we spin up new server with launch node, then set host_vars to python3 temporarily, then delete host var override when trusty is gone | 22:07 |
| *** jamesmcarthur has quit IRC | 22:07 | |
| corvus | clarkb: xenial has both, right? so we can leave the override. but if we start upgrading from trusty to bionic, we'll need to do that. | 22:08 |
| clarkb | corvus: no xenial is python3 only by default (this is how I ended up in this situation) | 22:09 |
| *** apetrich has quit IRC | 22:09 | |
| clarkb | you can install python2 as well, but we don't do that in our bootstrapping until puppet runs | 22:09 |
| *** jamesmcarthur has joined #openstack-infra | 22:09 | |
| clarkb | but we can't run puppet without ansible :/ | 22:09 |
| corvus | clarkb: right but we typically install python2 on xenial | 22:09 |
| corvus | clarkb: we can just add python2 to the packages ansible installs | 22:09 |
| clarkb | corvus: that is still a chicken and egg though because ansible can't run to install python2 if there is no python2 | 22:10 |
| clarkb | we would need to do this as part of launch node I think | 22:10 |
| corvus | clarkb: launch-node runs ansible under py3 | 22:10 |
| clarkb | I guess it runs our regular base ansible stuff there too? then ya we could just add it there | 22:10 |
| corvus | yes | 22:10 |
| clarkb | ok let me write that change too | 22:10 |
| * clarkb writes a list first | 22:10 | |
| corvus | shouldn't it run puppet as well? if so, i wonder why that didn't install py2 | 22:11 |
| *** slaweq has joined #openstack-infra | 22:11 | |
| clarkb | it does not, mordred's plan was to have our puppet cron come through and update things normally after ansible bootstraps it | 22:11 |
| clarkb | this is the step I'm currently waiting on that is broken by lack of python2 on xenial | 22:12 |
| corvus | gotcha | 22:13 |
| corvus | so yeah, i think installing required python packages in base is sensible :) | 22:13 |
| *** jamesmcarthur has quit IRC | 22:14 | |
| *** apetrich has joined #openstack-infra | 22:14 | |
| pabelanger | we could use raw to boot strap python from ansible, done that in the past | 22:15 |
| pabelanger | bootstrap* | 22:15 |
| *** slaweq has quit IRC | 22:16 | |
| clarkb | let me try to summarize, trusty cannot ansible + python3 this means we can install python2 on xenial nodes to have transition from trusty to xenial, then when all nodes (like our mirrors) are xenial and not trusty we can switch them to python3 explicitly. Eventually we will be python3 across the board (except centos 7) and can make python3 the default and have exceptions for the small number of | 22:19 |
| clarkb | centos nodes | 22:19 |
| clarkb | this does not address the problem of launching a new trusty node though | 22:19 |
| fungi | i also have not revisited our reasons for running cgit on centos; may be time to check those assumptions again now that bionic has a newer version still than xenial | 22:21 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Install python2.7 on xenial nodes https://review.openstack.org/596913 | 22:22 |
| fungi | er, oh actually it was the git smart http backend which was the problem in xenial, wasn't it? (trying to write packfiles in tempdirs within the bare repo trees running afoul of file permissions and so breaking shallow cloning) | 22:22 |
| clarkb | corvus: pabelanger ^ I think that should handle it | 22:23 |
| clarkb | https://review.openstack.org/#/c/596911/1 is the python3 specific override for the mirror nodes that that commit message talks about as well | 22:23 |
| clarkb | fungi: yup, though I think we should test if you can shallow clone from review.o.o to see if that is still an issue | 22:26 |
| *** owalsh has quit IRC | 22:26 | |
| clarkb | if we can get reviews for https://review.openstack.org/#/c/596913/ and/or https://review.openstack.org/#/c/596911/ I will continue pushing on the vexxhost region and mirror | 22:29 |
| clarkb | while I wait on that I am going to manually trigger the cloud launcher against our test node account in that region to set up security groups and ssh keys. I think once that is one and once mirror is configured we can start booting nodes there | 22:29 |
| *** owalsh has joined #openstack-infra | 22:29 | |
| clarkb | as a followup I need to explicitly set python2 on all current trusty nodes (which I need to make a listing for figuring out upgrades anyway) but I want to help mnaser make progress on the cloud side short term | 22:31 |
| ianw | clarkb: i guess it doesn't make any difference to testinfra that because the testinfra nodes already have python2? | 22:31 |
| clarkb | ianw: https://review.openstack.org/#/c/596905/1 no I think we need a change like that and for trusty to address the base test infra stuff | 22:32 |
| *** slaweq has joined #openstack-infra | 22:32 | |
| clarkb | (but I'm still catching up on all that testing chagnes from last week so may be wrong) | 22:32 |
| clarkb | basically we need to force centos7 and trusty to python2 acorss the board (production and in testing) then we can set python3 to our global default | 22:33 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/project-config master: Enable Searchlight to Use StoryBoard https://review.openstack.org/596916 | 22:35 |
| corvus | clarkb: 596913 lgtm | 22:37 |
| corvus | and oh, run-base isn't self testing there... | 22:37 |
| *** slaweq has quit IRC | 22:37 | |
| clarkb | I should probably also save enough time at the end of my day to send an email to the infra list about what I have laerned about launching new nodes | 22:37 |
| clarkb | ok sjc1 test node account should be set up properly | 22:38 |
| *** nicolasbock has quit IRC | 22:39 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Use python2 on centos/trusty in system-config-run-base https://review.openstack.org/596905 | 22:39 |
| *** pbourke has quit IRC | 22:40 | |
| clarkb | corvus: thanks ^ lg5tm | 22:40 |
| corvus | that should add the trusty bit and also make the change self-testing | 22:40 |
| clarkb | I went ahead and manually installed python2.7 on the new sjc1 mirror node since you +2'd the change to have ansible do it | 22:41 |
| clarkb | this should get it puppeted as a mirror. Once that is done I think we are free to go ahead and have nodepool start booting instances there (cloud launcher is done) | 22:41 |
| clarkb | mnaser: ^ fyi | 22:41 |
| clarkb | oh and I need to add dns | 22:41 |
| *** pbourke has joined #openstack-infra | 22:41 | |
| corvus | (i wonder if zuul could just declare a file matcher as matched if a job is modified...) | 22:41 |
| *** jamesmcarthur has joined #openstack-infra | 22:44 | |
| pabelanger | ianw: I've been experimenting with molecule and docker images, for testing ansible things. I've long though of writing an role in ansible, using raw tasks, to bootstrap ansible. Been considering doing it for a friday afternoon project. | 22:44 |
| *** nicolasbock has joined #openstack-infra | 22:44 | |
| openstackgerrit | Kendall Nelson proposed openstack-infra/project-config master: Add Freezer to StoryBoard https://review.openstack.org/596918 | 22:44 |
| *** jamesmcarthur has quit IRC | 22:48 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Install python2.7 on xenial nodes https://review.openstack.org/596913 | 22:52 |
| clarkb | apparnetly if you want /usr/bin/python you have to install the `python` package too | 22:53 |
| clarkb | anyway that should do it I hop | 22:53 |
| clarkb | now to figure out if beaker-rspec exposes the operating system and release to me in some usful way | 22:53 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-openstack_infra_spec_helper master: Use python2 with ansible on Trusty and Centos 7 https://review.openstack.org/596922 | 23:00 |
| clarkb | corvus: I think ^ should do it | 23:00 |
| *** bnemec has quit IRC | 23:00 | |
| clarkb | (gave up on figuring out the ruby and just did it via the shell <_<) | 23:00 |
| clarkb | now to add some depends on to see if that fixes it | 23:00 |
| *** graphene has quit IRC | 23:01 | |
| *** bnemec has joined #openstack-infra | 23:01 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Default ansible to python3 https://review.openstack.org/596894 | 23:02 |
| *** graphene has joined #openstack-infra | 23:02 | |
| *** nicolasbock has quit IRC | 23:07 | |
| *** tosky has quit IRC | 23:08 | |
| *** r-daneel has quit IRC | 23:08 | |
| *** jamesmcarthur has joined #openstack-infra | 23:12 | |
| *** jcoufal has quit IRC | 23:16 | |
| *** jamesmcarthur has quit IRC | 23:16 | |
| clarkb | sent a followup on all that I have laerned today to the infra list, hopefully that is useful | 23:23 |
| *** Tahvok_ has joined #openstack-infra | 23:24 | |
| *** Tahvok has quit IRC | 23:24 | |
| *** Tahvok_ is now known as Tahvok | 23:24 | |
| *** boden has quit IRC | 23:25 | |
| * mnaser reads buffer | 23:25 | |
| *** eventingmonkey has quit IRC | 23:26 | |
| corvus | clarkb: ha, nice :) | 23:26 |
| mnaser | dhellmann / fungi: pushed out the change for signed js jobs | 23:26 |
| *** Jeffrey4l has quit IRC | 23:26 | |
| *** eventingmonkey has joined #openstack-infra | 23:27 | |
| fungi | thanks mnaser! | 23:27 |
| clarkb | corvus: looks like that change doesn't work because inventory values have lowest precedence | 23:27 |
| clarkb | or wait no inventory host var should override group_vars | 23:28 |
| *** Jeffrey4l has joined #openstack-infra | 23:28 | |
| corvus | oooh i think i see the problem with run-base at least | 23:29 |
| *** zul has quit IRC | 23:29 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: write-inventory: add ansible_python_interpreter variable https://review.openstack.org/596961 | 23:30 |
| corvus | clarkb: ^ | 23:30 |
| corvus | but i don't know why apply would fail | 23:31 |
| corvus | clarkb: http://logs.openstack.org/94/596894/3/check/infra-puppet-apply-3-centos-7/71d81db/job-output.txt.gz#_2018-08-27_23_12_56_977244 | 23:33 |
| mnaser | clarkb: ok cool, i guess once dns is done we can push https://review.openstack.org/#/c/596830/ through? | 23:33 |
| clarkb | oh I think I know | 23:33 |
| mnaser | i removed my -W so feel free to push it through and monitor when you want | 23:33 |
| clarkb | corvus: the spec helper doesn't apply to the apply tests only the spec tests | 23:33 |
| *** dingyichen has joined #openstack-infra | 23:34 | |
| corvus | there's a *third* place we install ansible and run base? | 23:34 |
| corvus | clarkb: tools/prep-apply.sh | 23:35 |
| clarkb | corvus: yup, I'll update the patch in a sec | 23:35 |
| corvus | basically the same thing it looks like | 23:35 |
| clarkb | corvus: tests/puppet_apply.sh is where we write the inventory file | 23:35 |
| clarkb | er tools/ | 23:36 |
| openstackgerrit | Merged openstack-infra/project-config master: add signing steps to javascript publishing job https://review.openstack.org/596907 | 23:36 |
| *** tpsilva has quit IRC | 23:39 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Default ansible to python3 https://review.openstack.org/596894 | 23:39 |
| clarkb | corvus: ^ that should do it maybe? | 23:39 |
| openstackgerrit | Merged openstack-infra/system-config master: Use python2 on centos/trusty in system-config-run-base https://review.openstack.org/596905 | 23:40 |
| *** anteaya has joined #openstack-infra | 23:42 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Install limestone CA on openstack using hosts https://review.openstack.org/593029 | 23:43 |
| corvus | clarkb: seems about right | 23:43 |
| *** jamesmcarthur has joined #openstack-infra | 23:43 | |
| corvus | clarkb: may need a depends-on: https://review.openstack.org/596961 | 23:43 |
| clarkb | mnaser: yup, if the mirror node turns into a mirror node before I have to afk for dinner I'll +2 that change to indicate I think all other pieces are good to go | 23:44 |
| clarkb | just waiting for puppet to run now | 23:44 |
| clarkb | 2018-08-27 23:43:20,169 p=29181 u=root | fatal: [mirror01.sjc1.vexxhost.openstack.org]: FAILED! => {"changed": false, "msg": "Could not find the requested service puppet: host"} | 23:45 |
| clarkb | it seems that may be we don't install puppet anymore with launch node? | 23:45 |
| clarkb | yup, so this is another missing piece | 23:47 |
| clarkb | we need a way to install puppet during launch node for nodes that should continue to be puppeted. I'll follow up on my email but tonight is last night my parents are here so I need to pop out shortly for dinner | 23:47 |
| *** dpawlik has joined #openstack-infra | 23:47 | |
| ianw | clarkb: i can take a look if you like | 23:47 |
| clarkb | ianw: sure if you want to sort out how you think we should be doing that with launch node mirror01.sjc1.vexxhost.openstack.org is a good test node :) | 23:48 |
| mnaser | might be slight less latency for you :P | 23:48 |
| clarkb | ianw: feel free to delete that instance and its volume as well if you want to start over | 23:49 |
| clarkb | launching a new one you will need to set the boot from volume flag and I was setting the volume size to 200 (change for that in launch node just merged) | 23:50 |
| pabelanger | ianw: clarkb: I'd guess making line 140 in https://review.openstack.org/#/c/592765/7/launch/launch-node.py:140 condition would help | 23:50 |
| pabelanger | might be the simplest fix for now | 23:50 |
| clarkb | ya, I think monty had intended on decoupling puppetizing from launch node | 23:50 |
| clarkb | instead we may just want a followup playbook to install puppet | 23:51 |
| clarkb | (and we can update the launch readme with that info) | 23:51 |
| ianw | yeah that was what i was thinking | 23:51 |
| pabelanger | that would work also | 23:51 |
| ianw | the only thing is, install_puppet.sh has quite a winding path | 23:51 |
| ianw | it's not just "apt-get install puppet" | 23:51 |
| clarkb | ianw: maybe just run install_puppet.sh with ansible to start? | 23:51 |
| *** dpawlik has quit IRC | 23:52 | |
| ianw | clarkb: yes, brute force works :) | 23:52 |
| ianw | there's the puppet-collections stuff, then we use system packages for trusty(? maybe, can't remember) then back to puppetlabs for bionic | 23:54 |
| *** jamesmcarthur has quit IRC | 23:54 | |
| clarkb | on the plus side I've leanred alot about the new setup going through all this :) | 23:56 |
| *** jamesmcarthur has joined #openstack-infra | 23:56 | |
| ianw | i wonder though, if install_puppet.sh i idempotent | 23:56 |
| ianw | is | 23:56 |
| clarkb | I don't think we ever ran it that way at least | 23:56 |
| clarkb | (it was only ever run when a new ndoe was launched) | 23:56 |
| clarkb | ok gonna step away for a bit now. Feel free to use that instance as a test node. It isn't even in dns yet | 23:57 |
| ianw | yeah, which means a hacky stamp-file type thing to use it as-is i guess | 23:57 |
| clarkb | ianw: could just make it a followup step to launch node in the launch readme | 23:57 |
| clarkb | ianw: rather than run it via cron just run it the once after running launch-node.py | 23:57 |
| *** jamesmcarthur has quit IRC | 23:58 | |
| ianw | let me see how bad it looks as an ansible role, i might be over thinking it. otherwise we can look at plan b | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
