| *** akrpan-pure has quit IRC | 00:19 | |
| tristanC | when akrpan-pure comes back, we might want to try running the same command with REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-bundle.crt | 00:33 |
|---|---|---|
| *** lseki has joined #softwarefactory | 01:44 | |
| *** lseki has quit IRC | 04:11 | |
| *** jpena|off is now known as jpena | 08:17 | |
| sfbender | Merged software-factory/sf-config master: Avoid wipe of /var/lib/software-factory/ansible/group_vars/all.yaml https://softwarefactory-project.io/r/20404 | 10:19 |
| sfbender | Merged software-factory/sf-elements master: nodepool-minimal: ensure the ca trust source directory exists https://softwarefactory-project.io/r/20377 | 10:22 |
| sfbender | Javier Peña created DLRN master: [kojidriver] Refresh downstream git before updating hash https://softwarefactory-project.io/r/20414 | 11:45 |
| *** rfolco has joined #softwarefactory | 12:37 | |
| *** jpena is now known as jpena|lunch | 13:00 | |
| *** apevec has joined #softwarefactory | 13:32 | |
| *** jpena|lunch is now known as jpena | 14:00 | |
| sfbender | Tristan de Cacqueray created software-factory/dhall-zuul master: examples: normalize main package binding https://softwarefactory-project.io/r/20415 | 14:04 |
| sfbender | Tristan de Cacqueray created software-factory/dhall-zuul master: Add Zuul.Pipeline https://softwarefactory-project.io/r/20416 | 14:04 |
| sfbender | Tristan de Cacqueray created software-factory/dhall-zuul master: Add Zuul.Pipeline.periodic helper function https://softwarefactory-project.io/r/20419 | 16:47 |
| *** akrpan-pure has joined #softwarefactory | 17:04 | |
| akrpan-pure | Hey tristanC, saw your message last night, I'll give that a try (REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-bundle.crt) | 17:04 |
| *** jpena is now known as jpena|off | 17:16 | |
| sfbender | Tristan de Cacqueray created software-factory/dhall-zuul master: Add Zuul.Pipeline.post helper function https://softwarefactory-project.io/r/20420 | 17:25 |
| akrpan-pure | Sweet, that worked! Thanks again tristanC, I'm sure I'll have some other issue in a bit but making good progress now | 17:26 |
| tristanC | akrpan-pure: great, that's good to know. I guess we forgot to add that environment variable for local sfmanager usage when using self signed certificate | 17:27 |
| akrpan-pure | Well now I'm in Gerrit (User Settings -> HTTP Password) and trying to generate a new HTTP password to push some config updates and it's throwing 403's at me; is this even the right way to go about pushing config changes? I couldn't get SSH keys to work right | 17:33 |
| akrpan-pure | ... oh I'm dumb, disregard, found out how | 17:35 |
| *** lseki has joined #softwarefactory | 18:16 | |
| akrpan-pure | Is the config-update job supposed to show up in Zuul? I don't see it in there (and thus am unable to get my nodepool changes to apply) | 18:33 |
| akrpan-pure | NVM, got it manually | 18:41 |
| akrpan-pure | And noooowwww the config-update job has a permission denied (publickey) trying to run the update playbook | 18:54 |
| akrpan-pure | And also permission denied trying to create the logs directory | 18:55 |
| akrpan-pure | Looks like /var/www is owned by root:root, and /var/www/logs doesn't exist | 18:57 |
| tristanC | akrpan-pure: what is the name of the failing task? | 19:24 |
| akrpan-pure | In the job itself it failed on "run config update" (that's the publickey error), the post-run failed on "Create log directories" with "There was an issue creating /var/www/logs as requested: [Errno 13] Permission denied: '/var/www/logs'" | 19:26 |
| akrpan-pure | Also my apologies that I just keep dumping errors here XD I'm never quite sure if it's a misconfiguration on my side or if I found an actual issue, so I want to check just in case | 19:27 |
| tristanC | akrpan-pure: trying to understand the ssh permission denied, could you check the `fqdn` attribute of the secret in your config project zuul.d/_secret_install_server.yaml file is equal to the hostname of the install server? | 19:39 |
| akrpan-pure | FQDN matches, yep | 19:39 |
| tristanC | and from the install-server, could you run `chmod 400 /var/lib/software-factory/bootstrap-data/ssh_keys/zuul_worker_rsa; ssh -i /var/lib/software-factory/bootstrap-data/ssh_keys/zuul_worker_rsa root@localhost` ? | 19:41 |
| akrpan-pure | SSH works with that | 19:42 |
| akrpan-pure | Want me to re-run the config job? | 19:42 |
| tristanC | akrpan-pure: just to be sure it's not a fluke yes please | 19:44 |
| tristanC | akrpan-pure: otherwise it's an odd issue, looking at the `run config update` task, it should be executing something similar to the test ssh command | 19:44 |
| akrpan-pure | Same failure, hrmmm | 19:45 |
| tristanC | akrpan-pure: how about the `ssh_known_hosts` attribute of the install_server_.yaml secret, does it match too? | 19:47 |
| akrpan-pure | It does match one (not all) in /root/.ssh/known_hosts, if that's what you mean? | 19:48 |
| tristanC | it needs to match the output of `ssh-keyscan -t rsa install-server-host` | 19:49 |
| akrpan-pure | Now that everything is in one node (including install server), it does match if I run `ssh-keyscan -t rsa os-ci-sf-all.cinder-ci.dev.purestorage.com` (which is the FQDN of the node) | 19:50 |
| akrpan-pure | Also sorry, heading out for an hour then stuck in meetings, but if you leave anything to run I'll do it asap | 19:50 |
| tristanC | if the services was moved after deployment, then it might be worth restarting zuul to ensure it is using the right secrets | 19:51 |
| tristanC | e.g., when sf-config push config repo change to update secrets, i think we need to force a zuul-scheduler reconfiguration to ensure it is in sync | 19:53 |
| tristanC | and about the post-run failure, it seems like it is not running on the correct logserver host either, e.g. the host should have been configured with: https://softwarefactory-project.io/cgit/software-factory/sf-config/tree/ansible/roles/sf-logserver/tasks/setup.yml#n13 | 19:56 |
| *** apevec has quit IRC | 20:06 | |
| sfbender | Tristan de Cacqueray created software-factory/dhall-zuul master: Add Zuul.Semaphore https://softwarefactory-project.io/r/20423 | 20:08 |
| sfbender | Merged software-factory/dhall-zuul master: examples: normalize main package binding https://softwarefactory-project.io/r/20415 | 20:39 |
| *** rfolco has quit IRC | 20:40 | |
| *** rfolco has joined #softwarefactory | 20:40 | |
| sfbender | Merged software-factory/dhall-zuul master: Add Zuul.Pipeline https://softwarefactory-project.io/r/20416 | 20:58 |
| sfbender | Tristan de Cacqueray created software-factory/dhall-nodepool master: Add 0.2.0 release notes https://softwarefactory-project.io/r/20425 | 20:58 |
| sfbender | Merged software-factory/dhall-zuul master: Add Zuul.Pipeline.periodic helper function https://softwarefactory-project.io/r/20419 | 20:59 |
| sfbender | Merged software-factory/dhall-zuul master: Add Zuul.Pipeline.post helper function https://softwarefactory-project.io/r/20420 | 21:01 |
| sfbender | Merged software-factory/dhall-nodepool master: Add 0.2.0 release notes https://softwarefactory-project.io/r/20425 | 21:02 |
| sfbender | Merged software-factory/dhall-zuul master: Add Zuul.Semaphore https://softwarefactory-project.io/r/20423 | 21:04 |
| *** sduthil has quit IRC | 22:16 | |
| akrpan-pure | tristanC: I didn't do any moves, this is a one-node install that's been one-node since I installed it, that's why it's so confusing. Is the logserver host configurable in sfconfig.yaml? Or is it autogenerated? | 22:28 |
| tristanC | akrpan-pure: then there is some wrong somewhere... let's see, in the other issue you mentioned /var/www being owned by root:root | 22:32 |
| akrpan-pure | Correct, yeah | 22:32 |
| tristanC | akrpan-pure: that seems to indicate the setup phase didn't worked as expected, it should have gone through https://softwarefactory-project.io/cgit/software-factory/sf-config/tree/ansible/roles/sf-logserver/tasks/setup.yml#n13 | 22:33 |
| akrpan-pure | Want me to run sfconfig again? Perhaps with skip install? | 22:33 |
| tristanC | akrpan-pure: i meant /var/www/logs not existing (or owned by root:root) | 22:33 |
| akrpan-pure | Yeah, /var/www/logs doesn't exist and /var/www is owned by root:root | 22:34 |
| tristanC | akrpan-pure: yes please, re-run sfconfig and makes sure your arch file at least contains the role listed here: https://softwarefactory-project.io/cgit/software-factory/sf-config/tree/refarch/minimal.yaml | 22:34 |
| akrpan-pure | It's got all the roles: running again now | 22:35 |
| sfbender | Merged software-factory/sf-config master: Feed custom-links from sfconfig.yaml to info.json https://softwarefactory-project.io/r/20403 | 22:35 |
| akrpan-pure | Just hit that ansible step and it says changed, so that's a good sign: wonder why it didn't get caught beforE? | 22:38 |
| tristanC | that's a good question, it's odd because it seems like further task got applied (such as the install-server authorized_keys for zuul_worker) | 22:39 |
| akrpan-pure | I know the first time I ran it it hit a Kibana error because it wasn't up all the way (503 I think), then I ran it again and it fixed it since it was up | 22:40 |
| akrpan-pure | Yeah, this re-run has changed=29, ok=830. Some were expected because of the fact I've changed some config before, but some were definitely missed steps | 22:41 |
| akrpan-pure | *shrug* | 22:41 |
| tristanC | hmm, weird. let's see if config-update works now | 22:43 |
| akrpan-pure | Hurrah! It all worked, both logs and the config-update itself | 22:44 |
| akrpan-pure | I guess something just *didn't finish* for some reason | 22:44 |
| tristanC | ha great, that's good to hear! | 22:44 |
| tristanC | akrpan-pure: i'm about to go on vacation until next year, but the other maintainers might be able to help next week if you have other issues. Just ask here or through taiga | 22:46 |
| akrpan-pure | Sure thing, thanks a ton for all your help! Hopefully at this point it's like old nodepool/zuul config that I'm kinda familiar with so I can wrestle with that | 22:46 |
| tristanC | akrpan-pure: you're welcome, happy to help :-) | 22:48 |
| akrpan-pure | And the cherry on top, my nodepool config change went through and the image is already uploaded | 22:49 |
| akrpan-pure | Admittedly thoroughly impressed with the end result, definitely liking what you all have built here, just have to get it set up first :P | 22:49 |
| tristanC | thank you for the kind words, it's good to see software factory being put to work | 22:50 |
| akrpan-pure | Definitely. Have a good vacation! | 22:55 |
| *** akrpan-pure has quit IRC | 22:56 | |
| *** dmsimard5 has joined #softwarefactory | 23:50 | |
| *** dmsimard has quit IRC | 23:52 | |
| *** dmsimard5 is now known as dmsimard | 23:52 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!