| *** ysandeep|out is now known as ysandeep | 03:24 | |
| *** ysandeep is now known as ysandeep|afk | 04:34 | |
| *** ysandeep|afk is now known as ysandeep | 05:45 | |
| *** ysandeep is now known as ysandeep|afk | 07:28 | |
| jrosser_ | have the ironic jobs started to "just work" ? | 07:47 |
|---|---|---|
| noonedeadpunk | well, at least your patch failed in different place... | 07:48 |
| noonedeadpunk | because of oom | 07:48 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add log collection from /etc/ironic-inspector https://review.opendev.org/c/openstack/openstack-ansible/+/845891 | 07:48 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Define corosync interface for facts gathering https://review.opendev.org/c/openstack/openstack-ansible/+/845894 | 08:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_masakari master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_masakari/+/845798 | 08:01 |
| noonedeadpunk | manila failure is interesting... For some reason here https://opendev.org/openstack/openstack-ansible-os_manila/src/branch/master/tasks/manila_backends.yml#L34-L45 it tries to reach keystone through http. Likely smth off with openrc file... | 08:22 |
| * noonedeadpunk spawning aio | 08:24 | |
| jrosser_ | --insecure means http rather than "dont validate"? | 08:26 |
| noonedeadpunk | mmm.... | 08:29 |
| noonedeadpunk | I haven't checked that actually | 08:29 |
| jrosser_ | `--verify | --insecure. Verify or ignore server certificate (default: verify)` | 08:32 |
| jrosser_ | though thats for the `openstack` command - manila cli could easily have some special behaviour/bugs here | 08:32 |
| noonedeadpunk | yeah, it's as always, already found in code | 08:32 |
| noonedeadpunk | https://github.com/openstack/python-manilaclient/blob/master/manilaclient/shell.py#L551 | 08:32 |
| noonedeadpunk | I wonder if it would be more easy just to provide auth_url there as well | 08:33 |
| noonedeadpunk | but yeah, will check what's inside openrc first... | 08:34 |
| noonedeadpunk | jrosser_: Can I ask you to take a look on https://review.opendev.org/q/topic:osa%252Fservice_tokens ? As if it looks fair I will go on pushing this to all other service repos | 08:43 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/845903 | 09:08 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/845904 | 09:14 |
| jrosser_ | it looks reasonable - though i find the roadmap very confusing about this stuff | 09:17 |
| noonedeadpunk | well... We should have had that somewhere in Ocata I believe | 09:18 |
| jrosser_ | some of this is related to application credentials isnt it | 09:18 |
| noonedeadpunk | service_token_roles likely does | 09:18 |
| noonedeadpunk | no, sory, worng paste | 09:18 |
| noonedeadpunk | service_type | 09:19 |
| noonedeadpunk | And system_scope is another beast kind of | 09:20 |
| noonedeadpunk | But I believe we can solve that just in service_setup role that is in plugins now | 09:21 |
| noonedeadpunk | Once sdk/collection will be ready... | 09:21 |
| noonedeadpunk | But roadmap there is also weird/vague | 09:21 |
| jrosser_ | ok the patches look reasonable - I see that we need to land the support then switch openstack_service_token_roles_required during Z cycle | 09:29 |
| noonedeadpunk | yup | 09:29 |
| noonedeadpunk | it's missing a release note though | 09:30 |
| noonedeadpunk | will push general one shortly | 09:30 |
| *** ysandeep|afk is now known as ysandeep | 09:38 | |
| *** tosky_ is now known as tosky | 09:42 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/845913 | 09:50 |
| noonedeadpunk | ok, I reproduced issue with manila | 09:54 |
| noonedeadpunk | and it;s interesting | 09:55 |
| noonedeadpunk | https://paste.openstack.org/show/bqU96B8sXdmChpNMD9SO/ | 09:57 |
| jrosser_ | errrr | 09:58 |
| noonedeadpunk | yeah, as I said it's interesting :) | 10:01 |
| noonedeadpunk | having latest client doesn't help obviously | 10:03 |
| noonedeadpunk | `openstack share type list` seems working though | 10:04 |
| noonedeadpunk | or well | 10:04 |
| noonedeadpunk | SSLCertVerificationError | 10:05 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila master: Delegate share type creation to setup host https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/845917 | 10:18 |
| noonedeadpunk | But for SSLCertVerificationError it;s just I needed to re-load session | 10:19 |
| jrosser_ | i looked a bit at the config_template tests failing, thats a bit odd too https://paste.opendev.org/show/bpq2LHy6W7zThlnuKl5L/ | 10:38 |
| noonedeadpunk | huh, I believe I was fighting/fixing that | 10:52 |
| jrosser_ | it seems to ignore indent | 10:53 |
| jrosser_ | i was looking at how to_nice_yaml worked https://github.com/ansible/ansible/blob/f270b4e224174557963120e75bfc81acf1cdde61/lib/ansible/plugins/filter/core.py#L57-L63 | 10:54 |
| jrosser_ | i'm also not entirely sure that the tests are actually testing what we think they are | 10:54 |
| jrosser_ | as config_template is also installed as a collection | 10:54 |
| noonedeadpunk | I guess they do as all rest are jsut symlinks | 10:55 |
| noonedeadpunk | maybe I'm mixing config_template with release tool.... | 10:55 |
| noonedeadpunk | as there I also was missing indent | 10:55 |
| noonedeadpunk | jrosser_: basically this should add indent https://opendev.org/openstack/ansible-config_template/src/branch/master/plugins/action/config_template.py#L72 | 10:58 |
| noonedeadpunk | ah here it is https://review.opendev.org/c/openstack/ansible-config_template/+/838141 | 10:59 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-config_template master: Use ensure SafeDumper used yaml configs https://review.opendev.org/c/openstack/ansible-config_template/+/838141 | 10:59 |
| noonedeadpunk | That's why I recall fixing this :) | 11:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-config_template master: Release config-template to 1.3.0 https://review.opendev.org/c/openstack/ansible-config_template/+/845699 | 11:00 |
| jrosser_ | huh thats unobvious | 11:00 |
| noonedeadpunk | oh yes, I spent quite some time to get to this... | 11:01 |
| jrosser_ | i wonder how to_nice_yaml works then | 11:11 |
| noonedeadpunk | I think there'd be a way if not parenting from AnsibleDumper. And eventually there's a ruamel.yaml module that support indent natively. But don't think it's used in pretty_yaml filter... | 11:15 |
| *** dviroel|afk is now known as dviroel | 11:28 | |
| jrosser_ | centos9s ["No package corosync available."], "item": "corosync", "msg": "Failed to install some of the specified packages" | 11:38 |
| jrosser_ | we need HighAvailability repo for that | 11:39 |
| noonedeadpunk | yeah I already looking on that | 11:42 |
| mgariepy | good morning | 11:46 |
| noonedeadpunk | hey o/ | 11:54 |
| *** tosky_ is now known as tosky | 11:55 | |
| jrosser_ | noonedeadpunk: https://github.com/jrosser/ansible-role-pacemaker_corosync/tree/patch-1 | 12:01 |
| jrosser_ | not sure if that repo has tests, as i just edited that in the web editor thing | 12:01 |
| noonedeadpunk | Right now I'm adding tests :) | 12:02 |
| noonedeadpunk | Orwell, fixing them | 12:02 |
| noonedeadpunk | As travis is not a thing anymore :( | 12:02 |
| jrosser_ | oh yeah and also https://blog.aquasec.com/travis-ci-security | 12:03 |
| mgariepy | what are you guys using for baremetal provisioning? | 12:13 |
| mgariepy | any recommendation ? | 12:13 |
| mgariepy | i had maas working at some point but only for dev and i have a lot of issue with ipmi stuff.. | 12:14 |
| noonedeadpunk | Well... We're still on maas but looking for Ironic implementation | 12:15 |
| mgariepy | i don't like being forced to use jq to have a nice output out of the cli lol | 12:15 |
| mgariepy | s/nice/readable/ | 12:16 |
| noonedeadpunk | But htere's https://github.com/maas/python-libmaas ? | 12:17 |
| noonedeadpunk | jrosser_: I pushed https://github.com/noonedeadpunk/ansible-role-pacemaker_corosync/pull/4 for adding CI but I bet it will fail now | 12:19 |
| mgariepy | thanks for python-libmaas i'll take a look. | 12:27 |
| admin1 | i was evaluating https://tinkerbell.org/ at some point .. still plan to evaluate it | 12:36 |
| noonedeadpunk | you can check bifrost which are set of ansible playbooks for deploying standalone ironic | 12:37 |
| noonedeadpunk | but well, dunno why we do same things with them separately... | 12:38 |
| noonedeadpunk | ok, corosync role should be passing now except centos 9 | 12:43 |
| noonedeadpunk | jrosser_: I have some feeling that HA is enabled in some different manner in centos 9 | 13:01 |
| noonedeadpunk | `Error: No matching repo to modify: ha.` | 13:02 |
| jrosser_ | it might be HighAvailability | 13:03 |
| noonedeadpunk | nope... | 13:03 |
| jrosser_ | oh | 13:04 |
| noonedeadpunk | highavailability | 13:05 |
| noonedeadpunk | ugh | 13:05 |
| noonedeadpunk | I believe that's the diff needed https://paste.openstack.org/show/bNUyjl4gyJwvtt4HRrTa/ | 13:12 |
| noonedeadpunk | It's a disaster how to work on same code in github... | 13:14 |
| noonedeadpunk | *on same PR | 13:14 |
| noonedeadpunk | I really dunno how to update existing PR | 13:15 |
| noonedeadpunk | seems I need github cli or smth... | 13:16 |
| *** tosky__ is now known as tosky | 13:17 | |
| jrosser_ | i'll try | 13:17 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-plugins master: Wait for all volumes when starting glusterd.service https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845987 | 13:21 |
| noonedeadpunk | ok, no, I can't not edit someones PR as for that I need to push to the forked repo which obviosly nobody has permission... | 13:25 |
| *** tosky is now known as Guest2169 | 13:29 | |
| *** tosky__ is now known as tosky | 13:29 | |
| noonedeadpunk | I feel physical pain when I have to interact with github after gerrit | 13:30 |
| jrosser_ | i think i fixed it | 13:30 |
| jrosser_ | https://github.com/noonedeadpunk/ansible-role-pacemaker_corosync/pull/5 | 13:30 |
| noonedeadpunk | yeah | 13:30 |
| noonedeadpunk | I think you did:) | 13:30 |
| jrosser_ | but omg the workflow hurts my head after all this gerrit | 13:30 |
| noonedeadpunk | it's totally not designed for collaborative development | 13:31 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-plugins master: Wait for all volumes when starting glusterd.service https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845987 | 13:32 |
| jrosser_ | i wonder if it has to be '9': as the key, not 9: | 13:33 |
| noonedeadpunk | I guess so... | 13:34 |
| noonedeadpunk | Sorry :) | 13:34 |
| jrosser_ | is that supposed to run the tests again | 13:37 |
| jrosser_ | this is pretty opaque | 13:37 |
| jrosser_ | oh it's doing it | 13:38 |
| noonedeadpunk | It requires me to approve running CI for $reason... | 13:38 |
| noonedeadpunk | I don't know | 13:38 |
| noonedeadpunk | Ok, I guess I fixed that as well... | 13:39 |
| jrosser_ | oh well i think there are $concerns if the PR tries to grab $things from your self-hosted guthub runner | 13:39 |
| jrosser_ | specifically this is problematic for opensource repos | 13:40 |
| noonedeadpunk | I guess it's more concerns that new users couldn't abuse github runners | 13:43 |
| noonedeadpunk | but yeah | 13:43 |
| noonedeadpunk | ok, merged | 13:44 |
| jrosser_ | did you see this https://review.opendev.org/c/openstack/devstack/+/845805 | 13:50 |
| spatel | very interesting | 13:55 |
| jrosser_ | if thats applicable for anything other than tests, i don't know | 13:55 |
| spatel | memory is cheap so not sure how much it will help | 13:56 |
| *** tosky__ is now known as tosky | 13:58 | |
| noonedeadpunk | well, it's really interesting. But I wouldn't do that probably outside of CI | 13:59 |
| opendevreview | Danny Nelhams proposed openstack/openstack-ansible-os_magnum master: Fixed dest typo in config_template https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/845993 | 14:06 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Bump SHA for pacemaker-corosync role https://review.opendev.org/c/openstack/openstack-ansible/+/845994 | 14:07 |
| *** ysandeep is now known as ysandeep|afk | 14:08 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_masakari master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_masakari/+/845798 | 14:09 |
| damiandabrowski[m] | hmm, we don't have many tests in here https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/845987 | 14:20 |
| damiandabrowski[m] | should I create some DNM change in openstack-ansible repo dependent on the above, just to test if things work fine? | 14:20 |
| opendevreview | Merged openstack/openstack-ansible master: Add log collection from /etc/ironic-inspector https://review.opendev.org/c/openstack/openstack-ansible/+/845891 | 14:22 |
| noonedeadpunk | damiandabrowski[m]: yeas, that would make sense:) | 14:23 |
| jrosser_ | looks like we are OOM for manila https://zuul.opendev.org/t/openstack/build/c654e1db5fc84b2797e5def30de6298a/log/logs/openstack/aio1-utility/tempest_run.log.txt#1169 | 14:24 |
| jrosser_ | noonedeadpunk: damiandabrowski[m] https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/843286 | 14:24 |
| jrosser_ | we should have a review party | 14:24 |
| jrosser_ | i guess the size for the manila test flavor needs to be larger now | 14:25 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible master: [DNM] test 845987 https://review.opendev.org/c/openstack/openstack-ansible/+/845997 | 14:26 |
| damiandabrowski[m] | noonedeadpunk: thanks! | 14:27 |
| noonedeadpunk | jrosser_: yeah... I picked it sooo hardly... | 14:27 |
| noonedeadpunk | previously | 14:27 |
| jrosser_ | ahha https://github.com/openstack/manila/commit/aeb5fa0195ac334ea2606d585872863ff03aebc8 | 14:30 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Increase manila flavor size https://review.opendev.org/c/openstack/openstack-ansible/+/845998 | 14:31 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila master: Delegate share type creation to setup host https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/845917 | 14:32 |
| noonedeadpunk | pf | 14:32 |
| noonedeadpunk | we don't have 512 | 14:32 |
| noonedeadpunk | We were failing ith 384 | 14:32 |
| jrosser_ | sounds like they switched the service VM to jammy? | 14:36 |
| jrosser_ | slightly unclear commit message there | 14:36 |
| noonedeadpunk | Yup I believe they did | 14:37 |
| jrosser_ | https://github.com/openstack/manila/commit/1674364fc335cb7c78dd36320dea7c13fb583530 | 14:37 |
| jrosser_ | yes looks like the whole thing is just fatter now | 14:37 |
| noonedeadpunk | linux version 5.15.0-37-generic | 14:37 |
| noonedeadpunk | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c65/845917/1/check/openstack-ansible-deploy-aio_metal-ubuntu-focal/c654e1d/logs/openstack/aio1-utility/stestr_results.html | 14:37 |
| noonedeadpunk | I wonder what would be bare minimum... | 14:41 |
| noonedeadpunk | damiandabrowski[m]: would be great if you could review https://review.opendev.org/q/topic:osa%252Fmanage_lb | 14:42 |
| noonedeadpunk | Wonder what's with zun upgrade jobs.... | 14:43 |
| damiandabrowski[m] | ack | 14:43 |
| noonedeadpunk | there's more to review though :p | 14:44 |
| noonedeadpunk | like https://review.opendev.org/q/topic:osa%252Fservice_tokens | 14:45 |
| jrosser_ | something fails with the neutron service tokens stuff https://d16e1fe661f5b3fa026a-f5cc3860b4c6d4d7b6c9f878adee54b5.ssl.cf5.rackcdn.com/845904/1/check/openstack-ansible-deploy-aio_metal-ubuntu-focal/9823ed8/logs/openstack/aio1-utility/stestr_results.html | 14:46 |
| noonedeadpunk | yeah, just spotted that | 14:46 |
| noonedeadpunk | maybe it should depend on nova... | 14:47 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/845904 | 14:48 |
| noonedeadpunk | as it's failed on request to nova there | 14:50 |
| noonedeadpunk | but that's weird, as request was issued from user generated by tempest... | 14:51 |
| *** ysandeep|afk is now known as ysandeep|out | 14:57 | |
| noonedeadpunk | I wonder if OCtavia upgrade fails because of certificates migration... | 14:57 |
| jrosser_ | thats the biggest change - though i would hope that the testing would catch that before we merge it | 14:58 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/c3a498da5471450080a35de1564eddd7/log/logs/host/octavia-worker.service.journal-12-15-36.log.txt#7302 | 14:58 |
| noonedeadpunk | looks like not... | 14:59 |
| noonedeadpunk | `No route to host` | 14:59 |
| jrosser_ | there should be an ip on br-lbaas for that https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/prepare_networking.yml#L144-L146 | 15:05 |
| noonedeadpunk | I can recall changing network or smth actually | 15:06 |
| noonedeadpunk | yup https://github.com/openstack/openstack-ansible/commit/0637c998d77f4dd878c26a73cfc10dd4fe3ca12b | 15:10 |
| noonedeadpunk | that could result in upgrade jobs failure actually... | 15:10 |
| noonedeadpunk | but we merged tons of stuff after that | 15:10 |
| spatel | what is the best tool to deploy ceph in production? | 15:11 |
| spatel | cephadm or ceph-deploy? | 15:11 |
| noonedeadpunk | picking between these I;d pick ceph-deploy or jsut manuall isntallation... | 15:12 |
| noonedeadpunk | But ceph-deploy is deprecated fro a while now | 15:12 |
| spatel | why would you pick ceph-deploy ? | 15:13 |
| spatel | cephadm also looks good not sure how good it would be on production | 15:13 |
| noonedeadpunk | It's likely not about ceph-deploy but more about cephadm | 15:14 |
| noonedeadpunk | I have some doubts how flexible it is, for example when you need to add some custom mgr plugin | 15:15 |
| noonedeadpunk | and whole mess with docket, including maintining registry | 15:15 |
| noonedeadpunk | feels a bit like overkill for installing 10 packages | 15:16 |
| noonedeadpunk | and syncing config across hosts | 15:16 |
| spatel | hmm so cephadm use docker? | 15:16 |
| spatel | not sure if ceph-ansible still functional? | 15:17 |
| noonedeadpunk | yup, or well, not docker, podman | 15:18 |
| *** dviroel is now known as dviroel|lunch | 15:18 | |
| spatel | what do you guys using to deploy ceph in production? | 15:19 |
| noonedeadpunk | you can probably pick though | 15:19 |
| spatel | i want to make sure upgrade is easy and simple :) | 15:20 |
| noonedeadpunk | was ceph upgrade a problem since Hammer? | 15:21 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_aodh master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/846010 | 15:22 |
| spatel | in past i was using OSA provided ceph-ansible but now i am thinking to deploy ceph outside OSA | 15:22 |
| noonedeadpunk | it was always matter of sed for repo and running apt upgrade kind of... | 15:22 |
| noonedeadpunk | just in correct order service wise | 15:23 |
| spatel | i think idea of cephadm because it use docker for apps and easy to upgrade etc.. | 15:23 |
| noonedeadpunk | deploying ceph outside osa is indeed smth you should do I believe | 15:23 |
| spatel | Yes that is what now i am doing | 15:23 |
| noonedeadpunk | well again - it was never hard kind of imo | 15:24 |
| noonedeadpunk | thinking about that - likely openstack upgrade is not hard as well :D | 15:24 |
| spatel | lol | 15:24 |
| jrosser_ | spatel: you should pick something that lets you run a very specific pinned version | 15:25 |
| jrosser_ | that you validate and test before rolling out newer | 15:25 |
| noonedeadpunk | and for that supposedly you need to maintain own repo and build/mirror packages there... | 15:25 |
| spatel | yes! | 15:25 |
| jrosser_ | ceph bonfires tend to be large / out of control | 15:25 |
| spatel | We are not using ceph in production but now i have new requirement to run on production and looking for better way to deploy ceph (easy way to operate) | 15:27 |
| spatel | I have two choice cephadm or ceph-ansible | 15:27 |
| spatel | lots of folk saying cephadm is very good compare to ceph-ansible (not sure why :) ) | 15:28 |
| noonedeadpunk | I think it's fast first of all | 15:28 |
| noonedeadpunk | And likely cephadm is future like we it or not | 15:29 |
| noonedeadpunk | (or maybe redhat will sunset it in 2 years and you will have to migrate somehow) | 15:29 |
| spatel | so cephadm is redhat baby? | 15:29 |
| noonedeadpunk | why it's podman by default?:D | 15:30 |
| noonedeadpunk | all ceph is kind of redhat driven now | 15:30 |
| spatel | Ouch! | 15:30 |
| mgariepy | ceph-stream is comming. ;p | 15:31 |
| mgariepy | lol | 15:31 |
| noonedeadpunk | not funny mgariepy | 15:31 |
| noonedeadpunk | not funny.... | 15:31 |
| noonedeadpunk | （ﾉ´д｀） | 15:31 |
| mgariepy | LOL sorry | 15:31 |
| noonedeadpunk | mgariepy: btw would be helpful if you put smth like +1 here https://jira.mariadb.org/browse/MDEV-28842 :) | 15:38 |
| noonedeadpunk | or somehow vote on it | 15:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_barbican master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_barbican/+/846013 | 15:40 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_blazar master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_blazar/+/846015 | 15:43 |
| mgariepy | let me login to check what i can do on this one. | 15:48 |
| mgariepy | hoo voted for it ! | 15:49 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_cloudkitty master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/846019 | 15:53 |
| noonedeadpunk | as basically what monty said - nobody asked for these packages so we decided not to build... | 15:54 |
| noonedeadpunk | UNless ppl will come and ask | 15:54 |
| mgariepy | do you need one for 22.04? | 15:58 |
| noonedeadpunk | nah, they planned it for 10.6.9 :( | 15:58 |
| mgariepy | ok | 15:58 |
| spatel | lol ceph-stream | 16:00 |
| mgariepy | spatel, noonedeadpunk said it wasn't funny ;p | 16:01 |
| spatel | Redhat is capable to do that | 16:01 |
| mgariepy | well redhat or ibm ? | 16:01 |
| noonedeadpunk | That is why no fun here | 16:01 |
| noonedeadpunk | Too close to truth ;( | 16:01 |
| spatel | Next ceph question, i have 15 node for ceph storage. can i use MDS same nodes? | 16:02 |
| spatel | I don't want to give dedicated HW for MDS :( | 16:02 |
| noonedeadpunk | I bet they even were discussing that... To have Ceph Stream and Ceph as part of their platform... | 16:03 |
| noonedeadpunk | I think mds worth running on same hosts where MONs and MGRs are | 16:04 |
| noonedeadpunk | And consider these as controll planes for ceph - worth separating these from OSDs. At least inside containers. | 16:04 |
| spatel | I am also planning to run MON, MGR on same 15 box | 16:04 |
| noonedeadpunk | You need to run just on 3 of them | 16:04 |
| noonedeadpunk | You totally don't need 15 mons | 16:05 |
| spatel | customer doesn't have money to have extra HW :( | 16:05 |
| spatel | yes.. i am planning to use 3 node for MON but it will be part of 15 OSD nodes | 16:05 |
| spatel | in short shared service | 16:05 |
| noonedeadpunk | fine as long as contenerized | 16:05 |
| noonedeadpunk | or you can put them on popenstack controllers as well hehe | 16:06 |
| spatel | Why containerized ? | 16:06 |
| spatel | why not deploy on metal | 16:06 |
| noonedeadpunk | upgrades? | 16:06 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Use the integrated repo infra jobs for testing https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/843286 | 16:06 |
| spatel | hmm | 16:07 |
| noonedeadpunk | as if you update ceph-mon package, by dependency it will update ceph-common which will update everything | 16:07 |
| noonedeadpunk | and you need to update ceph-osd as last thing after mons/mgrs/mds are restarted | 16:07 |
| spatel | make sense.. but if i use ceph-ansible then does it going to use container? or i just use LXD container and deploy inside that | 16:07 |
| noonedeadpunk | You can use just lxc/lxd | 16:08 |
| spatel | perfect | 16:08 |
| noonedeadpunk | and yeah, ceph-ansible has own implementation of docker isolation | 16:08 |
| spatel | let me run some lab test using all tool to see which is best for me | 16:08 |
| spatel | thanks for the tips :) | 16:09 |
| mgariepy | you need to restart osd the latest. | 16:12 |
| mgariepy | ceph-ansible does that by default also. | 16:13 |
| mgariepy | it doesn't restart services on pkg upgrade | 16:13 |
| noonedeadpunk | yeah, that's true, services are not restarted when package is updated (unless as of today). But do you really want to run service when libs underneath it are gone kind of... | 16:14 |
| noonedeadpunk | and I had cases when between mon/osd update several days could pass | 16:14 |
| mgariepy | my clusters are not big enough for that :D | 16:15 |
| noonedeadpunk | so running in that state is quite risky... | 16:15 |
| mgariepy | yeah indeed. | 16:15 |
| *** dviroel|lunch is now known as dviroel | 16:15 | |
| noonedeadpunk | Not sure if it was about cluster size or org :D | 16:15 |
| mgariepy | ha lol | 16:15 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_designate master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_designate/+/846026 | 16:19 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_heat master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/846029 | 16:23 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_gnocchi master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_gnocchi/+/846030 | 16:26 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ironic master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/846033 | 16:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ironic master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/846033 | 16:47 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ironic master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/846033 | 16:47 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_masakari master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_masakari/+/846035 | 16:49 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_masakari master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_masakari/+/846035 | 16:50 |
| noonedeadpunk | hm. Neutron fully fails still... | 16:52 |
| noonedeadpunk | that's weird... | 16:52 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/846037 | 16:58 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/845904 | 16:58 |
| noonedeadpunk | ok, nice, seems that 350 mb is enough for jammy | 17:00 |
| noonedeadpunk | https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/845917 passed rally and uploading logs now | 17:01 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/845801 | 17:01 |
| noonedeadpunk | We also need to merge https://review.opendev.org/c/openstack/openstack-ansible/+/845894 damiandabrowski[m] if around would be great. As this also needs backporting | 17:03 |
| noonedeadpunk | and https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/845767/1 as otherwise glance not passing :) | 17:03 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/846040 | 17:11 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_mistral master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/846041 | 17:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_senlin master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_senlin/+/846043 | 17:18 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_sahara master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_sahara/+/846044 | 17:21 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/846046 | 17:33 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_trove master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/846049 | 17:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_tacker master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_tacker/+/846050 | 17:41 |
| spatel | any idea what is wrong here - https://paste.opendev.org/show/bQ7LvFPA1RNjzHbbuYJl/ | 17:42 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_tacker master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_tacker/+/846050 | 17:42 |
| noonedeadpunk | ah, it's where ceph-ansible identify monitor address based on what it has | 17:43 |
| noonedeadpunk | supper messy part and we had tons of troubles with it... | 17:43 |
| noonedeadpunk | and have you defined mon_group_name in inventory? | 17:44 |
| noonedeadpunk | or weel, among variables? | 17:44 |
| noonedeadpunk | as we have smth like that https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/all/ceph.yml#L19-L24 | 17:45 |
| spatel | mon_group_name no i didn't define that | 17:48 |
| noonedeadpunk | not sure if issue is in that or not... but I guess it should be set anyway | 17:48 |
| spatel | hmm what should i define in mon_group_name? | 17:49 |
| spatel | This is what my inventory look like - https://paste.opendev.org/show/b6Vskp7qZM2y2HfKF7bz/ | 17:49 |
| noonedeadpunk | mons ? | 17:51 |
| spatel | i have single server and want to deploy everything on single machine | 17:51 |
| spatel | This doc i am following - https://dev.to/ingoleajinkya/deploying-docker-based-ceph-cluster-using-ceph-ansible-3cc7 | 17:52 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/846051 | 17:52 |
| spatel | its old but in my case i am using latest-pacific with latest ceph-ansible code | 17:52 |
| noonedeadpunk | um. and previous tasks are all skipped, right? | 17:54 |
| noonedeadpunk | all `set_fact _monitor_addresses .....` | 17:55 |
| noonedeadpunk | ah, ok, yu're using defaults | 17:56 |
| noonedeadpunk | https://github.com/ceph/ceph-ansible/blob/bb849a55861e3900362ec46e68a02754b2c892ec/roles/ceph-defaults/defaults/main.yml#L41-L45 | 17:57 |
| noonedeadpunk | But it's indeed important to understand if all `set_fact _monitor_addresses .....` were skipped? | 17:57 |
| spatel | here is the full output of play - https://paste.opendev.org/show/bbr3KjkUmXgNTqUylrSn/ | 17:58 |
| spatel | look like skip | 17:58 |
| noonedeadpunk | huh, no. | 17:58 |
| noonedeadpunk | set_fact _monitor_addresses to monitor_interface - ipv4 is pass | 17:59 |
| spatel | but why skip? | 17:59 |
| noonedeadpunk | I'd assume trying to debug and print var _monitor_addresses here https://github.com/ceph/ceph-ansible/blob/bb849a55861e3900362ec46e68a02754b2c892ec/roles/ceph-facts/tasks/set_monitor_address.yml#L52 | 18:01 |
| noonedeadpunk | as it can be, that your bond0.64 interface don't have ip assigned on it or smth like that... | 18:01 |
| spatel | it does has ip address on bond0.64 | 18:02 |
| spatel | bond0.64: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 | 18:02 |
| spatel | inet 10.64.2.0 netmask 255.255.248.0 broadcast 10.64.7.255 | 18:02 |
| noonedeadpunk | and it was there at once? As maybe stale facts or smth... | 18:03 |
| noonedeadpunk | Dunno | 18:03 |
| spatel | let me do print variable | 18:04 |
| spatel | just curious why its failing on line 52 which is ipv6 block - - name: set_fact _monitor_addresses to monitor_interface - ipv6 | 18:05 |
| spatel | noonedeadpunk this is the output i got for print - https://paste.opendev.org/show/bZ51qJR4fplCfSLQeV88/ | 18:08 |
| spatel | may be bond interface not supported?? | 18:09 |
| jrosser_ | did you print the literal string there rather than template the contents | 18:20 |
| jrosser_ | using ‘’ instead of “”? | 18:20 |
| spatel | i did this but same result - msg: '{{ _monitor_addresses }}' | 18:23 |
| spatel | also i have tried to set monitor_address: 10.64.2.0 in all.yml and still no luck | 18:24 |
| spatel | may be its bug or something else | 18:24 |
| noonedeadpunk | try cleaning host facts... | 18:25 |
| spatel | what is the ansible command to clean facts? | 18:26 |
| jrosser_ | print variables with var: | 18:26 |
| spatel | like this ? - https://paste.opendev.org/show/bor6qGHPo5eSxaDWAhfJ/ | 18:28 |
| jrosser_ | just the variable name | 18:28 |
| spatel | https://paste.opendev.org/show/bT2niaj4jJ37ivrXnTnk/ | 18:30 |
| spatel | same result :( | 18:30 |
| spatel | I tried this to clean any facts etc but same result - rm -rf ~/.ansible/* | 18:32 |
| opendevreview | Merged openstack/openstack-ansible-os_cinder master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/845794 | 18:34 |
| jrosser_ | it looks like it needs another round of passing through the templating engine | 18:37 |
| opendevreview | Merged openstack/openstack-ansible-os_nova master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/845793 | 18:39 |
| opendevreview | Merged openstack/ansible-role-uwsgi master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/ansible-role-uwsgi/+/845795 | 18:41 |
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/845787 | 18:43 |
| *** dviroel is now known as dviroel|afk | 18:47 | |
| opendevreview | Merged openstack/openstack-ansible-os_placement master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/845797 | 18:53 |
| opendevreview | Merged openstack/openstack-ansible-os_keystone master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/845741 | 18:58 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_cloudkitty master: Support service tokens https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/846019 | 19:17 |
| jrosser_ | really no idea about how we make an upgrade job work with these network changes https://github.com/openstack/openstack-ansible/commit/0637c998d77f4dd878c26a73cfc10dd4fe3ca12b | 19:23 |
| jrosser_ | as the container networking is all changed as well as neutron | 19:23 |
| noonedeadpunk | well, I think we just bootstrap this on whatever release and we don't re-run aio preparation after upgrade | 19:30 |
| noonedeadpunk | so openstack_user_config remains intact for upgrade jobs | 19:30 |
| noonedeadpunk | so yeah, it shouldn't affect in theory... | 19:30 |
| noonedeadpunk | and eventually upgrade jobs fail in many roles | 19:31 |
| noonedeadpunk | some fail though on being unable to reach mirrors via ipv6 | 19:32 |
| noonedeadpunk | like really most of things fail on upgrade specifically | 19:34 |
| noonedeadpunk | except neutron which is just whole red.... | 19:34 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: [DNM] Test CI https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/846063 | 19:38 |
| opendevreview | Merged openstack/openstack-ansible-os_mistral master: Remove mention of haproxy-endpoints role https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/845800 | 19:42 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!