| opendevreview | Sven Anders proposed openstack/openstack-ansible-os_keystone master: use keystone custom certificates from remote host https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/804202 | 06:21 |
|---|---|---|
| opendevreview | Sven Anders proposed openstack/openstack-ansible-os_keystone master: Handle host with unset ansible_host https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/804206 | 06:42 |
| opendevreview | Sven Anders proposed openstack/openstack-ansible-os_keystone master: Make Keystone_all group configurable https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/804207 | 06:44 |
| *** rpittau|afk is now known as rpittau | 07:23 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Add support of OVN for CentOS-8-Stream https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/803987 | 07:49 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Use list of cluster member for ovn ml2 agent to directly connect https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/803990 | 08:00 |
| opendevreview | Merged openstack/openstack-ansible stable/train: Remove Debian Stable testing https://review.opendev.org/c/openstack/openstack-ansible/+/803405 | 09:49 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron master: Replace centos-8.x jobs with centos-8-stream https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/804036 | 10:09 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron master: Set ovn hostname using nodename facts https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/803988 | 10:09 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron master: Fix OVN metadata protocol to point https https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/803989 | 10:09 |
| *** rpittau is now known as rpittau|afk | 11:28 | |
| admin1 | anyone here using ceph with 2 different pools ( hdd/ssd) with 2 different pools .. i have 2 types of cinder backends . but no matter what i select, it only goes to one .. here is my cinder.conf on how its set .. https://pastebin.com/imQ8LawX | 11:33 |
| agemuend | Hi all, we're new to openstack-ansible but have managed RDO based clusters manually before. On a new deployment we're seeing issues with image upload. On the dashboard we just receive a generic error message, through the CLI we get: "('Connection aborted.', BrokenPipeError(32, 'Broken pipe')". There are no glance log files in the container and the journal is empty as well. How do I normally debug such things with openstack-ansible? | 11:37 |
| admin1 | i think you need to enable a special heat variable in order to enable image upload via dashboard | 11:39 |
| admin1 | via cli it will work well | 11:39 |
| agemuend | Okay, but first more generally, what is the log strategy in osa with lxc. Should it be in the hosts journal, the containers journal, in log files, or where? | 11:40 |
| agemuend | I'm a bit confused by that | 11:41 |
| agemuend | Regarding CLI, the debug log shows that it communicates with glance successfully before, but when it comes to the PUT request `/v2/images/{ID}/file` it fails with BrokenPipeError | 11:44 |
| agemuend | Unfortunately I cant find any server logs as mentioned | 11:45 |
| admin1 | the logs are in the journal and not log file | 11:48 |
| admin1 | journalctl -u "username" would give you access to the logs | 11:48 |
| admin1 | neutron i think still has logs in the log files .. for the rest, its journal | 11:48 |
| admin1 | agemuend, what is the backend ? | 11:48 |
| admin1 | ceph ? | 11:49 |
| admin1 | you can set the debug = true to get more output | 11:49 |
| admin1 | are you seeing the error when you do openstack --debug image create ... from the util container ? | 11:50 |
| agemuend | Yes, its ceph indeed | 11:51 |
| agemuend | So its in the journal inside the lxc container, right? | 11:51 |
| admin1 | yes | 11:51 |
| agemuend | Is there some kind of best practice or even included role to configure a logserver? | 11:53 |
| agemuend | Due to the 3 management nodes its quiet a hassle to attach to all the containers and log into the journal there | 11:53 |
| agemuend | *look | 11:54 |
| agemuend | But thank you | 11:54 |
| agemuend | Ah indeed, I see a rbd.PermissionError: [errno 1] RBD permission error (error creating image) now | 11:54 |
| admin1 | if you are getting an error like this , its easy .. you can shut down or put in MAINT 2 nodes via hapropxy and just check the logs in the active one | 11:54 |
| admin1 | there used ot be a good logging rule, but i am also looking for one good role myself | 11:55 |
| admin1 | i think there is a way you can say you want the old style logs back, which you can send to something like graylog | 11:55 |
| admin1 | its some variable, which does the logs in file .. but i don't recall that exact variable name | 11:55 |
| agemuend | Okay, I'll check that, we could just send them to our existing graylog server then which would be easier for us | 11:57 |
| agemuend | Thanks again | 11:57 |
| admin1 | this issue solved ? | 11:57 |
| agemuend | I think so, at least I see in ceph auth ls that the glance user has only rx instead of rwx on the glance pool. probably a typo | 11:59 |
| agemuend | Yes, that worked indeed, thanks | 12:04 |
| jrosser | agemuend: the container journals are all bind mounted onto the hosts | 12:06 |
| jrosser | you do not need to collect journals from within the containers | 12:06 |
| jrosser | so you can pick whichever tool you like (I use elasic journalbeat) to collect from the host if you set the paths up correctly, and the tool understands collecting from multiple journals | 12:06 |
| jrosser | OSA is not opinionated about how log collection is done, we find everyone has their own preference so baking in a particular log collection stack is not helpful | 12:07 |
| jrosser | agemuend: if you want to use graylog there is some stuff here https://github.com/openstack/openstack-ansible-ops/tree/master/graylog - deployment of graylog which you wont want, forwarding to graylog which may be helpful | 12:09 |
| jrosser | the openstack-ansible-ops repo is contributed code and not part of the formal OSA release, so 'it-is-what-it-is' but hopefully useful | 12:10 |
| agemuend | ah nice | 12:13 |
| agemuend | thanks for the pointer | 12:13 |
| kleini | how can I read the journal log files on the host written for containes in /openstack/log/containername? | 12:14 |
| admin1 | those are text files | 12:20 |
| admin1 | you can just cat/more/less | 12:20 |
| tabacha | Hi I am Sven Anders I am working at ScaleupTech GmbH in Hamburg Germany and we want to use Ansible to Deploy our new OpenStack Cluster. We have already two old Cluster which were set up manualy. | 12:21 |
| tabacha | We do not want to set up out Cluster with the guide from OpenStack-Ansible but "cherry-pick" some Ansible roles which are suiteable for us. As the roles are small/distinct this looks very good. | 12:22 |
| admin1 | tabacha, in the long run, you will be playing catch-up with maintenance and keeping up the cherry picking | 12:23 |
| jrosser | admin1: the journals are binary not next | 12:24 |
| admin1 | that i know .. i was only referrring to the saved log files | 12:24 |
| jrosser | oh sorry | 12:24 |
| tabacha | Last week we set up Keystone with the os_keystone role, this looks good, but we found some small thinks that we would like to change. I am new to OpenStack but I use Ansible for "a long" :-) time. | 12:25 |
| kleini | https://paste.opendev.org/show/808009/ <- those are journals and not text. how can I read those journals as they contain the logs of the OpenStack services | 12:25 |
| admin1 | try maybe journalctl --file $path .. may work | 12:25 |
| admin1 | tabacha, for those small things , you can always override | 12:26 |
| jrosser | tabacha: it is sad you do not want to use the openstack-ansible framework to use those roles - is there a good reason for this? | 12:26 |
| agemuend | For me those journal folders are actually empty | 12:27 |
| admin1 | well upto you .. in our org we use openstack ansible as-is .. no changes .. this means we don't have to worry about update testing, upgrade testing, documentation, etc .. as we even follow the IP ranges as-is .. that way, a very small team is able to manage a lot of platforms | 12:27 |
| admin1 | and because we don't do any changes, we don't have to worry about if the new version will break our changes .. | 12:27 |
| agemuend | While talking about this, whats the easiest way to insert a configuration file change in lets say, cinder.conf? | 12:28 |
| jrosser | agemuend: you should learn about our config_template ansible module | 12:29 |
| agemuend | And while testing, a couple of times changes didnt arrive in the containers, we had to specifically delete them | 12:29 |
| agemuend | Could you give me a pointer to some docs? | 12:29 |
| jrosser | here is the documentation for config_template https://docs.openstack.org/ansible-config_template/latest/ | 12:30 |
| jrosser | for example here is how it is used for cinder config https://github.com/openstack/openstack-ansible-os_cinder/blob/master/tasks/cinder_post_install.yml#L25-L40 | 12:30 |
| agemuend | That documentation is not very extensive ;) | 12:31 |
| jrosser | hold on | 12:31 |
| jrosser | well, to be fair that is the documentation for that ansible module | 12:31 |
| agemuend | So config_overrides is a dict of key values? | 12:31 |
| jrosser | what you want to know about is how it is used in the context of OSA | 12:31 |
| jrosser | which would be this https://docs.openstack.org/openstack-ansible/wallaby/reference/configuration/using-overrides.html | 12:32 |
| tabacha | @jrosser one reason is that we already have a Company wide Ansible Repo and want to integrate openstack into this. As I told you bevore we have 3 distinct clusters. So its problematic when there is for example one group called: keystone_all as we need keystone_all_cluster1, keystone_all_custer2 ... | 12:32 |
| jrosser | config_template is a derivative of the ansible template module with the ability to 'sideload' an extra set of key/values in a dict | 12:32 |
| agemuend | nice, the last link is exactly what I needed, thanks. I'll go test this for a while | 12:34 |
| tabacha | An other reason is that we want to understand who our cluster is setup in depth so we install one service after an other and we have already some good ansible roles which conflicts with os. | 12:34 |
| jrosser | i would encourage you to get involved with openstack-ansible as a whole rather than take the roles individually | 12:35 |
| jrosser | there is a *lot* of orchestration in our the playbooks which use the roles which make upgrades and general cluster management tractible | 12:35 |
| jrosser | and the 'wiring' that is done in the openstack-ansible group_vars is another key piece which glues all the roles together with sensible settings | 12:36 |
| tabacha | Okay I have created change 804202 , 804206 and 804207 in gerrit via git review jrosser you told me to run git review again after I changed the commit message. How do I do this? When I run git review i got: ! [remote rejected] HEAD -> refs/for/master%topic=keystone_all_group_configurable (no new changes) | 12:37 |
| tabacha | error: failed to push some refs to 'ssh://tabacha@review.opendev.org:29418/openstack/openstack-ansible-os_keystone.git' | 12:37 |
| jrosser | you would edit your local files to update the change | 12:37 |
| jrosser | git commit --amend to get them into the commit | 12:37 |
| tabacha | I have modified commit message via Web Interface | 12:37 |
| jrosser | then git review to push a new revision | 12:37 |
| jrosser | can you give me the link? | 12:38 |
| tabacha | https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/804207 | 12:39 |
| tabacha | I manged it by: git commit --amend --no-edit && git push origin HEAD:refs/for/master | 12:40 |
| jrosser | there is only the first patchset there | 12:40 |
| jrosser | oh, well you don't ever git push | 12:40 |
| tabacha | yes I will update the other later on. | 12:40 |
| jrosser | no, i mean i only see the first revision | 12:40 |
| tabacha | I have only changed the commit message | 12:41 |
| jrosser | ok, have you pressed 'publish edit' ? | 12:42 |
| opendevreview | Sven Anders proposed openstack/openstack-ansible-os_keystone master: Make Keystone_all group configurable https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/804207 | 12:42 |
| jrosser | the commit message is part of the patch so changing it will make a new revision | 12:42 |
| jrosser | ^ thats it :) | 12:42 |
| opendevreview | Sven Anders proposed openstack/openstack-ansible-os_keystone master: Handle host with unset ansible_host https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/804206 | 12:43 |
| opendevreview | Sven Anders proposed openstack/openstack-ansible-os_keystone master: use keystone custom certificates from remote host https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/804202 | 12:44 |
| tabacha | okay, thank you | 12:44 |
| jrosser | ok, so this is all fairly big changes, particularly if you need to do it across all the roles | 12:44 |
| jrosser | and i hope i have been clear about the SSL changes, that the code you want to change will be removed with the introduction of our new pki role | 12:46 |
| jrosser | as others have pointed out we are definately going to be making breaking changes in your context, which will not be breaking changes for someone deploying "normally" with openstack-ansible | 12:47 |
| tabacha | We will review and test any change bevore it goes to our repo. | 12:47 |
| tabacha | The alternative for us is not to push back some changes which could be also good for openstack-ansible in general. | 12:49 |
| jrosser | spatel: hey o/ | 12:58 |
| spatel | hey | 12:59 |
| jrosser | got a bunch of those os_neutron patches merged | 12:59 |
| spatel | sweet!!! | 12:59 |
| jrosser | looks like it's still failing in a strange way on calico here https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/803987/5 | 12:59 |
| spatel | let me take a look | 13:00 |
| jrosser | what is most odd is that https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/803990/5 is passing, and as far as i can see this doesnt touch anything to do with calico | 13:04 |
| spatel | jrosser look like not able to get IP - https://f0bccbeb237308849467-c4783fd37c32977d9e7087e921c209d3.ssl.cf1.rackcdn.com/803987/5/check/openstack-ansible-deploy-aio_metal_calico-ubuntu-bionic/03acf86/logs/host/utility/tempest_run.log.txt | 13:06 |
| spatel | very odd.. i can spin up my lab and give it a try.. but if we are running out of time then we can do NV | 13:07 |
| mgariepy | https://zuul.opendev.org/t/openstack/build/03acf862e4eb4d9582e54f491e381a71/log/logs/host/etcd.service.journal-09-06-34.log.txt#113 | 13:10 |
| mgariepy | that's kinda weird. but i don't know anything about etcd or calico . | 13:10 |
| mgariepy | beside that they exist. and do ""stuff"" | 13:11 |
| spatel | we need boots on ground to fix.. based on logs hard to know.. i am trying to spin up aio to see what is going on | 13:17 |
| opendevreview | Satish Patel proposed openstack/openstack-ansible-openstack_hosts master: Add nova dependency repo for distro install https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/803475 | 13:20 |
| agemuend | By the way guys, we also had this bug: https://bugs.launchpad.net/openstack-ansible/+bug/1922699 | 13:26 |
| agemuend | I just added fmt to the includepkgs list in roles/lxc_hosts/tasks/lxc_install_dnf.yml to fix it | 13:27 |
| jrosser | agemuend: are you able to make a patch for that? | 13:30 |
| jrosser | which is to here i guess https://opendev.org/openstack/openstack-ansible-lxc_hosts/src/branch/master/tasks/lxc_install_dnf.yml#L67 ? | 13:31 |
| jrosser | it's a shame we don't have any testing of centos/ceph to pick that up, needs people interested in centos deployments to step up and look after that | 13:33 |
| spatel | I can be centos tester but mostly i do test related all core service and not storage but i can setup lab | 13:40 |
| spatel | jrosser curious why do we have calico for only ubuntu-bionic ? | 13:41 |
| spatel | but not focal ? | 13:41 |
| jrosser | just tech debt really | 13:41 |
| jrosser | needs updating | 13:41 |
| jrosser | thats a really simple patch just to see how far it gets | 13:41 |
| spatel | can we set NV and merge that patch for now and later we can do followup because i can see that Job passed yesterday so look like some stupid reason its failing | 13:42 |
| jrosser | as it's calico, yes thats fine to make it NV | 13:42 |
| jrosser | we need to resolve the OVN stuff as priority | 13:42 |
| spatel | i haven't heard anyone running calico networking :) ovn is most popular and defacto now | 13:42 |
| jrosser | lots still missing from OVN | 13:43 |
| jrosser | ipv6 is a big gap | 13:43 |
| jrosser | anyway | 13:43 |
| spatel | yes ipv6 is missing | 13:45 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Make calico non voting https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/804240 | 13:45 |
| jrosser | spatel: ^ there | 13:45 |
| spotz | Thanks spatel for volunteering! | 13:45 |
| agemuend | jrosser: yes thats the place | 13:46 |
| spatel | np :) happy to help wherever possible and actually i am helping myself :) | 13:46 |
| spatel | jrosser did you see my comment here? - https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/803630 | 13:48 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Switch calico job from bionic to focal https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/804242 | 13:49 |
| jrosser | spatel: you could put the creation of that dir somewhere like this https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/templates/prep-scripts/centos_8_prep.sh.j2#L27 | 13:50 |
| spatel | in that patch /etc/yum/vars is symlink that is why i have included both yum and dnf dir - https://paste.opendev.org/show/808012/ | 13:50 |
| jrosser | i wonder why they are not there in the container image | 13:51 |
| spatel | there is nothing wrong to include whole directory.. if has just few files inside, who knows somewhere they are dependent | 13:52 |
| spatel | if you want i can do mkdir -p /etc/yum/vars and just include that and exclude dnf (in future redhat may remove yum then we will stuck again) | 13:53 |
| spatel | jrosser i manage to finish my upgrade from V -> W but mysql was big hit.. | 14:03 |
| spatel | finally i upgraded mysql to 10.6.4 | 14:03 |
| jrosser | ah yes you missed my comment yesterday | 14:03 |
| jrosser | naaaah | 14:03 |
| jrosser | look at this https://github.com/openstack/openstack-ansible-galera_server/blob/master/handlers/galera_upgrade_hook.yml | 14:04 |
| jrosser | your first attempt at upgrade failed becasue of too many connections | 14:04 |
| jrosser | so it will have never run the handers at the end of the playbook | 14:04 |
| jrosser | the handlers fix up the mess left by 10.5.9 bugs | 14:04 |
| spatel | hmm | 14:05 |
| spatel | then i un-install mysql and re-install but still had that permission issue.. | 14:05 |
| spatel | finally this is how i solved that - https://paste.opendev.org/show/808013/ | 14:06 |
| spatel | playbook was failing to add admin account and i did by hand then | 14:06 |
| spatel | I am going to do this exercise again to see if i can re-produce | 14:07 |
| spatel | jrosser this patch also ready for merge - https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/803475 | 15:05 |
| spatel | do you still think we should fix this patch before merge or we are good? - https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/803630 | 15:06 |
| admin1 | checking here I have 2 ceph pools .. and 2 cinder backends . but when i create volumes, it only goes to 1 backend .. Here is my ceph.conf config and the ceph pools .. https://pastebin.com/imQ8LawX | 15:09 |
| spatel | that is cinder job to decide not ceph right? | 15:11 |
| admin1 | yes | 15:19 |
| admin1 | this is what is in my variables https://pastebin.com/LbqTtmX1 | 15:19 |
| admin1 | based on that when i create a fast-io volume, it should go to the svolumes pool . but not the case | 15:20 |
| jrosser | you've made the volume types with the cli? | 15:20 |
| admin1 | no | 15:20 |
| admin1 | osa did it for me | 15:20 |
| admin1 | i only add images, flavors and network from cli | 15:20 |
| jrosser | what does `openstack volume type list` show? | 15:22 |
| admin1 | shows this -> https://pastebin.com/hgEMYGDd | 15:25 |
| admin1 | volume type show shows this -> https://pastebin.com/h7dJgceJ | 15:26 |
| admin1 | do i need to put a diff volume_backend_name ? | 15:27 |
| admin1 | i only have rbd_pool: svolumes vs rbd_pool: hvolumes | 15:27 |
| admin1 | while i put the same volume_backend_name | 15:27 |
| jrosser | volume_backend_name is the thing it needs to know | 15:27 |
| admin1 | oh .. | 15:27 |
| admin1 | ok | 15:27 |
| admin1 | i will make those to match the rbd_pool name | 15:28 |
| admin1 | one more .. regarding overriding varibles by using group_vars .. config here: https://pastebin.com/3tGGS8Zp | 15:28 |
| admin1 | for only 1 server, i want to use a diff allocation ratio .. | 15:28 |
| admin1 | im is a shortcut i made for $path/inventory-manage -l | 15:34 |
| spatel | curious what disk cache mode you guys set for good performance? | 15:34 |
| jrosser | look at ansible variable precedence | 15:34 |
| jrosser | anything in user_*.yml overiddes all of host/group vars | 15:34 |
| *** sshnaidm is now known as sshnaidm|afk | 15:35 | |
| admin1 | oh .. then i did and expected the opposite :) | 15:35 |
| admin1 | what is the recommended way to override variables for just 1 host | 15:36 |
| jrosser | user_*.yml is passed to ansible-playbook using -e | 15:36 |
| jrosser | this is the highest precedence | 15:36 |
| jrosser | you need to move your nova_nova_conf_overrides out of user_variables.yml completely | 15:37 |
| admin1 | so create nova_compute.yml in group_vars and put the config there, and also put the host.yaml also there ? | 15:39 |
| admin1 | for just 1 line, i used to use host_vars in the user_config directly | 15:41 |
| jrosser | group_vars/<group_name>.yml and host_vars/<host_name>.yml | 15:44 |
| jrosser | you can test it with ansible debug module | 15:44 |
| jrosser | or a trivial playbook to print the value | 15:45 |
| jrosser | spatel: \o/ calico passes on focal https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/804242 | 15:56 |
| spatel | :) | 15:56 |
| spatel | nice! | 15:56 |
| spatel | look like it was bionic issue then | 15:57 |
| jrosser | perhaps, it's strange that it does pass sometimes | 15:58 |
| spatel | i think something handling it back to handover ip and sometime it get timeout | 16:00 |
| spatel | new OS is way to go for all new technologies | 16:02 |
| spatel | jrosser do you have any idea here, on Host machine i am getting write 32k IOPS but on VM i am getting 3k IOPS | 16:15 |
| spatel | i think issue is somewhere cache mode, my vm using cache mode = none | 16:16 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Allow fmt package to be installed from epel repo https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/804273 | 16:16 |
| jrosser | spatel: for what, ceph? | 16:18 |
| spatel | no ceph | 16:18 |
| spatel | all local disk | 16:18 |
| spatel | I have SSD on compute nodes | 16:18 |
| jrosser | you use virtio-scsi? | 16:18 |
| jrosser | oh right so nova epemeral disks then | 16:19 |
| jrosser | not cinder | 16:19 |
| spatel | Yes virtio-scsi option is set in my image metadata | 16:20 |
| spatel | https://documentation.suse.com/sles/11-SP4/html/SLES-kvm4zseries/cha-qemu-cachemodes.html | 16:20 |
| spatel | may be this is what i can try and see if i helps | 16:21 |
| jrosser | you use lvm? | 16:23 |
| spatel | no LVM | 16:23 |
| spatel | oh wait i do have LVM | 16:23 |
| spatel | but if i get 30k IOPS on compute machine directly but getting 3k IOPS on VM running on it that means something else going on | 16:24 |
| spatel | i am running fio benchmark tool | 16:24 |
| jrosser | well you're going through the virtualisation layer which is going to have some kind of overhead | 16:25 |
| spatel | yes but this is very big difference | 16:26 |
| spatel | 30k vs 3k :) | 16:26 |
| spatel | i am going to try out this and see - Cache=writebackI/O from the guest is cached on the host. | 16:26 |
| jrosser | you could see if there are any iops limits with virsh | 16:29 |
| spatel | sure! let me get bottom of this issue.. | 16:30 |
| spatel | i set disk_cachemodes = writeback in nova.conf and re-build vm but it doesn't do anything | 16:33 |
| spatel | i found this but its old bug - https://bugs.launchpad.net/nova/+bug/1727558 | 16:46 |
| spatel | jrosser epic win!!! | 17:06 |
| spatel | now i am getting same IOPS on vms | 17:07 |
| spatel | disk_cachemodes="file=writeback" | 17:07 |
| spatel | This is very important option i think :) | 17:07 |
| spatel | write: IOPS=31.3k, BW=122MiB/s (128MB/s)(7910MiB/64688msec); 0 zone resets | 17:07 |
| opendevreview | Merged openstack/openstack-ansible-os_cloudkitty stable/train: Updated from OpenStack Ansible Tests https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/690352 | 17:17 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Update galera to 10.9.12 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/804284 | 17:41 |
| spatel | jrosser your galera patch headline saying Update galera to 10.9.12 :) | 17:48 |
| spatel | it should be 10.5.12 | 17:48 |
| jrosser | oh! | 17:48 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Update galera to 10.5.12 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/804284 | 17:48 |
| jrosser | waaay to optimistic there with 10.9.x | 17:49 |
| spatel | it freaked me out, because yesterday i updated 10.6.4 which is most latest version :) | 17:50 |
| spatel | is this for wallaby right? | 17:51 |
| jrosser | nope, wallaby is released | 17:51 |
| jrosser | given the horror we had with both 10.5.8 and 10.5.9 i'd rather not touch W | 17:52 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Partial Revert "Bump MariaDB version to 10.5.9" https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/804289 | 17:54 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Partial Revert "Bump MariaDB version to 10.5.9" https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/804289 | 17:58 |
| spatel | jrosser soon i will do upgrade on production may be in 2 weeks, i am thinking to not upgrade my 10.5.6 version of mysql which is stable at present. (do you think that will cause any issue)? | 18:39 |
| jrosser | it's up to you really | 19:05 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Partial Revert "Bump MariaDB version to 10.5.9" https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/804289 | 19:08 |
| spatel | jrosser sure! i will try to not upgrade then, will do when next Xena release come out | 19:14 |
| jrosser | you know the deal with the upgrade jobs, they run on every patch so the 10.5.8 -> .9 path is well understood | 19:27 |
| jrosser | you were just unlucky in the lab with connection limit | 19:27 |
| jrosser | it would be bad to create FUD about galera upgrades | 19:27 |
| spatel | hmm | 19:39 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!