| noonedeadpunk | I'm not sure there's a path from lxb to ovs though? | 07:40 |
|---|---|---|
| snadge | im running 21.2.6 because reasons, on a centos 7 deployment, and im seeing fibrechannel issues. This v7000 san is the bane of my existence. the gist of the error is unable to find a fibrechannel volume, in the os_brick.initiatior | 07:55 |
| snadge | it looks like its looking for a lun2 in /dev/disk/by-path/pcietc-lun-2 ... where nothing lun-2 exists.. something is bugging out in the fibre channel driver, and as best as I can tell its set up in the same way that openstack train was | 08:01 |
| snadge | i was getting different errors because systool wasn't installed, i've installed sysfsutils and get results from systool -c fc_host -v and systool fc_transport -v .. cinder is loaded and status seems ok | 08:13 |
| *** arxcruz is now known as arxcruz|training | 08:16 | |
| snadge | ive been looking at os_brick/initiator/linuxfc.py and os_brick/initiator/connectors/fibre_channel.py to try to make sense of the errors, but feel im a bit out of my depth.. any pointers of where I could look next, or tips on how to troubleshoot volume creation issues with fibrechannel/san would be much appreciated | 08:17 |
| *** sshnaidm|afk is now known as sshnaidm | 09:08 | |
| noonedeadpunk | I think that #openstack-cinder folks might have more ideas as they have more experience with different vendors and related to them issue. So worth asking there as well | 09:11 |
| ierdem | hi, I've started resize process yesterday and it is still on "resize" state. I don't want to lose that VM, is there any way to gracefully cancel resize operation and roll-back VM to old state? | 10:46 |
| noonedeadpunk | I guess `openstack server resize revert` doesn't really work? | 10:51 |
| noonedeadpunk | I think you might need to reset state to active and maybe shut down/power on. | 10:52 |
| ierdem | noonedeadpunk, which one I should try first? revert or reset state? | 11:00 |
| noonedeadpunk | I believe revert would work only when instance is in verify_resize state... | 11:09 |
| noonedeadpunk | but it wouldn't hurt trying | 11:10 |
| noonedeadpunk | reset state would just update database with the state. But it may bring inconsistency if some process is still running | 11:10 |
| noonedeadpunk | so that's why I said about power off/ power on - to ensure that nova can operate VM and then most likely it will cancell all pending stuff if any | 11:11 |
| *** kleini_ is now known as kleini | 12:24 | |
| ierdem | noonedeadpunk, I set state of VM to active and tried to restart/shutOff VM. Unfortunately Nova could not operate VM, and now when I try to send command to VM from horizon, it stucks on. Any suggestions? | 12:35 |
| noonedeadpunk | What is in logs? | 12:38 |
| noonedeadpunk | Are you sure that rest of the cloud operates normally? | 12:39 |
| ierdem | yes, cloud env is working normally. I can not reach VM log via horizon, on the compute node of VM there is just this https://paste.opendev.org/show/808510/ in nova-compute log about VM | 12:45 |
| ierdem | by the way, I can reach VM via ssh and I can reboot it, it works for now | 12:46 |
| ierdem | noonedeadpunk, I've checked the compute node. While resizing the instance, nova has tried to do live-migration and couldn't succeed. Normally, VM should run on compute20 but at the time of live migration nova has changed the compute node to compute5, and failed. Now, 'openstack server show' cmd is showing that instance on running compute20 | 13:09 |
| ierdem | So, i think problem occured when migrating instance. How can I solve this? | 13:10 |
| noonedeadpunk | anything for nova server-migration-list ? | 13:10 |
| noonedeadpunk | https://docs.openstack.org/nova/latest/admin/live-migration-usage.html#what-to-do-when-the-migration-times-out also might help | 13:11 |
| ierdem | noonedeadpunk, nova migration-list: https://paste.opendev.org/show/808512/, compute5 logs: https://paste.opendev.org/show/808511/ | 13:13 |
| noonedeadpunk | worth trying to abort it then | 13:14 |
| ierdem | ok, trying now | 13:14 |
| ierdem | noonedeadpunk, couldn't abort: https://paste.opendev.org/show/808513/, openstack server-show: https://paste.opendev.org/show/808514/ | 13:17 |
| ierdem | should I change state of VM to 'migrating' from 'active'? | 13:18 |
| noonedeadpunk | damn( | 13:19 |
| noonedeadpunk | you can change that only from DB now :( | 13:19 |
| ierdem | noonedeadpunk, which should I change, vm_state or task_state? | 13:32 |
| noonedeadpunk | iirc vm_state but that's not 100% | 13:33 |
| ierdem | ok, setting it to "migrating". I ll just try | 13:35 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible master: remove unnecessary revive and add support for metal deployments https://review.opendev.org/c/openstack/openstack-ansible/+/806933 | 13:39 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible master: Implement RabbitMQ cluster rolling restart feature https://review.opendev.org/c/openstack/openstack-ansible/+/804109 | 13:43 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-os_tempest master: Update cirros version to 0.5.2 https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/803487 | 13:52 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-ops master: A playbook to rolling restart controllers(paying special attention to galera and rabbitMQ) https://review.opendev.org/c/openstack/openstack-ansible-ops/+/806937 | 14:09 |
| ierdem | noonedeadpunk, I've changed the task_state to 'migrating', vm_state should be 'active'. Now, nova could not abort migration because the state of live-migartion is "pre-migrating". ERROR (BadRequest): Migration 692 state of instance 349d84c7-866b-44da-9315-0a48afd0cca7 is pre-migrating https://paste.opendev.org/show/808519/ | 14:11 |
| noonedeadpunk | Sorry, I have no idea of clean way for doing that. I think you can kind of adjust database and set migration.status to error or smth... | 14:16 |
| noonedeadpunk | But I don't have direct answer what exactly to do | 14:16 |
| ierdem | noonedeadpunk, thank you :) If I find a solution, I will write | 14:19 |
| noonedeadpunk | But what I'd do - continue messing up with DB to be able to cancel migration with cli | 14:22 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-ops master: A playbook to rolling restart controllers(paying special attention to galera and rabbitMQ) https://review.opendev.org/c/openstack/openstack-ansible-ops/+/806937 | 15:00 |
| spatel | does anyone running openstack in remote rental hardware? | 15:10 |
| spatel | I want to to run openstack in EU and ASIA region so talking to rental party to rent out some servers. | 15:11 |
| spatel | I have question, what if rental party say we can't give you bunch of VLAN like br-vxlan,br-vlan,br-mgmt etc.? | 15:12 |
| spatel | I heard they just give you 2x10G nic with public IP :( | 15:12 |
| spatel | i am trying to push hard for customized vlans | 15:13 |
| mgariepy | call ovh ;p | 15:23 |
| spatel | ovh? | 15:24 |
| mgariepy | spatel, https://www.ovh.com/ca/en/solutions/vrack/ | 15:27 |
| mgariepy | but they might no have DC everywhere you need :) | 15:28 |
| spatel | neat!!! | 15:29 |
| spatel | mgariepy do you have personal experience with them or you just know them | 15:30 |
| mgariepy | no i don't have professional experience with them | 15:31 |
| mgariepy | neither personal. | 15:31 |
| mgariepy | i know a few ppl that rented some servers over the years but not much more than this. | 15:32 |
| spatel | okay! looking good so far not sure about service and SLA | 15:32 |
| noonedeadpunk | I know about hetzner in eu but not all feedback was positive :) | 15:41 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-ops master: A playbook to rolling restart controllers(paying special attention to galera and rabbitMQ) https://review.opendev.org/c/openstack/openstack-ansible-ops/+/806937 | 15:42 |
| spatel | Its not easy to deploy openstack in remote DC :( | 15:45 |
| fungi | spatel: i use ovh's openstack clouds (both personally and also as a sysadmin of opendev, since they're donating a lot of quota for our nodepool), but haven't tried their dedicated server offering | 15:48 |
| fungi | as far as the company themselves, yes they're fairly pleasant to work with, and also great outspoken supporters of openstack | 15:49 |
| spatel | good to know! in worst case i may talk to them to see what they can do for me | 15:49 |
| kleini | OVH lost a complete data center in EU due to rain, fire or something like this. They did not have replicated their data to a second site and therefore lost all data. | 18:44 |
| spatel | kleini good to know! | 20:17 |
| mgariepy | it was a fire | 21:40 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!