| *** goldyfruit has joined #openstack-karbor | 00:39 | |
| *** goldyfruit has quit IRC | 01:43 | |
| *** goldyfruit has joined #openstack-karbor | 02:41 | |
| *** goldyfruit has quit IRC | 04:00 | |
| *** sri_ has joined #openstack-karbor | 05:38 | |
| sri_ | jiaopengju, hi | 05:39 |
|---|---|---|
| sri_ | jiaopengju, when i try to restore from backup i am getting error's from cinder volume backup saying it's not able to find volume-id-snap | 06:14 |
| sri_ | jiaopengju, i am using ceph as cinder backup driver | 06:15 |
| sri_ | jiaopengju, is cinder backup driver should be karbor ? | 06:15 |
| openstackgerrit | inspurericzhang proposed openstack/karbor master: Replace git.openstack.org with opendev.org https://review.opendev.org/694934 | 08:16 |
| jiaopengju | cinder backup driver is independent with karbor | 09:52 |
| jiaopengju | karbor do not concern which driver that cinder backup use | 09:53 |
| jiaopengju | sri_, is there some errors in karbor protection log when you do restore? | 09:53 |
| sri_ | jiaopengju, there no error message in protection log | 09:54 |
| sri_ | jiaopengju, but volume goes error_restoring state | 09:55 |
| jiaopengju | this means the request has been sent to cinder, but cinder failed to do the restore | 09:55 |
| sri_ | jiaopengju, ack, I will look into it | 09:56 |
| sri_ | jiaopengju, how do i setup multi site setup for karbor ? | 09:56 |
| sri_ | jiaopengju, I've two regions so region-two karbor point to region-one database ? | 09:58 |
| jiaopengju | multi site means the checkpoint store in the same bank, not using the same db | 10:06 |
| sri_ | jiaopengju, so if both sites pointing to same bank we can do the restore ? | 10:07 |
| sri_ | Yesterday you asked me i am using same DB ? that's way i am confused | 10:08 |
| jiaopengju | sri_, if both sites pointing to same bank we can do the restore | 10:45 |
| jiaopengju | db only store the plan, and secheduled strategy, etc. checkpoints are stored in bank | 10:46 |
| sri_ | jiaopengju, understood | 10:46 |
| sri_ | jiaopengju, can you check the this process is this look ok ? http://paste.openstack.org/show/786332/ | 10:47 |
| sri_ | the complete restore process | 10:47 |
| jiaopengju | why I asked if you are using same DB, is because you said 'when we create a proction paln in one openstack-A it's automatically shows'sup in openstack-B !' | 10:47 |
| jiaopengju | are you using cinder snapshot plugin? | 10:49 |
| sri_ | jiaopengju, Ok, i mean both openstack-A and openstack-b point to same Bank ! i think we miss understood | 10:50 |
| sri_ | no i'm not using cinder plugin | 10:50 |
| jiaopengju | please show me your provider configuration file, I can not find it in the history message | 10:51 |
| sri_ | jiaopengju, http://paste.openstack.org/show/786334/ | 10:52 |
| jiaopengju | can you show me the result of 'cinder backup-list' after the checkpoint was created successfully | 10:55 |
| sri_ | jiaopengju, http://paste.openstack.org/show/786335/ | 10:56 |
| jiaopengju | can you restore the volume from cinder backup after you delete the server and the volume but not using karbor | 10:58 |
| sri_ | jiaopengju, ok let me do | 10:59 |
| sri_ | it's failing with same error Error Restoring | 11:00 |
| jiaopengju | you can see the log file of cinder backup, to find what happened | 11:01 |
| sri_ | but when create backup of volume without karbor i can restore successfully | 11:01 |
| sri_ | jiaopengju, cinder backup logs http://paste.openstack.org/show/786336/ | 11:02 |
| jiaopengju | I see | 11:03 |
| jiaopengju | This means you are using snapshot to do backup, but you delete the base volume, this will make the snapshot can not be use | 11:04 |
| sri_ | jiaopengju, how can i fix that ? | 11:05 |
| sri_ | jiaopengju, I've tried to restore without deleting volume but still same issue | 11:06 |
| jiaopengju | are you using the cinder version from community? | 11:06 |
| sri_ | jiaopengju, yes | 11:06 |
| jiaopengju | I've tried to restore without deleting volume but still same issue ----------- in this case, what happened? | 11:06 |
| jiaopengju | please show me the configuration file of cinder backup | 11:07 |
| sri_ | jiaopengju, cinder backup config file http://paste.openstack.org/show/786337/ | 11:08 |
| jiaopengju | sri_, https://github.com/openstack/cinder/blob/master/cinder/backup/drivers/ceph.py#L946 | 11:16 |
| jiaopengju | please see this: If the source volume is an RBD we will attempt to do an | 11:16 |
| jiaopengju | incremental/differential backup, otherwise a full copy is performed. | 11:16 |
| jiaopengju | If this fails we will attempt to fall back to full copy. | 11:16 |
| jiaopengju | the source volume and the backup backend are both ceph rbd, so it will using snapshot. In this case, if you delete the source volume, the backup can not be restored | 11:17 |
| jiaopengju | 'I've tried to restore without deleting volume but still same issue' ----------- in this case, what happened? | 11:18 |
| jiaopengju | I want to know what happened in this case | 11:18 |
| sri_ | jiaopengju, same error ImageNotFound: [errno 2] error opening image volume-xxxxxxxbackup.base at snapshot None | 11:18 |
| sri_ | jiaopengju, let me do the quick test now | 11:19 |
| sri_ | jiaopengju, this time i keep the volume but same issue http://paste.openstack.org/show/786339/ | 11:31 |
| sri_ | jiaopengju, this is only happening when karbor creating the backup | 11:32 |
| jiaopengju | can you check if it work without using karbor? only do backuo and restore in cinder | 11:32 |
| sri_ | jiaopengju, i did it works | 11:33 |
| jiaopengju | let me see | 11:35 |
| sri_ | jiaopengju, http://paste.openstack.org/show/786341/ | 11:38 |
| jiaopengju | can you do annotation from line 244 to 250 in this py file? then restart the karbor protection.https://github.com/openstack/karbor/blob/master/karbor/services/protection/protection_plugins/volume/cinder_protection_plugin.py#L244 | 11:39 |
| jiaopengju | I think i have find out the reason | 11:39 |
| jiaopengju | this seems to be a bug if using ceph driver | 11:39 |
| jiaopengju | for cinder backup. | 11:40 |
| jiaopengju | you can report it on storyboard, and we will try to fixed it | 11:40 |
| sri_ | jiaopengju, i am running karbor in container i need find out how to make change to python file | 11:41 |
| sri_ | jiaopengju, sure i will file bug in storyboard | 11:41 |
| sri_ | jiaopengju, is there any quick fix i can do ? " can you do annotation from line 244 to 250 in this py file? then restart the karbor protection.https://github.com/openstack/karbor/blob/master/karbor/services/protection/protection_plugins/volume/cinder_protection_plugin.py#L244" | 11:42 |
| sri_ | is that it ? | 11:42 |
| jiaopengju | maybe, karbor will first do snapshot for the volume and then do backup from the snapshot, after all finished, will delete the snapshot. But in ceph rbd backend of cinder backup, the snapshot maybe can not be deleted | 11:44 |
| jiaopengju | you can change using other cinder backup driver | 11:45 |
| sri_ | jiaopengju, sure let me try | 11:45 |
| jiaopengju | for example swift interface | 11:46 |
| jiaopengju | https://github.com/openstack/cinder/tree/master/cinder/backup/drivers | 11:46 |
| jiaopengju | here are all the supported drivers | 11:46 |
| sri_ | jiaopengju, I don't have swift running in my env but i will try setup swift and give it a try | 11:47 |
| jiaopengju | https://github.com/openstack/cinder/blob/master/cinder/backup/drivers/posix.py | 11:47 |
| jiaopengju | posix maybe easier if you only use it for testing | 11:48 |
| sri_ | jiaopengju, ok let me test that | 11:48 |
| jiaopengju | you can config the file path for cinder backup, many examples on the internet :) | 11:49 |
| sri_ | jiaopengju, ok i will give it try | 11:50 |
| jiaopengju | sri_, can you tell me where you are from and which company if it's convenient? :) | 11:51 |
| sri_ | I am currently in Nirobi kenya | 11:51 |
| sri_ | the company I work for Sybyl | 11:52 |
| jiaopengju | cool | 11:52 |
| sri_ | originally i am from India | 11:52 |
| jiaopengju | if you have some thoughts or new ideas about karbor, please let me know | 11:53 |
| jiaopengju | Maybe I can't reply IRC message in time. You can email me jiaopengju@cmss.chinamobile.com or jiaopengju@qq.com | 11:53 |
| sri_ | jiaopengju, sure I've just started looking into karbor i am happy to help | 11:54 |
| jiaopengju | welcome :) | 11:54 |
| sri_ | i really appreciate your work, i know you're fighting alone with Karbor war | 11:55 |
| jiaopengju | thanks :) | 11:56 |
| sri_ | jiaopengju, this this looks right ? i've changed cinder backup driver to posix http://paste.openstack.org/show/786342/ | 12:01 |
| jiaopengju | maybe not right, PosixBackupDriver should be contained in the driver name | 12:02 |
| sri_ | jiaopengju, yes you're right "backup_driver = cinder.backup.drivers.posix.PosixBackupDriver" | 12:03 |
| sri_ | jiaopengju, cinder-backup container is not happy :( | 12:06 |
| sri_ | jiaopengju, i think it's late for you, we will catch up tomorrow | 12:08 |
| *** goldyfruit has joined #openstack-karbor | 12:18 | |
| *** goldyfruit has quit IRC | 13:08 | |
| *** sri_ has quit IRC | 13:45 | |
| *** goldyfruit has joined #openstack-karbor | 14:19 | |
| *** goldyfruit_ has joined #openstack-karbor | 14:28 | |
| *** goldyfruit has quit IRC | 14:30 | |
| *** goldyfruit___ has joined #openstack-karbor | 19:55 | |
| *** goldyfruit_ has quit IRC | 19:57 | |
| *** openstackgerrit has quit IRC | 20:35 | |
| *** goldyfruit_ has joined #openstack-karbor | 20:58 | |
| *** goldyfruit___ has quit IRC | 21:00 | |
| *** goldyfruit_ has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!