| *** jamesdenton has joined #openstack-operators | 00:30 | |
| *** pkoniszewski has quit IRC | 00:34 | |
| *** rebase has joined #openstack-operators | 00:36 | |
| *** rebase has quit IRC | 00:40 | |
| *** jamesdenton has quit IRC | 00:42 | |
| *** pkoniszewski has joined #openstack-operators | 00:47 | |
| *** jamesdenton has joined #openstack-operators | 00:47 | |
| *** pkoniszewski has quit IRC | 00:51 | |
| *** jamesden_ has joined #openstack-operators | 00:55 | |
| *** jamesdenton has quit IRC | 00:58 | |
| *** trungnv has quit IRC | 01:01 | |
| *** pkoniszewski has joined #openstack-operators | 01:02 | |
| *** chlong has quit IRC | 01:04 | |
| *** trungnv has joined #openstack-operators | 01:14 | |
| *** chlong has joined #openstack-operators | 01:19 | |
| *** VW has joined #openstack-operators | 01:22 | |
| *** VW_ has joined #openstack-operators | 01:27 | |
| *** VW has quit IRC | 01:30 | |
| *** slaweq has joined #openstack-operators | 01:31 | |
| *** slaweq has quit IRC | 01:35 | |
| *** ircuser-1 has quit IRC | 01:35 | |
| *** rebase has joined #openstack-operators | 01:39 | |
| *** jamesden_ has quit IRC | 01:48 | |
| *** rebase has quit IRC | 01:50 | |
| *** VW_ has quit IRC | 02:14 | |
| *** VW has joined #openstack-operators | 02:14 | |
| *** VW_ has joined #openstack-operators | 02:15 | |
| *** VW has quit IRC | 02:15 | |
| *** rebase has joined #openstack-operators | 02:24 | |
| *** rebase has quit IRC | 02:29 | |
| *** VW_ has quit IRC | 02:49 | |
| *** VW has joined #openstack-operators | 02:49 | |
| *** rebase has joined #openstack-operators | 02:51 | |
| *** pkoniszewski has quit IRC | 02:52 | |
| *** pkoniszewski has joined #openstack-operators | 02:52 | |
| *** VW has quit IRC | 02:54 | |
| *** rebase has quit IRC | 02:56 | |
| *** rebase has joined #openstack-operators | 03:05 | |
| *** RockyRaccoon has joined #openstack-operators | 03:12 | |
| *** rebase has quit IRC | 03:17 | |
| *** VW has joined #openstack-operators | 03:25 | |
| *** pkoniszewski has quit IRC | 03:27 | |
| *** pkoniszewski has joined #openstack-operators | 03:31 | |
| *** pkoniszewski has joined #openstack-operators | 03:32 | |
| *** VW has quit IRC | 03:35 | |
| *** slaweq has joined #openstack-operators | 03:46 | |
| *** slaweq has quit IRC | 03:51 | |
| *** gkadam has joined #openstack-operators | 04:00 | |
| *** gkadam has quit IRC | 04:01 | |
| *** gkadam has joined #openstack-operators | 04:03 | |
| *** Dinesh_Bhor has joined #openstack-operators | 04:10 | |
| *** Dinesh_Bhor has quit IRC | 04:24 | |
| *** Dinesh_Bhor has joined #openstack-operators | 04:25 | |
| *** catintheroof has joined #openstack-operators | 04:36 | |
| *** catintheroof has quit IRC | 04:42 | |
| *** Caterpillar has joined #openstack-operators | 06:10 | |
| *** Oku_OS-away is now known as Oku_OS | 06:17 | |
| *** rcernin has joined #openstack-operators | 06:19 | |
| *** arnewiebalck_ has joined #openstack-operators | 06:27 | |
| *** rcernin has quit IRC | 06:36 | |
| *** yprokule has joined #openstack-operators | 06:37 | |
| *** rcernin has joined #openstack-operators | 06:40 | |
| *** pcaruana has joined #openstack-operators | 06:52 | |
| *** simon-AS559 has joined #openstack-operators | 06:57 | |
| *** simon-AS5591 has joined #openstack-operators | 06:59 | |
| *** simon-AS559 has quit IRC | 07:02 | |
| *** simon-AS559 has joined #openstack-operators | 07:07 | |
| *** simon-AS559 has quit IRC | 07:07 | |
| *** simon-AS559 has joined #openstack-operators | 07:07 | |
| *** simon-AS5591 has quit IRC | 07:10 | |
| *** RockyRaccoon has quit IRC | 07:16 | |
| *** RockyRaccoon has joined #openstack-operators | 07:16 | |
| *** RockyRaccoon has quit IRC | 07:18 | |
| *** RockyRaccoon has joined #openstack-operators | 07:18 | |
| *** RockyRaccoon has quit IRC | 07:19 | |
| *** tesseract has joined #openstack-operators | 07:19 | |
| *** racedo has joined #openstack-operators | 07:40 | |
| *** slaweq has joined #openstack-operators | 07:47 | |
| *** fragatina has joined #openstack-operators | 07:47 | |
| *** slaweq has quit IRC | 07:51 | |
| *** rarcea has joined #openstack-operators | 08:19 | |
| *** yprokule_ has joined #openstack-operators | 08:23 | |
| *** trungnv has quit IRC | 08:25 | |
| *** trungnv has joined #openstack-operators | 08:25 | |
| *** yprokule has quit IRC | 08:25 | |
| *** yprokule_ is now known as yprokule | 08:25 | |
| *** dbecker has joined #openstack-operators | 08:28 | |
| *** derekh has joined #openstack-operators | 08:39 | |
| *** racedo has quit IRC | 08:39 | |
| *** racedo has joined #openstack-operators | 09:04 | |
| *** electrofelix has joined #openstack-operators | 09:14 | |
| *** rmart04 has joined #openstack-operators | 09:17 | |
| *** yprokule_ has joined #openstack-operators | 09:25 | |
| *** yprokule has quit IRC | 09:28 | |
| *** yprokule_ is now known as yprokule | 09:28 | |
| *** electrofelix has quit IRC | 10:55 | |
| *** electrofelix has joined #openstack-operators | 10:56 | |
| *** electrofelix has quit IRC | 10:57 | |
| *** liverpooler has quit IRC | 11:03 | |
| *** electrofelix has joined #openstack-operators | 11:04 | |
| *** aojea has joined #openstack-operators | 11:33 | |
| *** simon-AS559 has quit IRC | 12:04 | |
| *** simon-AS559 has joined #openstack-operators | 12:06 | |
| *** simon-AS5591 has joined #openstack-operators | 12:08 | |
| *** dminer has joined #openstack-operators | 12:10 | |
| *** simon-AS559 has quit IRC | 12:11 | |
| *** catintheroof has joined #openstack-operators | 12:27 | |
| *** catintheroof has quit IRC | 12:28 | |
| *** simon-AS5591 has quit IRC | 12:47 | |
| *** simon-AS559 has joined #openstack-operators | 12:56 | |
| *** mriedem has joined #openstack-operators | 12:59 | |
| *** simon-AS559 has quit IRC | 13:01 | |
| *** jamesdenton has joined #openstack-operators | 13:17 | |
| *** aojea has quit IRC | 13:22 | |
| *** liverpooler has joined #openstack-operators | 13:27 | |
| *** emccormick has joined #openstack-operators | 13:42 | |
| *** dminer has quit IRC | 13:45 | |
| *** fragatina has quit IRC | 13:47 | |
| *** fragatina has joined #openstack-operators | 13:47 | |
| *** catintheroof has joined #openstack-operators | 13:52 | |
| *** superdan is now known as dansmith | 13:56 | |
| *** makowals has quit IRC | 14:04 | |
| *** makowals has joined #openstack-operators | 14:07 | |
| *** rebase has joined #openstack-operators | 14:15 | |
| *** rebase has quit IRC | 14:19 | |
| *** jamesdenton has quit IRC | 14:23 | |
| *** jamesdenton has joined #openstack-operators | 14:26 | |
| *** jamesdenton has quit IRC | 14:32 | |
| *** jamesdenton has joined #openstack-operators | 14:33 | |
| *** gkadam has quit IRC | 14:35 | |
| *** makowals has quit IRC | 14:37 | |
| *** jamesdenton has quit IRC | 14:37 | |
| *** jamesdenton has joined #openstack-operators | 14:40 | |
| *** VW has joined #openstack-operators | 14:49 | |
| *** makowals has joined #openstack-operators | 14:50 | |
| *** makowals has quit IRC | 14:56 | |
| *** chyka has joined #openstack-operators | 15:02 | |
| *** makowals has joined #openstack-operators | 15:03 | |
| *** chyka has quit IRC | 15:04 | |
| *** makowals has quit IRC | 15:05 | |
| *** chyka has joined #openstack-operators | 15:07 | |
| *** Oku_OS is now known as Oku_OS-away | 15:09 | |
| *** makowals has joined #openstack-operators | 15:13 | |
| zioproto | yankcrime: that script was working. It was just a lack of OS_REGION_NAME https://github.com/uzh/ceph-tools/pull/1/files | 15:20 |
|---|---|---|
| zioproto | I extended it also to detect orphan snapshots | 15:20 |
| *** aojea has joined #openstack-operators | 15:21 | |
| *** rmart04 has quit IRC | 15:29 | |
| *** aojea has quit IRC | 15:30 | |
| *** pcaruana has quit IRC | 15:31 | |
| *** hieulq_ has joined #openstack-operators | 15:37 | |
| *** rcernin has quit IRC | 15:45 | |
| *** aojea has joined #openstack-operators | 15:54 | |
| *** Apoorva has joined #openstack-operators | 15:55 | |
| *** rebase has joined #openstack-operators | 15:56 | |
| *** gyee has joined #openstack-operators | 16:04 | |
| *** rebase has quit IRC | 16:06 | |
| *** rebase has joined #openstack-operators | 16:06 | |
| *** cemason has joined #openstack-operators | 16:07 | |
| *** cemason has quit IRC | 16:07 | |
| *** cemason has joined #openstack-operators | 16:07 | |
| *** yprokule has quit IRC | 16:10 | |
| *** slaweq has joined #openstack-operators | 16:15 | |
| *** slaweq has quit IRC | 16:19 | |
| *** rebase has quit IRC | 16:20 | |
| *** rebase has joined #openstack-operators | 16:20 | |
| *** shewless has joined #openstack-operators | 16:21 | |
| shewless | Hello. I'm in the process of migrating all instances off of a compute node so that I can do maintenance to it. I have a few instances that failed to migrate cleanly.. I'm trying to figure out why. When the migration fails it puts the instance in an error state.. and I have to edit the nova database because it seems like it "half" moves the instance | 16:23 |
| shewless | Now that I have the instance back on the original node the network doesn't work | 16:23 |
| shewless | To further complicate thing the firewall_driver on the original node is "hybrid" and the new node is "openvswitch" | 16:23 |
| shewless | the instance is currently on the original node (hybrid driver) but I can see from the virsh dumpxml that the openvswitch is being used instead of a tap with linux bridge | 16:24 |
| shewless | so: #1 how can I "rebuild" the instance xml so it's not using openvswitch firewall driver and #2 how can I successfully migrate this instance to another node | 16:25 |
| shewless | any guidance would be apprieted | 16:25 |
| *** wvera has joined #openstack-operators | 16:30 | |
| *** rebase has quit IRC | 16:37 | |
| *** makowals has quit IRC | 16:38 | |
| *** rebase has joined #openstack-operators | 16:44 | |
| *** makowals has joined #openstack-operators | 16:46 | |
| *** cemason has quit IRC | 16:49 | |
| *** rebase has quit IRC | 16:50 | |
| *** hieulq_ has quit IRC | 16:55 | |
| *** rebase has joined #openstack-operators | 16:56 | |
| *** FarmerJoe has joined #openstack-operators | 16:58 | |
| *** derekh has quit IRC | 16:59 | |
| *** CrazyFish has joined #openstack-operators | 17:05 | |
| *** dminer has joined #openstack-operators | 17:06 | |
| *** racedo has quit IRC | 17:11 | |
| *** racedo has joined #openstack-operators | 17:20 | |
| *** FarmerJoe has quit IRC | 17:22 | |
| *** racedo has quit IRC | 17:32 | |
| *** aojea has quit IRC | 17:33 | |
| *** slaweq has joined #openstack-operators | 17:36 | |
| *** racedo has joined #openstack-operators | 17:37 | |
| *** racedo has quit IRC | 17:38 | |
| *** slaweq has quit IRC | 17:41 | |
| *** rarcea has quit IRC | 17:44 | |
| *** racedo has joined #openstack-operators | 17:46 | |
| *** racedo has quit IRC | 17:49 | |
| *** fragatina has quit IRC | 17:54 | |
| *** tesseract has quit IRC | 17:54 | |
| *** racedo has joined #openstack-operators | 17:54 | |
| *** wvera has quit IRC | 17:55 | |
| *** racedo has quit IRC | 17:58 | |
| *** emagana has joined #openstack-operators | 17:59 | |
| *** VW_ has joined #openstack-operators | 18:00 | |
| *** VW_ has quit IRC | 18:01 | |
| *** VW_ has joined #openstack-operators | 18:02 | |
| *** VW has quit IRC | 18:02 | |
| *** electrofelix has quit IRC | 18:04 | |
| CrazyFish | Question for you guys: | 18:13 |
| CrazyFish | I'm trying to migrate some instances off a compute node. Some of the instances were created before their initial image was deleted from glance. I'm now getting an SCP error because the backing file in /var/lib/nova/instances/_base is absent | 18:14 |
| CrazyFish | {u'message': u"Unexpected error while running command.\nCommand: scp wtllab-compute-2:/var/lib/nova/instances/_base/aed03ac4fa2f2dde6873ade2f13842b318cc2e2a /var/lib/nova/instances/_base/aed03ac4fa2f2dde6873ade2f13842b318cc2e2a\nExit code: 1\nStdout: u''\nStderr: u'ssh: Cou", u'code': 500, u'details': u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 375, | 18:15 |
| CrazyFish | Has anyone seen this type of issue before? | 18:16 |
| CrazyFish | I get a similar problem when I try to boot or snapshot the instances: | 18:17 |
| CrazyFish | Command: qemu-img convert -f qcow2 -O qcow2 /var/lib/nova/instances/c70a67f1-ee40-4dc9-8a1d-e19f4e6167f0/disk /var/lib/nova/instances/snapshots/tmpZKRv8W/fd0e1260f0b54e4a83323240a7f0d27f Exit code: 1 Stdout: u'' Stderr: u"qemu-img: Could not open '/var/lib/nova/instances/c70a67f1-ee40-4dc9-8a1d-e19f4e6167f0/disk': Could not open backing file: Could not open '/var/lib/nova/instances/_base/aed03ac4fa2f2dde6873ade2f13842b3 | 18:19 |
| *** rebase has quit IRC | 18:20 | |
| *** rebase has joined #openstack-operators | 18:21 | |
| *** CrazyFish has quit IRC | 18:24 | |
| *** CrazyFish has joined #openstack-operators | 18:25 | |
| *** catinthe_ has joined #openstack-operators | 18:31 | |
| *** catintheroof has quit IRC | 18:33 | |
| shewless | what exactly is the _base file? is it a "cache" of an image that is used as a backing file for many instances that use the same "image"? | 18:54 |
| *** cemason has joined #openstack-operators | 19:05 | |
| *** aojea has joined #openstack-operators | 19:08 | |
| *** VW_ has quit IRC | 19:12 | |
| *** VW has joined #openstack-operators | 19:13 | |
| jlk | I explain some of that in https://derpops.bike/2016/10/07/openstack-nova-and-hypervisor-disk-consumption/ | 19:24 |
| *** slaweq has joined #openstack-operators | 19:30 | |
| *** aojea has quit IRC | 19:38 | |
| *** catintheroof has joined #openstack-operators | 19:39 | |
| *** aojea has joined #openstack-operators | 19:41 | |
| *** catinthe_ has quit IRC | 19:42 | |
| *** aojea has quit IRC | 19:43 | |
| shewless | jlk: thanks.. reading now | 19:46 |
| *** craigs has joined #openstack-operators | 19:46 | |
| shewless | jlk: is there any case where a _base image cache could be deleted even though there are instances that still refer to it? | 19:46 |
| craigs | anyone having issues with rabbitmq containers in osa newton 14.2.4? | 19:47 |
| *** VW has quit IRC | 20:01 | |
| *** liverpooler has quit IRC | 20:09 | |
| *** VW has joined #openstack-operators | 20:13 | |
| *** VW has quit IRC | 20:13 | |
| *** VW has joined #openstack-operators | 20:13 | |
| *** jamesdenton has quit IRC | 20:19 | |
| *** rebase has quit IRC | 20:20 | |
| *** rebase has joined #openstack-operators | 20:21 | |
| jlk | shewless: I honestly don't know. I haven't been working on openstack operations for 7 or 8 months, I've forgotten a lot of that stuff | 20:25 |
| *** emccormick has quit IRC | 20:30 | |
| shewless | I think this is what happened: | 20:32 |
| *** fragatina has joined #openstack-operators | 20:32 | |
| shewless | Issued migrate for some instances. The migrate failed for some reason (still trying to determine why). The instance was left "in limbo" for a period of 24 hours or more. THen the _base image was cleaned up because it was determined it was "unused" | 20:33 |
| shewless | Since the image no longer exists in glance I think we are screwed. | 20:33 |
| shewless | Either that or for some reason the _base file was gone before we started.. | 20:34 |
| jlk | rough scenario | 20:34 |
| *** jamesdenton has joined #openstack-operators | 20:35 | |
| *** arnewiebalck_ has quit IRC | 20:38 | |
| *** ircuser-1 has joined #openstack-operators | 20:39 | |
| *** emccormick has joined #openstack-operators | 20:47 | |
| *** aojea has joined #openstack-operators | 20:48 | |
| *** rockyg has joined #openstack-operators | 20:52 | |
| *** rockyg has quit IRC | 20:56 | |
| *** rockyg has joined #openstack-operators | 20:59 | |
| *** harlowja has quit IRC | 21:02 | |
| klindgren | shewless, if you can find the _base file on another compute node you can jsut copy it over | 21:30 |
| klindgren | or look at glance caches | 21:30 |
| *** rebase_ has joined #openstack-operators | 21:31 | |
| *** rebase has quit IRC | 21:32 | |
| *** VW_ has joined #openstack-operators | 21:32 | |
| *** VW has quit IRC | 21:36 | |
| *** jamesdenton has quit IRC | 21:37 | |
| *** VW_ has quit IRC | 21:37 | |
| *** khyr0n has joined #openstack-operators | 21:44 | |
| *** aojea has quit IRC | 21:45 | |
| *** slaweq has quit IRC | 21:47 | |
| *** slaweq has joined #openstack-operators | 21:47 | |
| *** aojea has joined #openstack-operators | 21:48 | |
| *** slaweq has quit IRC | 21:51 | |
| *** slaweq has joined #openstack-operators | 21:55 | |
| *** Caterpillar has quit IRC | 21:56 | |
| *** mriedem is now known as mriedem_away | 21:56 | |
| *** mriedem_away has quit IRC | 22:00 | |
| *** harlowja has joined #openstack-operators | 22:06 | |
| *** dminer has quit IRC | 22:15 | |
| *** fragatina has quit IRC | 22:17 | |
| *** rockyg has quit IRC | 22:18 | |
| *** rebase_ has quit IRC | 22:20 | |
| *** rebase_ has joined #openstack-operators | 22:22 | |
| *** bauruine has quit IRC | 22:27 | |
| *** emccormick has quit IRC | 22:27 | |
| *** bauruine has joined #openstack-operators | 22:29 | |
| *** slaweq has quit IRC | 22:38 | |
| *** slaweq has joined #openstack-operators | 22:38 | |
| *** slaweq has quit IRC | 22:40 | |
| *** slaweq has joined #openstack-operators | 22:41 | |
| *** slaweq has quit IRC | 22:43 | |
| *** mriedem has joined #openstack-operators | 22:43 | |
| *** catintheroof has quit IRC | 22:47 | |
| *** chyka has quit IRC | 22:58 | |
| *** emagana has quit IRC | 23:34 | |
| *** emagana has joined #openstack-operators | 23:35 | |
| *** cweeks has quit IRC | 23:36 | |
| *** emagana has quit IRC | 23:39 | |
| *** catintheroof has joined #openstack-operators | 23:45 | |
| *** aojea has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!