| *** markvoelker has joined #openstack-operators | 00:06 | |
| *** Marga_ has quit IRC | 00:10 | |
| *** Marga_ has joined #openstack-operators | 00:10 | |
| *** vinsh has joined #openstack-operators | 00:12 | |
| *** derekh has quit IRC | 00:12 | |
| *** vinsh has quit IRC | 00:15 | |
| *** esker has joined #openstack-operators | 00:16 | |
| *** david-lyle has quit IRC | 00:28 | |
| *** mdorman has quit IRC | 00:28 | |
| *** klindgren has quit IRC | 00:32 | |
| *** esker has quit IRC | 00:34 | |
| *** blair has quit IRC | 00:34 | |
| *** SimonChung has quit IRC | 00:46 | |
| *** ferest has joined #openstack-operators | 00:57 | |
| *** esker has joined #openstack-operators | 01:01 | |
| *** signed8bit_ZZZzz is now known as signed8b_ | 01:12 | |
| *** j05h1 has joined #openstack-operators | 01:12 | |
| *** j05h1 has quit IRC | 01:15 | |
| *** j05h1 has joined #openstack-operators | 01:15 | |
| *** esker has quit IRC | 01:18 | |
| *** hitalia has joined #openstack-operators | 01:19 | |
| *** hitalia has quit IRC | 01:23 | |
| *** VW_ has joined #openstack-operators | 01:30 | |
| *** VW_ has quit IRC | 01:30 | |
| *** VW_ has joined #openstack-operators | 01:31 | |
| *** esker has joined #openstack-operators | 01:44 | |
| *** SimonChung has joined #openstack-operators | 01:44 | |
| *** blair has joined #openstack-operators | 01:51 | |
| *** j05h1 has quit IRC | 01:52 | |
| *** j05h1 has joined #openstack-operators | 01:52 | |
| *** ferest has quit IRC | 01:53 | |
| *** esker has quit IRC | 02:03 | |
| *** j05h1 has quit IRC | 02:07 | |
| *** esker has joined #openstack-operators | 02:29 | |
| *** j05h1 has joined #openstack-operators | 02:40 | |
| *** j05h1 has quit IRC | 02:42 | |
| *** j05h1 has joined #openstack-operators | 02:43 | |
| *** esker has quit IRC | 02:47 | |
| *** j05h1 has quit IRC | 02:54 | |
| *** j05h1 has joined #openstack-operators | 02:55 | |
| *** j05h1 has quit IRC | 02:59 | |
| *** signed8b_ is now known as signed8bit_ZZZzz | 03:01 | |
| *** blair has quit IRC | 03:04 | |
| *** esker has joined #openstack-operators | 03:13 | |
| *** signed8bit_ZZZzz is now known as signed8b_ | 03:16 | |
| *** signed8b_ is now known as signed8bit_ZZZzz | 03:17 | |
| *** esker has quit IRC | 03:24 | |
| *** esker has joined #openstack-operators | 03:24 | |
| *** j05h1 has joined #openstack-operators | 03:31 | |
| *** esker has quit IRC | 03:32 | |
| *** alop has quit IRC | 03:47 | |
| *** vinsh has joined #openstack-operators | 03:47 | |
| *** david-lyle has joined #openstack-operators | 03:48 | |
| *** vinsh has quit IRC | 03:51 | |
| *** j05h1 has quit IRC | 03:58 | |
| *** j05h1 has joined #openstack-operators | 03:59 | |
| *** blair has joined #openstack-operators | 04:09 | |
| *** j05h1 has quit IRC | 04:10 | |
| *** blair has quit IRC | 04:42 | |
| *** CongTo has joined #openstack-operators | 04:52 | |
| *** markvoelker has quit IRC | 04:58 | |
| *** markvoelker has joined #openstack-operators | 05:06 | |
| *** xavpaice_ has joined #openstack-operators | 05:15 | |
| *** fawadkhaliq has joined #openstack-operators | 05:24 | |
| *** xavpaice has quit IRC | 05:26 | |
| *** CongTo has quit IRC | 05:29 | |
| *** CongTo has joined #openstack-operators | 05:31 | |
| *** maishsk has joined #openstack-operators | 06:02 | |
| *** CongTo has quit IRC | 06:06 | |
| *** CongTo has joined #openstack-operators | 06:31 | |
| *** VW_ has quit IRC | 06:33 | |
| *** VW has joined #openstack-operators | 06:45 | |
| *** blair has joined #openstack-operators | 06:48 | |
| *** fifieldt has joined #openstack-operators | 06:56 | |
| *** fifieldt has quit IRC | 06:56 | |
| *** blair has quit IRC | 06:57 | |
| *** blair has joined #openstack-operators | 06:59 | |
| *** CongTo has quit IRC | 07:07 | |
| *** VW has quit IRC | 07:13 | |
| *** blair has quit IRC | 07:14 | |
| *** Marga_ has quit IRC | 07:31 | |
| *** derekh has joined #openstack-operators | 08:03 | |
| *** blair has joined #openstack-operators | 08:23 | |
| *** blair has quit IRC | 08:46 | |
| *** fawadkhaliq has quit IRC | 09:36 | |
| *** CongTo has joined #openstack-operators | 09:53 | |
| *** blair has joined #openstack-operators | 10:03 | |
| *** avozza has quit IRC | 10:25 | |
| *** ruagair has quit IRC | 10:26 | |
| *** ruagair has joined #openstack-operators | 10:30 | |
| *** zz_avozza has joined #openstack-operators | 10:32 | |
| *** zz_avozza is now known as avozza | 10:33 | |
| *** bell_juzo has joined #openstack-operators | 10:41 | |
| *** bell_juzo is now known as bell | 10:41 | |
| bell | Hi | 10:42 |
|---|---|---|
| bell | is anybody there ? | 10:42 |
| _nick | bell: just ask your question, if / when someone's around who can answer then they probably will | 11:03 |
| *** VW has joined #openstack-operators | 11:31 | |
| *** VW has quit IRC | 11:32 | |
| *** VW has joined #openstack-operators | 11:32 | |
| *** delattec has quit IRC | 11:50 | |
| *** cdelatte has quit IRC | 11:50 | |
| *** blair has quit IRC | 12:05 | |
| *** cdelatte has joined #openstack-operators | 12:07 | |
| *** derekh_ has joined #openstack-operators | 12:08 | |
| *** delattec has joined #openstack-operators | 12:09 | |
| *** derekh has quit IRC | 12:12 | |
| *** CongTo has quit IRC | 12:12 | |
| *** cdelatte has quit IRC | 12:12 | |
| *** signed8bit_ZZZzz has quit IRC | 12:14 | |
| *** maishsk has quit IRC | 12:31 | |
| *** matrohon has joined #openstack-operators | 12:41 | |
| *** Piet has quit IRC | 13:05 | |
| *** blair has joined #openstack-operators | 13:06 | |
| *** blair has quit IRC | 13:08 | |
| *** blair has joined #openstack-operators | 13:08 | |
| *** rlrevell has joined #openstack-operators | 13:11 | |
| *** blair has quit IRC | 13:13 | |
| *** radez_g0n3 is now known as radez | 13:19 | |
| *** j05h1 has joined #openstack-operators | 13:37 | |
| *** j05h1 has quit IRC | 13:41 | |
| *** j05h1 has joined #openstack-operators | 13:42 | |
| *** sgordon has joined #openstack-operators | 13:50 | |
| *** dboik has joined #openstack-operators | 13:52 | |
| *** dboik_ has joined #openstack-operators | 13:53 | |
| *** dboik has quit IRC | 13:57 | |
| *** markvoelker has quit IRC | 13:59 | |
| *** j05h1 has quit IRC | 14:05 | |
| *** blair has joined #openstack-operators | 14:09 | |
| *** dminer has joined #openstack-operators | 14:11 | |
| *** j05h1 has joined #openstack-operators | 14:13 | |
| *** blair has quit IRC | 14:14 | |
| *** jaypipes has joined #openstack-operators | 14:15 | |
| *** bell has quit IRC | 14:21 | |
| *** bvandenh has joined #openstack-operators | 14:26 | |
| *** vinsh has joined #openstack-operators | 14:30 | |
| *** Marga_ has joined #openstack-operators | 14:59 | |
| *** reed_ has joined #openstack-operators | 15:01 | |
| *** Marga_ has quit IRC | 15:03 | |
| *** Marga_ has joined #openstack-operators | 15:03 | |
| *** mnaser has joined #openstack-operators | 15:05 | |
| mnaser | anyone from ubuntu know if the packages are on their way soon for this? https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg02561.html | 15:06 |
| mnaser | https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+build/7424692 | 15:08 |
| *** blair has joined #openstack-operators | 15:10 | |
| *** blair has quit IRC | 15:15 | |
| *** matrohon has quit IRC | 15:16 | |
| *** alop has joined #openstack-operators | 15:37 | |
| *** bvandenh has quit IRC | 15:43 | |
| *** gyee has joined #openstack-operators | 15:44 | |
| *** shakamunyi has quit IRC | 15:51 | |
| *** barra204 has quit IRC | 15:51 | |
| *** esker has joined #openstack-operators | 15:53 | |
| *** SimonChung has quit IRC | 16:11 | |
| *** blair has joined #openstack-operators | 16:11 | |
| *** blair has quit IRC | 16:16 | |
| *** derekh_ has quit IRC | 16:37 | |
| *** Piet has joined #openstack-operators | 16:51 | |
| *** rlrevell has quit IRC | 16:52 | |
| *** rlrevell has joined #openstack-operators | 16:52 | |
| *** rlrevell1 has joined #openstack-operators | 16:53 | |
| *** rlrevell has quit IRC | 16:53 | |
| *** rlrevell has joined #openstack-operators | 16:53 | |
| *** signed8bit has joined #openstack-operators | 16:55 | |
| *** Marga_ has quit IRC | 16:57 | |
| *** rlrevell1 has quit IRC | 16:58 | |
| jlk | floppy drivers.... in 2015. | 16:59 |
| jlk | amiright?? | 16:59 |
| *** SimonChung has joined #openstack-operators | 17:03 | |
| *** blair has joined #openstack-operators | 17:12 | |
| *** blair has quit IRC | 17:17 | |
| *** harlowja has quit IRC | 17:20 | |
| *** harlowja has joined #openstack-operators | 17:20 | |
| *** maishsk has joined #openstack-operators | 17:23 | |
| *** fawadkhaliq has joined #openstack-operators | 17:31 | |
| *** SimonChung1 has joined #openstack-operators | 17:32 | |
| *** SimonChung has quit IRC | 17:32 | |
| *** SimonChung has joined #openstack-operators | 17:33 | |
| *** SimonChung1 has quit IRC | 17:33 | |
| *** reed_ has quit IRC | 17:41 | |
| *** belmoreira has joined #openstack-operators | 17:41 | |
| *** belmoreira has quit IRC | 17:43 | |
| *** SimonChung1 has joined #openstack-operators | 17:47 | |
| *** SimonChung has quit IRC | 17:47 | |
| mnaser | http://www.ubuntu.com/usn/usn-2608-1/ | 17:50 |
| mnaser | hurry up folks | 17:51 |
| *** fawadk has joined #openstack-operators | 17:58 | |
| rlrevell | mnaser: are there any reports of it being exploited? | 17:59 |
| mnaser | rlrevell: i dont believe there is a poc that's out, which makes it quite inaccessible out of the hands of script kiddies | 17:59 |
| mnaser | but the patch to fix it is out, so i wouldn't give it a long time | 17:59 |
| rlrevell | mnaser: looks like you also have to have malicious clients | 18:00 |
| rlrevell | customers i mean | 18:00 |
| mnaser | rlrevell: not necessarily, you don't always control the environment. | 18:00 |
| mnaser | operating a public cloud could also have people purposely sign up for the only reason of exploiting the environment too | 18:00 |
| *** fawadkhaliq has quit IRC | 18:01 | |
| jlk | yeah, pub clouds are the most at risk | 18:04 |
| *** fawadkhaliq has joined #openstack-operators | 18:04 | |
| jlk | random users, lots of juicy targets next door | 18:04 |
| *** fawadk has quit IRC | 18:06 | |
| mnaser | yep ^ | 18:09 |
| mnaser | but you can avoid the reboot | 18:09 |
| mnaser | by doing a suspend/resume | 18:09 |
| *** fawadk has joined #openstack-operators | 18:11 | |
| jlk | yeah, that's likely what we'll have our customers do | 18:13 |
| jlk | our customers are mostly single-tenant though, no shared clouds. | 18:13 |
| *** fawadkhaliq has quit IRC | 18:13 | |
| *** blair has joined #openstack-operators | 18:13 | |
| *** fawadk has quit IRC | 18:14 | |
| *** maishsk_ has joined #openstack-operators | 18:18 | |
| *** maishsk has quit IRC | 18:18 | |
| *** maishsk_ is now known as maishsk | 18:18 | |
| *** blair has quit IRC | 18:19 | |
| *** hitalia has joined #openstack-operators | 18:22 | |
| *** Marga_ has joined #openstack-operators | 18:28 | |
| *** Marga_ has quit IRC | 18:35 | |
| *** Marga_ has joined #openstack-operators | 18:35 | |
| rlrevell | mnaser: any idea to what extent the "attackers would be isolated by the libvirt AppArmor profile" thing mitigates the problem? | 18:43 |
| *** dminer has quit IRC | 18:44 | |
| mnaser | rlrevell: basically, it implies that the exploit gets you out to a land that is controlled by apparmor | 18:50 |
| mnaser | so you're at the mercy of apparmor to keep you save then | 18:51 |
| rlrevell | mnaser: and it looks like apparmor allows libvirt access to quite a bit | 18:51 |
| mnaser | yep.. i wouldnt count ion it | 18:51 |
| mnaser | after you upgrade kvm | 18:51 |
| mnaser | all you need to do is | 18:51 |
| mnaser | nova suspend <vm> | 18:51 |
| mnaser | nova resume <vm> | 18:51 |
| rlrevell | mnaser: i'm still at the lab stage so i can just reboot everything | 18:52 |
| *** yapeng has joined #openstack-operators | 18:54 | |
| mnaser | good enoguh | 18:55 |
| *** yapeng has quit IRC | 19:02 | |
| *** hitalia has quit IRC | 19:06 | |
| *** hitalia has joined #openstack-operators | 19:08 | |
| *** Rockyg_ has joined #openstack-operators | 19:12 | |
| *** blair has joined #openstack-operators | 19:15 | |
| *** blair has quit IRC | 19:20 | |
| *** turnerg has joined #openstack-operators | 19:21 | |
| *** hitalia has quit IRC | 19:27 | |
| *** belmoreira has joined #openstack-operators | 19:32 | |
| *** belmoreira has quit IRC | 19:33 | |
| *** Rockyg_ has quit IRC | 19:59 | |
| *** turnerg has quit IRC | 20:00 | |
| *** turnerg has joined #openstack-operators | 20:01 | |
| *** Rockyg has joined #openstack-operators | 20:01 | |
| *** turnerg has quit IRC | 20:01 | |
| *** turnerg has joined #openstack-operators | 20:03 | |
| *** delattec has quit IRC | 20:08 | |
| *** blair has joined #openstack-operators | 20:16 | |
| *** esker has quit IRC | 20:16 | |
| *** hitalia has joined #openstack-operators | 20:18 | |
| *** blair has quit IRC | 20:20 | |
| *** maishsk has quit IRC | 20:21 | |
| *** maishsk has joined #openstack-operators | 20:22 | |
| *** bvandenh has joined #openstack-operators | 20:22 | |
| *** turnerg has quit IRC | 20:28 | |
| xavpaice_ | suspend/resume takes quite a while though doesn't it? Minutes for us, at least | 20:32 |
| xavpaice_ | live migration seems better, but I need to understand it better before feeling confident | 20:32 |
| *** bvandenh has quit IRC | 20:33 | |
| *** turnerg has joined #openstack-operators | 20:35 | |
| *** vinsh_ has joined #openstack-operators | 20:43 | |
| *** vinsh has quit IRC | 20:46 | |
| *** belmoreira has joined #openstack-operators | 20:49 | |
| *** maishsk has quit IRC | 20:50 | |
| mnaser | xavpaice_: we use SSDs so it's quite fast | 20:51 |
| xavpaice_ | :) | 20:51 |
| mnaser | you have to factor in that the entire state of memory is written to disk, so the more memory on the server, the more data it has to write | 20:51 |
| mnaser | live migration can work out well, the only caveat is if you're using configdrive, live migration fails (even with block storage migration :\) | 20:52 |
| xavpaice_ | yeah, that's why it's so slow for those of us with slow disk on the hypervisors | 20:52 |
| *** radez is now known as radez_g0n3 | 20:52 | |
| xavpaice_ | I've had a few instances get kinda stuck during live migration, still trying to collect enough detail to figure out what's going on | 20:53 |
| mnaser | do these instances have a lot of memory? | 20:53 |
| mnaser | live migration actually copies the contents of memory then rescans it again for changes and sends those changes, and it keeps doing that until they're sync'd enough for it to pause the old instance, sync, and unpause | 20:54 |
| mnaser | if you have a lot of memory on the machine and your link speed is slow, the memory changes will be way faster than the speed of transfer, and you're stuck in this loop of constantly moving memory contents | 20:54 |
| mnaser | i hope that made sense | 20:54 |
| xavpaice_ | it does, and that's pretty much what I reckon is happening | 20:55 |
| mnaser | yeah there's not much you can do other than lowering the load/memory usage on that instance or upping the speed of the links | 20:55 |
| xavpaice_ | the network the live migration is running over is only 1Gbps, considering adding that vlan to the 10Gbps links we use for storage access | 20:56 |
| xavpaice_ | it appears also to be the instances we know are particularly busy | 20:56 |
| mnaser | yeah, that can explain the memory contents changing a lot at a qucik rate | 20:57 |
| mnaser | if it was fairly idle, memory would sit at the same | 20:57 |
| xavpaice_ | I've not yet attempted pause/migrate/resume - if that's even possible | 20:57 |
| mnaser | hm, not sure about that combination | 20:58 |
| *** vinsh has joined #openstack-operators | 20:59 | |
| *** vinsh_ has quit IRC | 21:03 | |
| *** ruagair_ has joined #openstack-operators | 21:05 | |
| jlk | it is | 21:05 |
| jlk | the non-live migrate does pretty much that | 21:06 |
| jlk | it's like resize | 21:06 |
| xavpaice_ | resize actually reboots the instance though | 21:07 |
| xavpaice_ | (as it needs to for most guest OS's) | 21:08 |
| jlk | oh true. yes. | 21:08 |
| xavpaice_ | nova migrate actually shows up in the logs like a resize | 21:08 |
| *** ruagair has quit IRC | 21:09 | |
| *** sgordon has quit IRC | 21:09 | |
| *** dmsimard has quit IRC | 21:09 | |
| mnaser | migrate uses the same codebase as resize actually | 21:09 |
| jlk | https://blueprints.launchpad.net/nova/+spec/migrate-non-active-instances | 21:10 |
| *** sgordon has joined #openstack-operators | 21:10 | |
| jlk | mnaser: migrate yes, live-migrate not as much | 21:10 |
| *** sgordon has quit IRC | 21:10 | |
| *** sgordon has joined #openstack-operators | 21:10 | |
| mnaser | yep | 21:10 |
| *** dmsimard has joined #openstack-operators | 21:11 | |
| *** Marga_ has quit IRC | 21:11 | |
| *** Marga_ has joined #openstack-operators | 21:12 | |
| xavpaice_ | unfortunately can't pause then migrate or live-migrate | 21:13 |
| xavpaice_ | so if there's an instance that's too busy to live-migrate, it's going to need a reboot | 21:13 |
| jlk | thankfully we don't support live migrate at all | 21:13 |
| *** hitalia has quit IRC | 21:14 | |
| xavpaice_ | do your customers not mind their VM's getting rebooted? | 21:14 |
| jlk | they might, but migration is "non-cloud" or so we tell them | 21:15 |
| jlk | more realistically we just don't have it wired all the way up yet, it's a future feature | 21:15 |
| xavpaice_ | wise choice IMO | 21:15 |
| xavpaice_ | so far, it's not been the most reliable thing for us | 21:16 |
| jlk | yup. | 21:16 |
| jlk | and as a maint tool, there is no guarantee that there will be enough space elsewhere to do the shuffle | 21:16 |
| harlowja | also fyi for u guys there is a #kvm channel in freenode that is also talking about this (currently/recently) | 21:16 |
| xavpaice_ | we try hard to assist customers to use apps/instances in a way that can easily withstand reboots one at a time | 21:16 |
| xavpaice_ | thanks! | 21:17 |
| mnaser | thanks harlowja | 21:17 |
| harlowja | np | 21:17 |
| *** blair has joined #openstack-operators | 21:17 | |
| * harlowja doesn't think it has a logger attached anywhere, but not sure | 21:17 | |
| *** jsnow has joined #openstack-operators | 21:21 | |
| *** `mjr has joined #openstack-operators | 21:21 | |
| *** blair has quit IRC | 21:21 | |
| harlowja | nope doesn't appear so, so if u intersted, thats where some of the qemu/kvm people are i think | 21:23 |
| harlowja | *in that channel | 21:23 |
| jsnow | my ears are ringin' | 21:24 |
| harlowja | :) | 21:24 |
| xavpaice_ | https://github.com/xavpaice/openstack-tools/blob/master/livemigrate_instances.py might come in handy btw, feedback/pr's most welcome | 21:25 |
| *** matrohon has joined #openstack-operators | 21:29 | |
| *** rlrevell has quit IRC | 21:31 | |
| *** turnerg has quit IRC | 21:36 | |
| *** hitalia has joined #openstack-operators | 21:37 | |
| *** SimonChung has joined #openstack-operators | 21:45 | |
| *** SimonChung1 has quit IRC | 21:45 | |
| *** SimonChung1 has joined #openstack-operators | 21:52 | |
| *** SimonChung has quit IRC | 21:52 | |
| *** belmoreira has quit IRC | 21:54 | |
| *** SimonChung has joined #openstack-operators | 22:02 | |
| *** SimonChung1 has quit IRC | 22:02 | |
| *** vinsh has quit IRC | 22:08 | |
| *** Marga_ has quit IRC | 22:11 | |
| *** Marga_ has joined #openstack-operators | 22:11 | |
| *** blair has joined #openstack-operators | 22:13 | |
| *** matrohon has quit IRC | 22:17 | |
| *** saneax has joined #openstack-operators | 22:33 | |
| *** Rockyg has quit IRC | 22:33 | |
| *** alop has quit IRC | 22:51 | |
| *** SimonChung1 has joined #openstack-operators | 23:02 | |
| *** SimonChung has quit IRC | 23:02 | |
| *** Rockyg has joined #openstack-operators | 23:07 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 23:16 | |
| *** Marga_ has quit IRC | 23:20 | |
| *** Marga_ has joined #openstack-operators | 23:21 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 23:31 | |
| *** klindgren has joined #openstack-operators | 23:39 | |
| *** klindgren has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!