*** ncode has quit IRC | 00:00 | |
*** danwent has joined #openstack-meeting | 00:01 | |
*** lloydde_ has quit IRC | 00:01 | |
*** adjohn has quit IRC | 00:03 | |
*** gyee has joined #openstack-meeting | 00:05 | |
*** s0mik has joined #openstack-meeting | 00:06 | |
*** ncode has joined #openstack-meeting | 00:12 | |
*** PotHix has quit IRC | 00:14 | |
*** gyee has quit IRC | 00:16 | |
*** markmcclain has quit IRC | 00:19 | |
*** hemna has quit IRC | 00:27 | |
*** file has joined #openstack-meeting | 00:28 | |
*** s0mik has quit IRC | 00:37 | |
*** danwent has quit IRC | 00:42 | |
*** nati_ueno has quit IRC | 00:48 | |
*** nati_ueno has joined #openstack-meeting | 00:48 | |
*** lloydde has joined #openstack-meeting | 00:54 | |
*** salv-orlando has quit IRC | 00:55 | |
*** zul has quit IRC | 00:57 | |
*** reed has quit IRC | 01:05 | |
*** zul has joined #openstack-meeting | 01:06 | |
*** Mandell has quit IRC | 01:06 | |
*** johnpostlethwait has quit IRC | 01:07 | |
*** dendro-afk is now known as dendrobates | 01:12 | |
*** markmcclain has joined #openstack-meeting | 01:34 | |
*** jakedahn is now known as jakedahn_zz | 01:38 | |
*** danwent has joined #openstack-meeting | 01:50 | |
*** Gordonz has quit IRC | 01:58 | |
*** lloydde has quit IRC | 02:00 | |
*** reed has joined #openstack-meeting | 02:15 | |
*** s0mik has joined #openstack-meeting | 02:26 | |
*** lloydde has joined #openstack-meeting | 02:37 | |
*** gallth is now known as tgall_foo | 02:41 | |
*** tgall_foo has quit IRC | 02:41 | |
*** tgall_foo has joined #openstack-meeting | 02:41 | |
*** dkehn has quit IRC | 02:41 | |
*** thinrhino has joined #openstack-meeting | 02:42 | |
*** jakedahn_zz is now known as jakedahn | 02:46 | |
*** dwcramer has quit IRC | 02:47 | |
*** lloydde has quit IRC | 02:50 | |
*** s0mik has quit IRC | 02:52 | |
*** ywu_ has quit IRC | 02:57 | |
*** s3u has joined #openstack-meeting | 03:03 | |
*** dwcramer has joined #openstack-meeting | 03:03 | |
*** thinrhino has quit IRC | 03:04 | |
*** jdurgin has quit IRC | 03:12 | |
*** s0mik has joined #openstack-meeting | 03:12 | |
*** dwcramer has quit IRC | 03:18 | |
*** ncode has quit IRC | 03:20 | |
*** dwcramer has joined #openstack-meeting | 03:20 | |
*** Mandell has joined #openstack-meeting | 03:26 | |
*** s0mik has quit IRC | 03:26 | |
*** jakedahn is now known as jakedahn_zz | 03:31 | |
*** jakedahn_zz is now known as jakedahn | 03:31 | |
*** dwcramer has quit IRC | 03:35 | |
*** Gordonz has joined #openstack-meeting | 03:58 | |
*** lloydde has joined #openstack-meeting | 04:00 | |
*** Gordonz has quit IRC | 04:02 | |
*** lifeless has quit IRC | 04:03 | |
*** lloydde has quit IRC | 04:05 | |
*** Toanster has joined #openstack-meeting | 04:06 | |
*** dendrobates is now known as dendro-afk | 04:13 | |
*** jakedahn is now known as jakedahn_zz | 04:13 | |
*** lifeless has joined #openstack-meeting | 04:15 | |
*** Toanster_ has joined #openstack-meeting | 04:19 | |
*** rkukura has quit IRC | 04:24 | |
*** thinrhino has joined #openstack-meeting | 04:28 | |
*** thinrhino has quit IRC | 04:29 | |
*** thinrhino has joined #openstack-meeting | 04:29 | |
*** rkukura has joined #openstack-meeting | 04:30 | |
*** rkukura has quit IRC | 04:37 | |
*** Guest22204 has quit IRC | 04:37 | |
*** sacharya has quit IRC | 04:38 | |
*** rkukura has joined #openstack-meeting | 04:40 | |
*** cp16net is now known as cp16net|away | 04:43 | |
*** garyk has quit IRC | 04:43 | |
*** cp16net|away is now known as cp16net | 04:43 | |
*** Mandell has quit IRC | 04:51 | |
*** ywu_ has joined #openstack-meeting | 04:57 | |
*** ywu_ has quit IRC | 05:02 | |
*** blamar has quit IRC | 05:06 | |
*** blamar has joined #openstack-meeting | 05:15 | |
*** s3u has quit IRC | 05:30 | |
*** garyk has joined #openstack-meeting | 05:40 | |
*** eglynn has joined #openstack-meeting | 05:57 | |
*** nati_ueno has quit IRC | 06:01 | |
*** danwent has quit IRC | 06:03 | |
*** Ghe_Rivero is now known as GheRivero | 06:10 | |
*** ttrifonov_zZzz is now known as ttrifonov | 06:18 | |
*** littleidea has quit IRC | 06:22 | |
*** rafaduran has joined #openstack-meeting | 06:44 | |
*** blamar has quit IRC | 06:49 | |
*** markmcclain has quit IRC | 06:50 | |
*** jhenner has joined #openstack-meeting | 06:55 | |
*** ywu_ has joined #openstack-meeting | 06:58 | |
*** ywu_ has quit IRC | 07:02 | |
*** almaisan-away is now known as al-maisan | 07:03 | |
*** Kiall is now known as zz_Kiall | 07:06 | |
*** AlexYang has joined #openstack-meeting | 07:11 | |
*** EmilienM has joined #openstack-meeting | 07:12 | |
*** zz_Kiall is now known as Kiall | 07:24 | |
*** thinrhino has quit IRC | 07:26 | |
*** thinrhino has joined #openstack-meeting | 07:27 | |
*** thinrhino has quit IRC | 07:31 | |
*** jhenner has quit IRC | 07:43 | |
*** thinrhino has joined #openstack-meeting | 08:08 | |
*** salv-orlando has joined #openstack-meeting | 08:09 | |
*** jhenner has joined #openstack-meeting | 08:14 | |
*** derekh has joined #openstack-meeting | 08:15 | |
*** salv-orlando has quit IRC | 08:17 | |
*** salv-orlando has joined #openstack-meeting | 08:17 | |
*** darraghb has joined #openstack-meeting | 08:18 | |
*** salv-orlando has quit IRC | 08:22 | |
*** salv-orlando has joined #openstack-meeting | 08:25 | |
*** salv-orlando has quit IRC | 08:27 | |
*** reed has quit IRC | 08:36 | |
*** thinrhino has quit IRC | 08:41 | |
*** thinrhino has joined #openstack-meeting | 08:41 | |
*** thinrhino has quit IRC | 08:46 | |
*** salv-orlando has joined #openstack-meeting | 08:55 | |
*** ywu_ has joined #openstack-meeting | 08:58 | |
*** salv-orlando has quit IRC | 09:00 | |
*** ywu_ has quit IRC | 09:02 | |
*** Kiall has quit IRC | 09:22 | |
*** Kiall has joined #openstack-meeting | 09:23 | |
*** jd___ has left #openstack-meeting | 09:34 | |
*** thinrhino has joined #openstack-meeting | 09:48 | |
*** shang_ has quit IRC | 09:53 | |
*** shang has joined #openstack-meeting | 09:54 | |
*** shang has quit IRC | 10:05 | |
*** sandywalsh has quit IRC | 10:13 | |
*** AlexYang has left #openstack-meeting | 10:22 | |
*** shang has joined #openstack-meeting | 10:22 | |
*** ywu_ has joined #openstack-meeting | 10:58 | |
*** ywu_ has quit IRC | 11:03 | |
*** milner has joined #openstack-meeting | 11:08 | |
*** sandywalsh has joined #openstack-meeting | 11:14 | |
*** sandywalsh_ has joined #openstack-meeting | 11:23 | |
*** sandywalsh has quit IRC | 11:24 | |
*** mnewby has joined #openstack-meeting | 11:39 | |
*** thinrhino has quit IRC | 11:44 | |
*** markvoelker has joined #openstack-meeting | 11:45 | |
*** ryanpetrello has joined #openstack-meeting | 11:54 | |
*** markvoelker has quit IRC | 12:13 | |
*** markvoelker has joined #openstack-meeting | 12:16 | |
*** salv-orlando has joined #openstack-meeting | 12:21 | |
*** dendro-afk is now known as dendrobates | 12:24 | |
*** ryanpetrello has quit IRC | 12:26 | |
*** salv-orlando has quit IRC | 12:27 | |
*** troytoman-away is now known as troytoman | 12:31 | |
*** troytoman is now known as troytoman-away | 12:33 | |
dolphm | ls | 12:40 |
---|---|---|
*** littleidea has joined #openstack-meeting | 12:46 | |
*** GheRivero has quit IRC | 12:49 | |
*** littleidea has quit IRC | 12:58 | |
*** ywu_ has joined #openstack-meeting | 12:59 | |
*** littleidea has joined #openstack-meeting | 13:00 | |
*** mnewby has quit IRC | 13:02 | |
*** ttrifonov is now known as ttrifonov_zZzz | 13:02 | |
*** ttrifonov_zZzz is now known as ttrifonov | 13:02 | |
*** ywu_ has quit IRC | 13:03 | |
*** dprince has joined #openstack-meeting | 13:03 | |
*** ayoung has joined #openstack-meeting | 13:03 | |
*** blamar has joined #openstack-meeting | 13:10 | |
*** dkehn has joined #openstack-meeting | 13:13 | |
*** blamar has quit IRC | 13:14 | |
*** spn has joined #openstack-meeting | 13:15 | |
*** Dr_Who has joined #openstack-meeting | 13:15 | |
*** Dr_Who has joined #openstack-meeting | 13:15 | |
*** dolphm_ has joined #openstack-meeting | 13:18 | |
*** dolphm_ has quit IRC | 13:18 | |
*** thinrhino has joined #openstack-meeting | 13:18 | |
*** mnewby has joined #openstack-meeting | 13:18 | |
*** ryanpetrello has joined #openstack-meeting | 13:22 | |
*** thinrhino has quit IRC | 13:24 | |
*** gallth has joined #openstack-meeting | 13:30 | |
*** Dr_Who has quit IRC | 13:31 | |
*** ryanpetrello has quit IRC | 13:34 | |
*** thinrhino has joined #openstack-meeting | 13:40 | |
*** sacharya has joined #openstack-meeting | 13:45 | |
*** littleidea has quit IRC | 13:50 | |
*** dwcramer has joined #openstack-meeting | 13:53 | |
*** mattray has joined #openstack-meeting | 13:53 | |
*** thinrhino has quit IRC | 13:53 | |
*** kiffer84 has quit IRC | 13:55 | |
*** thinrhino has joined #openstack-meeting | 13:55 | |
*** sacharya has quit IRC | 13:56 | |
*** kiffer84 has joined #openstack-meeting | 14:02 | |
*** hggdh has quit IRC | 14:04 | |
*** Toanster1 has joined #openstack-meeting | 14:07 | |
*** markmcclain has joined #openstack-meeting | 14:08 | |
*** hggdh has joined #openstack-meeting | 14:12 | |
*** gallth has quit IRC | 14:14 | |
*** PotHix has joined #openstack-meeting | 14:14 | |
*** AlanClark has joined #openstack-meeting | 14:17 | |
*** dtynan has joined #openstack-meeting | 14:18 | |
*** Toanster_ has quit IRC | 14:21 | |
*** Toanster has quit IRC | 14:22 | |
*** Toanster1 is now known as toanster | 14:25 | |
*** toanster has quit IRC | 14:26 | |
*** thinrhino has quit IRC | 14:26 | |
*** Toanster has joined #openstack-meeting | 14:27 | |
*** Mandell has joined #openstack-meeting | 14:28 | |
*** sacharya has joined #openstack-meeting | 14:30 | |
*** Dr_Who has joined #openstack-meeting | 14:31 | |
*** Dr_Who has joined #openstack-meeting | 14:31 | |
*** andrewbogott_afk is now known as andrewbogott | 14:31 | |
*** Mandell has quit IRC | 14:32 | |
*** Toanster has quit IRC | 14:32 | |
*** Toanster has joined #openstack-meeting | 14:33 | |
*** Mandell has joined #openstack-meeting | 14:37 | |
*** littleidea has joined #openstack-meeting | 14:39 | |
*** heckj has joined #openstack-meeting | 14:44 | |
*** alexpilotti has quit IRC | 14:44 | |
*** Dr_Who has quit IRC | 14:47 | |
*** Mandell has quit IRC | 14:49 | |
*** Dr_Who has joined #openstack-meeting | 14:49 | |
*** Dr_Who has joined #openstack-meeting | 14:49 | |
*** markmcclain has quit IRC | 14:58 | |
*** ywu_ has joined #openstack-meeting | 14:59 | |
*** ywu_ has quit IRC | 15:03 | |
*** heckj has quit IRC | 15:03 | |
*** alexpilotti has joined #openstack-meeting | 15:05 | |
*** markmcclain has joined #openstack-meeting | 15:06 | |
*** GheRivero has joined #openstack-meeting | 15:06 | |
*** littleidea has quit IRC | 15:07 | |
*** littleidea has joined #openstack-meeting | 15:13 | |
*** littleidea has quit IRC | 15:14 | |
*** lloydde has joined #openstack-meeting | 15:17 | |
*** andrewbogott is now known as andrewbogott_afk | 15:19 | |
*** littleidea has joined #openstack-meeting | 15:23 | |
*** reed has joined #openstack-meeting | 15:26 | |
*** rongze has joined #openstack-meeting | 15:28 | |
*** lloydde has quit IRC | 15:31 | |
*** jaypipes has joined #openstack-meeting | 15:32 | |
*** al-maisan is now known as almaisan-away | 15:33 | |
*** dolphm_ has joined #openstack-meeting | 15:34 | |
*** ryanpetrello has joined #openstack-meeting | 15:38 | |
*** rongze has quit IRC | 15:38 | |
*** GheRivero is now known as Ghe_Rivero | 15:41 | |
*** blamar has joined #openstack-meeting | 15:41 | |
*** Ghe_Rivero is now known as GheRivero | 15:41 | |
*** almaisan-away is now known as al-maisan | 15:42 | |
*** heckj has joined #openstack-meeting | 15:42 | |
*** blamar_ has joined #openstack-meeting | 15:42 | |
*** littleidea has quit IRC | 15:43 | |
*** troytoman-away is now known as troytoman | 15:45 | |
*** rongze has joined #openstack-meeting | 15:45 | |
*** andrewbogott_afk is now known as andrewbogott | 15:45 | |
*** blamar has quit IRC | 15:46 | |
*** blamar_ is now known as blamar | 15:46 | |
*** ttrifonov is now known as ttrifonov_zZzz | 15:51 | |
*** Gordonz has joined #openstack-meeting | 15:51 | |
*** ttrifonov_zZzz is now known as ttrifonov | 15:52 | |
*** bswartz has joined #openstack-meeting | 15:56 | |
*** winston-d has joined #openstack-meeting | 16:00 | |
*** rafaduran has quit IRC | 16:00 | |
*** rhagarty has joined #openstack-meeting | 16:01 | |
winston-d | hi guys | 16:01 |
jgriffith | Hello! | 16:01 |
jgriffith | #startmeeting cinder | 16:01 |
openstack | Meeting started Wed Sep 5 16:01:46 2012 UTC. The chair is jgriffith. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
openstack | The meeting name has been set to 'cinder' | 16:01 |
jgriffith | Sorry I'm a few minutes late | 16:01 |
jgriffith | busy busy | 16:02 |
*** ncode has joined #openstack-meeting | 16:02 | |
jgriffith | Who do we have? | 16:02 |
winston-d | hi, john | 16:02 |
jgriffith | winston-d: Hello! | 16:02 |
rongze | hi,every one | 16:02 |
winston-d | hi, rongze | 16:02 |
jgriffith | rongze: Hi ya | 16:02 |
jgriffith | Jus the three of us? | 16:03 |
*** clayg has joined #openstack-meeting | 16:03 | |
clayg | heheh | 16:03 |
winston-d | :) | 16:03 |
jgriffith | :) | 16:03 |
rongze | hehe | 16:03 |
jgriffith | So I had a couple of things I wanted to talk about | 16:03 |
dtynan | hello | 16:04 |
* jgriffith rifling desk for notes | 16:04 | |
DuncanT | Hey, sorry | 16:04 |
jgriffith | #topic snapshot deletes | 16:04 |
*** openstack changes topic to "snapshot deletes" | 16:04 | |
jgriffith | https://bugs.launchpad.net/cinder/+bug/1023755 | 16:04 |
uvirtbot | Launchpad bug 1023755 in nova "Unable to delete the volume snapshot" [Undecided,New] | 16:04 |
jgriffith | So this thorn in the side is staring to make a bit more sense | 16:04 |
*** al-maisan is now known as almaisan-away | 16:05 | |
jgriffith | It seems that performing the dd to LVM snapshot volums >= 1G tends to result in a kernel hang | 16:05 |
jgriffith | In the cases that it doesn't hang, unfortunately it's DOG slow | 16:05 |
jgriffith | It appears based on a little bit of googling that this is a kernel bug | 16:06 |
*** ryanpetrello has quit IRC | 16:06 | |
jgriffith | So, I've been trying to find other solutions | 16:06 |
jgriffith | But ultimately I'm wondering: | 16:06 |
jgriffith | 1. Do we need to zero out the snapshot LVM volume at all? | 16:06 |
DuncanT | The problem with not zeroing is you risk leaking your data to the next users of the space | 16:07 |
jgriffith | 2. If we do, any suggestions on another method that might be a bit less intensive? | 16:07 |
jgriffith | DuncanT: Yes, understood | 16:07 |
*** lloydde has joined #openstack-meeting | 16:07 | |
jgriffith | DuncanT: However, being a snap it's on the COW blocks that are actually there right? | 16:08 |
DuncanT | We do our scrubbing out-of-line, but that is kind of tricky with LVM | 16:08 |
bswartz | is that a limitation of LVM? | 16:08 |
winston-d | maybe we can do that in a async way | 16:08 |
jgriffith | DuncanT: With LVM it's not possible that I can see | 16:08 |
jgriffith | winston-d: How do you mean? | 16:08 |
*** Dr_Who has quit IRC | 16:08 | |
*** lloydde has quit IRC | 16:08 | |
jgriffith | winston-d: The problem is the kernel hangs if you try to dd the entire volume | 16:08 |
winston-d | kernel hangs is another problem, async is to deal with DOG slow. | 16:09 |
DuncanT | It would be an easy enough hack to break the dd into chunks? | 16:09 |
clayg | jgriffith: probably because it can't fit all the exceptions | 16:09 |
jgriffith | winston-d: Ohh... we already background it so that's not a killer anyway | 16:09 |
jgriffith | clayg: ? | 16:10 |
clayg | how many extents do we allocate for the snapshot (same size as original volume?) | 16:10 |
jgriffith | clayg: Yes (same size) | 16:10 |
jgriffith | clayg: Oh, I think I know where you're going | 16:10 |
jgriffith | clayg: I don't think that's the problem though | 16:10 |
clayg | so when you write to a snapshot you have to track a) the new data and b) the exception metadata | 16:10 |
jgriffith | clayg: Ok... keep going :) | 16:11 |
clayg | you can overflow a snapshot | 16:11 |
clayg | either with writes into the snapshot, or writes into the origin | 16:11 |
jgriffith | But here's the thing... | 16:11 |
jgriffith | You can repro this by: | 16:11 |
jgriffith | 1. creat volume | 16:11 |
jgriffith | 2. create snapshot | 16:11 |
jgriffith | 3. delete snapshot | 16:11 |
*** jdurgin has joined #openstack-meeting | 16:11 | |
jgriffith | Never mounted/wrote data etc | 16:12 |
clayg | delete snapshot == write full size of snapshot data into snapshot (zero it out) | 16:12 |
bswartz | who designed LVM to expose unwritten blocks as containing previously-written data instead of zeros? | 16:12 |
*** spn has quit IRC | 16:12 | |
clayg | in my experience - it won't fit | 16:12 |
jgriffith | clayg: Hmmm... my understanding was different but you make a good point | 16:12 |
jgriffith | clayg: The trouble is it's not like we get some percentage in and it pukes | 16:13 |
clayg | bswartz: instead of making raw volumes, you could make sparse volumes, you get an empty exception chain (no way to read existing data) - but there is a performance hit (reads and writes) | 16:13 |
jgriffith | clayg: It's unbelievably slow from the start and hits all sorts of kernel time outs etc | 16:13 |
clayg | jgriffith: once the exception list overflows and extent it's expensive to write new blocks (or read old ones really) | 16:14 |
jgriffith | clayg: makes sense.. kinda | 16:14 |
bswartz | We could zero on creation instead of deletion | 16:14 |
jgriffith | bswartz: Same problem and wouldn't solve the issue | 16:14 |
bswartz | that would sidestep the problem | 16:14 |
jgriffith | bswartz: It is the same problem | 16:14 |
jgriffith | bswartz: Remember in this case we never wrote anything | 16:14 |
clayg | jgriffith: well delete just becomes an lvremove | 16:14 |
DuncanT | Can you get the underlying disk range and blank it directly? Not familiar enough with lvm to know, sorry | 16:14 |
jgriffith | bswartz: So in essence we're doing exactly what you state | 16:15 |
jgriffith | clayg: yes | 16:15 |
clayg | jgriffith: and on create - since it's a raw volume, the dd preforms more like you would expect | 16:15 |
bswartz | if you zero on creation instead of deletion then you never need to zero snapshots ever | 16:15 |
*** dolphm_ has quit IRC | 16:15 | |
clayg | DuncanT: absolutely, ls /dev/mapper and a bit of dmsetup | 16:15 |
jgriffith | bswartz: Wait... are you saying zero on volume creation? | 16:15 |
*** ryanpetrello has joined #openstack-meeting | 16:15 | |
clayg | [16:14] bswartz | We could zero on creation instead of deletion | 16:15 |
jgriffith | bswartz: I'm confused by what you're suggesting? | 16:15 |
DuncanT | Except that a) customers tend to expect their data to be gone once they delete the volume b) create then becomes slow | 16:16 |
jgriffith | clayg: in essence that's what we're doing anyway! | 16:16 |
bswartz | if you zero newly created volumes then nobody will ever see old data in their volumes | 16:16 |
jgriffith | clayg: I'm saying in this particular case I never used the volume or the snap | 16:16 |
*** hemna has joined #openstack-meeting | 16:16 | |
jgriffith | clayg: So I don't see the difference, it's basicly creating then zeroing already | 16:16 |
*** Mandell has joined #openstack-meeting | 16:16 | |
jgriffith | zerioing is the problem | 16:16 |
winston-d | jgriffith, do you have link to kernel bug? | 16:16 |
bswartz | yeah but zeroing new volumes allows you to not bother zeroing snapshots | 16:17 |
bswartz | that's all I was getting at | 16:17 |
DuncanT | clayg: I thought you could read the tables out of the dev/mapper devices but I haven't tried... might end up making a nasty mess of the lvm metadata though | 16:17 |
clayg | DuncanT: yes absolutly on both counts | 16:17 |
jgriffith | winston-d: Nope, haven't tracked it down but found a bit of info from other folks having similar issues with 3.2 and dev/mapper files | 16:17 |
*** rhagarty has quit IRC | 16:17 | |
jgriffith | bswartz: How? | 16:17 |
jgriffith | bswartz: users aren't likely to snapshot an empty volume | 16:18 |
jgriffith | bswartz: I kinda see what you're saying but don't think it solves the secure data issue | 16:18 |
clayg | jgriffith: if curing 'creating' you write zeros to the entire volume - you never "expose" data across volumes | 16:19 |
jgriffith | The struggle I'm having is this... you can ensure the security leakage problem, or you can actually be able to delete snapshots :) | 16:19 |
jgriffith | clayg: bswartz: AHHHH | 16:19 |
jgriffith | clayg: bswartz: Finally, I see what you're saying :) | 16:19 |
clayg | jgriffith: np, I think DuncanT already made the valid cretiques why it's not a good idea | 16:19 |
clayg | but you have to admit it would *work* :P | 16:19 |
jgriffith | :) Yes, I believe it would | 16:20 |
clayg | s/a good idea/the ideal solution/ | 16:20 |
clayg | bswartz: it's a quite good idea | 16:20 |
DuncanT | If the otehr option is 'not working', take the slow option :-) | 16:20 |
DuncanT | We (HP) don't particularly care, we don't use LVM... | 16:21 |
rongze | it is lvm issuse... | 16:21 |
*** dolphm_ has joined #openstack-meeting | 16:21 | |
jgriffith | rongze: Yes, it's lvm only | 16:22 |
jgriffith | So it's ugly but a work around: | 16:22 |
winston-d | nobody use lvm iscsi in production, right? | 16:22 |
jgriffith | 1. zero out newly created volumes | 16:22 |
jgriffith | 2. zero out delete volumes still as well (try to meet expectations) | 16:22 |
jgriffith | 3. Skip zero out on snapshot delete | 16:23 |
clayg | jgriffith: seems quite reasonable to me | 16:23 |
clayg | step 4. Make it better later | 16:23 |
jgriffith | clayg: Yes! | 16:24 |
winston-d | jgriffith, make sense for bug fix | 16:24 |
jgriffith | Step 4 is very important :) | 16:24 |
DuncanT | Seems reasonable now - a flag to turn off all the zeroing (or only zero the first meg of new volumes, more sensibly) for test deployments might be appreciated too | 16:24 |
clayg | lol | 16:24 |
*** Gordonz has quit IRC | 16:24 | |
jgriffith | I suppose we could even do this intelligently based on kernel versin? | 16:24 |
jgriffith | DuncanT: That's my next topic! :) | 16:24 |
*** Gordonz has joined #openstack-meeting | 16:24 | |
jgriffith | Ok... seems like we have a plan for this one | 16:25 |
jgriffith | Any last thoughts before I move on? | 16:25 |
DuncanT | Not sure if it is a kernel version issue - fixed size snapshots that aren't bigger than the origin will always overflow if you fill them (which is what the basic dd does) | 16:25 |
jgriffith | DuncanT: Here's a proposal | 16:25 |
jgriffith | DuncanT: I'll try the same test with a snapshot > volume size | 16:26 |
jgriffith | DuncanT: My theory is that it doesn't have anything to do with the size | 16:26 |
DuncanT | ok | 16:26 |
jgriffith | DuncanT: But maybe/hopefully I'll be wrong | 16:26 |
jgriffith | DuncanT: If I come across something different I'll send a note out to everyone | 16:26 |
jgriffith | Sound good to everyone? | 16:26 |
DuncanT | Sounds good to me | 16:27 |
winston-d | yes | 16:27 |
jgriffith | clayg: thoughts? | 16:27 |
clayg | jgriffith: sounds good to me | 16:27 |
jgriffith | cool! Moving on | 16:27 |
clayg | I think "filling up the snapshot exception chain with zeros" is a bad idea ripe to be abandoned | 16:27 |
jgriffith | clayg: Well last night at about 11:00 I definitely agreed with you on that! | 16:28 |
clayg | ya, moving on | 16:28 |
jgriffith | :) | 16:28 |
jgriffith | #topic configurable zeroing of volumes on delete | 16:29 |
*** openstack changes topic to "configurable zeroing of volumes on delete" | 16:29 | |
jgriffith | We talked about setting this via flags and where it should live/be implemented | 16:29 |
jgriffith | here are my opinions: | 16:29 |
jgriffith | 1. It should be implemented in the driver | 16:29 |
jgriffith | That way it can be implemented/handled specifically however is needed | 16:30 |
jgriffith | Vendors that have more clever options can use them | 16:30 |
jgriffith | etc etc | 16:30 |
*** Dr_Who has joined #openstack-meeting | 16:30 | |
*** Dr_Who has joined #openstack-meeting | 16:30 | |
jgriffith | 2. Rather than using a flag and making it a global settging, make it an optional argument to the delete call | 16:30 |
jgriffith | This makes the most sense to me, an admin can use a flag to set an over-ride policy for all tenants/volumes | 16:31 |
DuncanT | 1. Tentatively agree 2. Disagree | 16:31 |
jgriffith | But if they want to provide an option to the tenant on a case by case basis they can | 16:31 |
jgriffith | DuncanT: Ok... Let's start with the tentative: | 16:31 |
jgriffith | Reasons not to implement in the driver? | 16:32 |
jgriffith | And where would you implement it? | 16:32 |
winston-d | are we talking about where should the flag be defined or implemented? | 16:33 |
DuncanT | I raised the point about it being something may volume providers might well want, meaning it is maybe better done as a library call, but I might well be wrong about the number of drivers that actually want the basic version, so I'm entirely happy to be told I was worrying excessively - it isn't like it is tricky code | 16:33 |
*** kindaopsdevy has joined #openstack-meeting | 16:33 | |
bswartz | I prefer (1) | 16:33 |
rongze | I suppot implement in driver | 16:33 |
jgriffith | DuncanT: I guess my reasoning for having it in the driver is to achieve exactly what you describe | 16:33 |
DuncanT | Yeah, ok, I'm convinced, keep it in the drive, I withdraw the tentative | 16:34 |
jgriffith | The third party drivers over-ride the delete operations anyway so they can do their magic however they see fit and the rest of the world doesn't have to know/care abou tit | 16:34 |
jgriffith | Sweet! | 16:34 |
clayg | jgriffith: only way that makes sense to me | 16:34 |
jgriffith | clayg: agreed | 16:34 |
DuncanT | I have a small worry that some 3rd party drives might not consider it, but I'm not prepared to worry overly :-) | 16:34 |
clayg | ... but I don't really see your point on not having it as a driver specific flag? | 16:35 |
jgriffith | DuncanT: That's the beauty of it | 16:35 |
DuncanT | s/drives/drivers/ | 16:35 |
jgriffith | DuncanT: They don't have to | 16:35 |
clayg | DuncanT: operators would not deploy those drivers? :P | 16:35 |
jgriffith | DuncanT: Then it's just not supported and they do whatever they *normally* do on a delete | 16:35 |
jgriffith | DuncanT: They have to implement delete_volume right? | 16:36 |
jgriffith | DuncanT: So they just ignore options regarding zeroing out etc | 16:36 |
jgriffith | Seems like the right way to go to me | 16:36 |
jgriffith | Or as clayg states, those drivers don't get to play :) | 16:36 |
jgriffith | Just kidding | 16:36 |
DuncanT | L-) | 16:37 |
DuncanT | :-) | 16:37 |
DuncanT | Gah, can't type | 16:37 |
DuncanT | Ok, shall we consider 2. ? | 16:37 |
clayg | so - if it's implemented in the driver - why isn't it a driver specific flag? | 16:37 |
jgriffith | Yes, if everybody is comfortable with item 1 | 16:37 |
bswartz | I'd just like to point out that leaving it up the drivers is effectively the situation we have right now | 16:37 |
jgriffith | bswartz: True.. but | 16:37 |
bswartz | so this isn't a change except for the LVM-based driver | 16:37 |
rongze | I think only lvm care about the flag... | 16:38 |
jgriffith | bswartz: The reality is the driver is the one who has to implement/do the work anyway | 16:38 |
jgriffith | bswartz: It may or may not be, depends on what the driver/vendor is capable of | 16:38 |
rongze | other drivers can do nothing | 16:38 |
*** dolphm_ has quit IRC | 16:39 | |
winston-d | rongze, why is that? | 16:39 |
jgriffith | rongze: Yeah, but some devices may have options here | 16:39 |
jgriffith | rongze: other than just LVM... for example an HP array has a shred method | 16:39 |
clayg | or it may not apply to a sparse volume, or file based backing store. | 16:39 |
jgriffith | rongze: And it may also have a DOD compliant shred | 16:40 |
clayg | customer could always zero volume before calling deleting | 16:40 |
jgriffith | So this would allow those to be selected/implemented | 16:40 |
rongze | yes | 16:40 |
*** danwent has joined #openstack-meeting | 16:40 | |
jgriffith | clayg: yes, I personally prefer the customer do what they want up front :) | 16:40 |
creiht | hah | 16:40 |
clayg | so back to... you were suggesting something about an addative addition to the api? | 16:40 |
jgriffith | clayg: Ahh... right | 16:41 |
jgriffith | So I'm not a huge fan of flags | 16:41 |
clayg | I LOVE FLAGS! | 16:41 |
clayg | oh wait.. | 16:41 |
jgriffith | They're global for every tennant, every volume etc etc | 16:41 |
* jgriffith slaps clayg upside the head | 16:41 | |
jgriffith | So say for example a tenant has a two volumes... | 16:42 |
jgriffith | One has credit card and billing info data stored on it | 16:42 |
jgriffith | The other has pictures of kittens | 16:42 |
clayg | is there a bug for this? who acctually raised the issue? I think it's totally reasonable that a deployer/operator would say (these are your volume options, this is our security policy - deal with it) | 16:42 |
DuncanT | I think, in general, zeroing volumes is a good and necessary thing | 16:43 |
jgriffith | clayg: So the issue was raised in a bug... lemme find it | 16:43 |
clayg | and then they just choose the impl (and flags) that best match the service level they want to apply | 16:43 |
DuncanT | I *really* don't think relying on the customer to do it is reasonable | 16:43 |
jgriffith | The reasoning was they wanted the ability to speed things up, this is mostly only applicable to LVM case | 16:43 |
*** matiu has quit IRC | 16:43 | |
DuncanT | The only time you might want to turn it off is a test build, where speed is more useful | 16:44 |
jgriffith | DuncanT: Fair, but really it sounds like maybe it's not even a necessary option any more? | 16:44 |
DuncanT | Getting data security right is hard, don't let the customer get it wrong where at all possible | 16:44 |
*** dolphm_ has joined #openstack-meeting | 16:44 | |
jgriffith | DuncanT: I can see your point | 16:44 |
*** anniec has quit IRC | 16:44 | |
*** darraghb has quit IRC | 16:44 | |
*** garyk has quit IRC | 16:44 | |
DuncanT | jgriffith: 'I built my openstack cloud and it leaked all my data to other users' is not a good headline | 16:44 |
clayg | DuncanT: yeah well, write zero's once, or some other silly "shred" business is where I tell the customer they're welcome to whatever. I think a simple wipe over the drive with zeros is all a deployer would want to do (but that's assuming it's a raw device, vhd's and other file based backends don't ever really "wipe" the just do their append thing) | 16:45 |
rongze | I agree DuncanT | 16:45 |
DuncanT | So even the devstack default should be 'safe' | 16:45 |
DuncanT | But a flag for power users might be appreciated | 16:45 |
jgriffith | clayg: DuncanT Ok, so I'm wondering if this is even something we should mess with then | 16:45 |
DuncanT | (simple wipe of zeros is as safe as a shread) | 16:45 |
clayg | DuncanT: +1 | 16:46 |
jgriffith | I mean really we don't want to do this anywhere else except maybe in testing, but even then it's not that big of a deal is it? | 16:46 |
bswartz | so are we proposing wiping with zeros on creation or deletion (for LVM volumes) | 16:46 |
clayg | I would suggest serious developers not even do it in testing - who raised the bug? | 16:46 |
clayg | bswartz: that seems to be the way we're going | 16:47 |
bswartz | but we're already wiping on deletion -- and that is the root cause of the bug (unless I misunderstand) | 16:47 |
jgriffith | https://bugs.launchpad.net/cinder/+bug/1022511 | 16:47 |
uvirtbot | Launchpad bug 1022511 in nova "Allow for configurable policy for wiping data when deleting volumes" [Undecided,In progress] | 16:47 |
DuncanT | bswartz: wiping snapshots is the (hang) problem, not normal volumes | 16:48 |
bswartz | my suggestion for address the bug was to move the zero operation from delete time to create time, and to no longer zero deleted snapshots | 16:48 |
jgriffith | bswartz: Correct... but the bug is another issue and it's ONLY snapshot LVM's | 16:48 |
jgriffith | bswartz: yes, and that's the way we're going with the bug | 16:48 |
jgriffith | bswartz: This topic was about the bug raised for folks wanting to be able to configure various methods of wiping data | 16:49 |
jgriffith | but it's sounding like maybe this is a none-issue | 16:49 |
jgriffith | We stick with zeroing on LVM and let the third part vendors do whatever they do | 16:49 |
clayg | yeah I'd mark the bug as "opinion" and ask the submitter to file a blueprint | 16:49 |
clayg | ^ for grizzly! | 16:50 |
uvirtbot | clayg: Error: "for" is not a valid command. | 16:50 |
bswartz | On NetApp, newly created volumes are sparse (all zeros) so it doesn't really affect us either way | 16:50 |
jgriffith | the initial reasoning behind the bug does make some sense | 16:50 |
clayg | uvirtbot: I hate you | 16:50 |
uvirtbot | clayg: Error: "I" is not a valid command. | 16:50 |
jgriffith | bswartz: Rigth but we have to think of the base case as well | 16:50 |
clayg | "environments where security is not a concern" is should hopefully be very few | 16:50 |
*** s0mik has joined #openstack-meeting | 16:50 | |
winston-d | clayg, haha, uvirtbot hits back really soon | 16:51 |
jgriffith | Ok... so based on our conversation I'm going to nix this for now | 16:51 |
DuncanT | I'd suggest a) leave it up to the driver b) default the lvm/iscsi driver to zero as agreed c) have a flag for power users who don't care about zeroing and just want to quickly test stuff on devstack | 16:51 |
jgriffith | clayg and virtbot are always entertaining | 16:51 |
jgriffith | DuncanT: a = yes, b = yes, c = I don't see the point | 16:52 |
jgriffith | I don't want devstack tests running without this | 16:52 |
jgriffith | The snapshot delete bug is the perfect example | 16:52 |
bswartz | I agree with jgriffith here | 16:52 |
DuncanT | jgriffith: I sometimes do load testing on devstack, and the dd is the biggest time consumer, by a large factor | 16:52 |
DuncanT | I can always continue to hack this by hand if nobody agrees with me :-) | 16:53 |
bswartz | if you want the default driver to not zero for performance reasons, you can hack the driver in your environment | 16:53 |
*** ryanpetrello has quit IRC | 16:53 | |
jgriffith | DuncanT: I see your point, I really do and I don't disagree entirely | 16:53 |
DuncanT | but I'm far from the only person who wants this: https://lists.launchpad.net/openstack/msg14333.html | 16:54 |
jgriffith | The problem is that if everybody gets in the habbit of doing this in their devstack tests we never see bugs like the LVM one above until it's too late | 16:54 |
clayg | maybe just have a flag for the lvm driver where you can which command to use (instead of dd, you could give it "echo") | 16:54 |
jgriffith | DuncanT: yes, lots of people state they want it, that's why I initially thought it would be good | 16:54 |
clayg | jgriffith: even if everyone else runs quick dd in their devsetups, me and you don't have to :) | 16:55 |
jgriffith | clayg: Yeah, that was my initial thought on all of this, but now I'm concerned about the test matrix :( | 16:55 |
jgriffith | :) | 16:55 |
jgriffith | clayg: Ok, that's a good compromise for me | 16:55 |
DuncanT | clayg++ | 16:55 |
jgriffith | I'll conceed | 16:55 |
jgriffith | Ok... so we move forward with implementing this: | 16:55 |
jgriffith | 1. It's a flag set by admin/power user | 16:56 |
jgriffith | 2. Probably not Folsom but Grizzly time frame | 16:56 |
clayg | either way, I think the bug as written is an opinion, and w/o a blue print all that other stuff doesn't belong in Folsom. | 16:56 |
*** jog0 has joined #openstack-meeting | 16:56 | |
jgriffith | clayg: agreed... | 16:56 |
*** jog0 has joined #openstack-meeting | 16:56 | |
jgriffith | Ok... anybody have anything else on this topic? | 16:56 |
jgriffith | Last minute pleas to change our minds and get it in Folsom etc? | 16:57 |
jgriffith | s/pleas/plees/ | 16:57 |
jgriffith | #topic docs | 16:57 |
*** openstack changes topic to "docs" | 16:57 | |
jgriffith | Ok... I need help! | 16:57 |
* DuncanT hides under the table | 16:58 | |
clayg | ehhe - I acctually really do have to leave | 16:58 |
clayg | that's funny | 16:58 |
jgriffith | Anybody and everybody we need to come up with good documentation | 16:58 |
jgriffith | clayg: Not funny at all :( | 16:58 |
jgriffith | Alright... I guess that closes that topic | 16:58 |
jgriffith | I'll deal with it | 16:58 |
clayg | but maybe I can help... I've been working on the api fixes - maybe I could start there? | 16:58 |
jgriffith | So really I'd even be happy if folks just wrote up a google doc on their driver and I'll convert it to the xml etc | 16:59 |
clayg | do the api docs go into sphix or that other openstack-docs project that anne deals with? | 16:59 |
jgriffith | Or even better any feature/change you implemented do a short write up on it and send it to me | 16:59 |
bswartz | I'm joining another meeting in 1 minute -- I do have plans to document the netapp drivers | 16:59 |
jgriffith | sphinx | 16:59 |
jgriffith | Oh... no | 16:59 |
jgriffith | openstack-docs | 16:59 |
clayg | yeah... I'll have to look into that | 16:59 |
jgriffith | I sent the link in the last meeting | 16:59 |
*** ywu_ has joined #openstack-meeting | 17:00 | |
clayg | last thought, I have to bolt, jenkins is all up in my reviews blowing up my change sets? | 17:00 |
clayg | it *really* looks like a environment problem and not a valid failure | 17:00 |
jgriffith | clayg: I'll have a look, been having issues the past few days just doing rechecks | 17:00 |
jgriffith | I'll check them out and run recheck if I dont see something in there | 17:00 |
jgriffith | #topic open discussion | 17:00 |
*** openstack changes topic to "open discussion" | 17:00 | |
jgriffith | alright, we have 30 seconds :) | 17:01 |
clayg | jgriffith: thanx | 17:01 |
jgriffith | clayg: NP | 17:01 |
*** clayg has left #openstack-meeting | 17:01 | |
jgriffith | Anybody have anything pressing they want to bring up? | 17:01 |
jgriffith | Keep in mind RC1 cut off at the end of this week | 17:01 |
jgriffith | Please keep an eye on reviews (I still have a couple large ones that I need eyes on) | 17:02 |
creiht | I have a quick question | 17:02 |
jgriffith | And there are others rolling in | 17:02 |
jgriffith | creiht: Go | 17:02 |
creiht | What should the expected behavior be for someone who tries to attach/detach a volume to an instance that has been shutdown? | 17:02 |
*** derekh has quit IRC | 17:02 | |
jgriffith | creiht: Hmmm... that's interesting | 17:03 |
creiht | indeed :) | 17:03 |
jgriffith | creiht: TBH I hadn't thought of it | 17:03 |
winston-d | successful i guess? | 17:03 |
jgriffith | creiht: First thought is that since the instances are ephemeral it should fail | 17:03 |
jgriffith | :) | 17:03 |
jgriffith | So much for my first thought | 17:03 |
winston-d | just like you install a new hd drive into your PC? | 17:03 |
*** lloydde has joined #openstack-meeting | 17:04 | |
jgriffith | yeah, but libvirt won't have a way to make the connection | 17:04 |
jgriffith | creiht: Have you tried this? | 17:04 |
*** ywu_ has quit IRC | 17:04 | |
jgriffith | Of course I'm not even familiar with how you have a *shutdown* instance | 17:05 |
jgriffith | But that's my own ignorance I have to deal with :) | 17:05 |
DuncanT | When are we supposed to have the backports to nova-volume done? | 17:05 |
jgriffith | DuncanT: I think that will start next week | 17:05 |
jgriffith | DuncanT: I looked a bit yesterday and we've been fairly good about doing this as we go anyway so it may not be so bad | 17:06 |
*** kindaopsdevy has left #openstack-meeting | 17:06 | |
DuncanT | You're right, there isn't much | 17:06 |
jgriffith | I'll probably do a great big meld on the /volume directory :) | 17:06 |
winston-d | can i restart a 'shutdown' instance? if so, then attach should be successful. | 17:06 |
jgriffith | winston-d: If you can then I can see your point | 17:06 |
jgriffith | winston-d: I just don't know how that works with libvirt | 17:06 |
jgriffith | Really, that becomes more of a nova-compute question and I'll have to play around with it | 17:07 |
jgriffith | #action jgriffith Look into creiht request about attach to shutdown instance | 17:07 |
winston-d | libvirt can track down the change into instance xml configuration, i think | 17:07 |
creiht | winston-d: yes | 17:08 |
jgriffith | winston-d: yes, I think you're correct | 17:08 |
jgriffith | winston-d: In which case it should just execute when the instance starts | 17:08 |
creiht | The more common use case is someone shuts down their instance | 17:08 |
creiht | and wants to detach a volume, so they can attach elsewhere | 17:08 |
jgriffith | creiht: In that case would it matter? | 17:09 |
jgriffith | creiht: In the case of LVM would the device still be considered as mounted? | 17:09 |
winston-d | creiht, so 'shutdown' a instance doesn't automatically detach volume? | 17:09 |
jgriffith | err... that last part didn't really make sense I don't think | 17:10 |
creiht | winston-d: it doesn't | 17:10 |
jgriffith | creiht: winston-d Would that be something we should implement? | 17:10 |
creiht | jgriffith: when a volume is attached, it can't be attached to another instance | 17:10 |
jgriffith | creiht: Yeah, understand | 17:10 |
creiht | jgriffith: I don't think we want to auto-detach on shutdown | 17:11 |
winston-d | creiht, then i think we should allow detach from shutdown instance. | 17:11 |
creiht | as you want volumes to persist if you reboot | 17:11 |
creiht | that's what I was thinking | 17:11 |
*** GheRivero has quit IRC | 17:11 | |
winston-d | creiht, i agree | 17:11 |
creiht | I just don't think anyone else has really thought it through | 17:11 |
creiht | it is one of those edge cases you don't think about until you have a customer trying to do it :) | 17:12 |
jgriffith | creiht: winston-d Seems like a good case to me | 17:12 |
jgriffith | Grizzly, we'll do a blueprint | 17:12 |
jgriffith | Seems like the best answer | 17:12 |
winston-d | what if someone live migrate a instance with volume attached? how are we dealing with this case? | 17:12 |
jgriffith | although I still have to understand the case of shutting down an instance and restarting it | 17:12 |
jgriffith | Didn't know/think you could even do that | 17:13 |
creiht | winston-d: in that case they are detached and re-attached | 17:13 |
winston-d | jgriffith, hotplug | 17:13 |
winston-d | creiht, yeah, that's what i guess. | 17:13 |
rongze | jgriffith, what blueprint? | 17:13 |
*** lloydde has quit IRC | 17:13 | |
jgriffith | rongze: I was suggesting that a blueprint should be created for: | 17:14 |
*** markmcclain has quit IRC | 17:14 | |
jgriffith | allowing detach from an instance that is shutdown | 17:14 |
winston-d | as well as attach to an instance that is shutdown? | 17:14 |
jgriffith | winston-d: Ooops.. yes, forgot that part | 17:14 |
rongze | nice blueprint | 17:14 |
jgriffith | :) | 17:15 |
jgriffith | Pretty cool actually | 17:15 |
rongze | using mobilephone to login irc is too bad.... | 17:15 |
jgriffith | LOL | 17:15 |
jgriffith | I tried that once... no fun | 17:15 |
jgriffith | creiht: does that sum up what you were thinking? | 17:16 |
winston-d | if you're just lurking... it might be ok | 17:16 |
creiht | jgriffith: I believe so | 17:16 |
jgriffith | cool, we can add additional thoughts/ideas once we get a blueprint posted | 17:16 |
creiht | thx | 17:16 |
jgriffith | creiht: Grizzly-1 | 17:17 |
jgriffith | Alright... I've gotta run unfortunately | 17:17 |
jgriffith | Thanks everyone! | 17:17 |
jgriffith | #endmeeting | 17:17 |
*** openstack changes topic to "cidner-usecases" | 17:18 | |
openstack | Meeting ended Wed Sep 5 17:17:58 2012 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:18 |
winston-d | thx, john | 17:18 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/cinder/2012/cinder.2012-09-05-16.01.html | 17:18 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/cinder/2012/cinder.2012-09-05-16.01.txt | 17:18 |
openstack | Log: http://eavesdrop.openstack.org/meetings/cinder/2012/cinder.2012-09-05-16.01.log.html | 17:18 |
*** dtynan has left #openstack-meeting | 17:18 | |
rongze | thx,john | 17:18 |
*** rafaduran has joined #openstack-meeting | 17:18 | |
*** rongze has left #openstack-meeting | 17:20 | |
*** ryanpetrello has joined #openstack-meeting | 17:20 | |
*** Dr_Who has quit IRC | 17:21 | |
*** matiu has joined #openstack-meeting | 17:22 | |
*** matiu has joined #openstack-meeting | 17:22 | |
*** lloydde has joined #openstack-meeting | 17:24 | |
*** anniec has joined #openstack-meeting | 17:24 | |
*** anniec_ has joined #openstack-meeting | 17:25 | |
*** dolphm_ has quit IRC | 17:25 | |
*** Dr_Who has joined #openstack-meeting | 17:26 | |
*** Dr_Who has quit IRC | 17:26 | |
*** Dr_Who has joined #openstack-meeting | 17:26 | |
*** lloydde has quit IRC | 17:28 | |
*** anniec has quit IRC | 17:28 | |
*** anniec_ is now known as anniec | 17:28 | |
*** jog0 has quit IRC | 17:29 | |
*** ryanpetrello has quit IRC | 17:33 | |
*** dendrobates is now known as dendro-afk | 17:40 | |
*** ywu has quit IRC | 17:41 | |
*** ryanpetrello has joined #openstack-meeting | 17:43 | |
*** littleidea has joined #openstack-meeting | 17:43 | |
*** ryanpetrello has quit IRC | 17:44 | |
*** winston-d has quit IRC | 17:48 | |
*** nati_ueno has joined #openstack-meeting | 17:52 | |
*** dolphm_ has joined #openstack-meeting | 17:53 | |
*** mattray has quit IRC | 17:54 | |
*** andrewbogott is now known as andrewbogott_afk | 17:54 | |
*** gyee has joined #openstack-meeting | 17:57 | |
*** AlanClark has quit IRC | 17:59 | |
*** dolphm_ has quit IRC | 18:03 | |
*** thinrhino has joined #openstack-meeting | 18:04 | |
*** dendro-afk is now known as dendrobates | 18:05 | |
*** thinrhino has quit IRC | 18:05 | |
*** andrewbogott_afk is now known as andrewbogott | 18:06 | |
*** markmcclain has joined #openstack-meeting | 18:06 | |
*** thinrhino has joined #openstack-meeting | 18:07 | |
*** bswartz has left #openstack-meeting | 18:16 | |
*** bcwaldon has left #openstack-meeting | 18:26 | |
*** jhenner has quit IRC | 18:28 | |
*** jhenner1 has joined #openstack-meeting | 18:28 | |
*** ayoung_ has joined #openstack-meeting | 18:31 | |
*** lloydde has joined #openstack-meeting | 18:34 | |
*** dolphm_ has joined #openstack-meeting | 18:36 | |
*** lloydde has quit IRC | 18:40 | |
*** jhenner1 has quit IRC | 18:41 | |
*** garyk has joined #openstack-meeting | 18:48 | |
*** novas0x2a|laptop has quit IRC | 18:52 | |
*** mattray has joined #openstack-meeting | 18:54 | |
*** danwent has quit IRC | 18:58 | |
*** s0mik has quit IRC | 18:59 | |
*** Dr_Who has quit IRC | 18:59 | |
*** ywu has joined #openstack-meeting | 19:00 | |
*** Dr_Who has joined #openstack-meeting | 19:01 | |
*** Dr_Who has joined #openstack-meeting | 19:01 | |
*** ywu has quit IRC | 19:04 | |
*** nati_ueno has quit IRC | 19:05 | |
*** nati_ueno has joined #openstack-meeting | 19:06 | |
*** thinrhino has quit IRC | 19:06 | |
*** dprince has quit IRC | 19:11 | |
*** alexpilotti has quit IRC | 19:14 | |
*** mnewby has quit IRC | 19:17 | |
*** mnewby has joined #openstack-meeting | 19:17 | |
*** novas0x2a|laptop has joined #openstack-meeting | 19:21 | |
*** s0mik has joined #openstack-meeting | 19:22 | |
*** jhenner has joined #openstack-meeting | 19:26 | |
*** jhenner has quit IRC | 19:36 | |
*** anniec has quit IRC | 19:41 | |
*** dendrobates is now known as dendro-afk | 19:55 | |
*** anniec has joined #openstack-meeting | 19:56 | |
*** n0ano has joined #openstack-meeting | 20:02 | |
*** troytoman is now known as troytoman-away | 20:02 | |
*** heckj_ has joined #openstack-meeting | 20:07 | |
*** heckj has quit IRC | 20:07 | |
*** dendro-afk is now known as dendrobates | 20:11 | |
*** chalupaul has quit IRC | 20:13 | |
*** dolphm has quit IRC | 20:14 | |
*** danwent has joined #openstack-meeting | 20:32 | |
*** chalupaul has joined #openstack-meeting | 20:34 | |
*** troytoman-away is now known as troytoman | 20:45 | |
*** ryanpetrello has joined #openstack-meeting | 20:48 | |
*** anniec_ has joined #openstack-meeting | 20:50 | |
*** anniec has quit IRC | 20:53 | |
*** anniec_ is now known as anniec | 20:53 | |
*** dolphm_ has quit IRC | 20:58 | |
*** anniec has quit IRC | 21:00 | |
*** ywu has joined #openstack-meeting | 21:00 | |
*** ywu has quit IRC | 21:05 | |
*** markvoelker has quit IRC | 21:06 | |
*** anniec has joined #openstack-meeting | 21:08 | |
*** anniec_ has joined #openstack-meeting | 21:09 | |
*** ryanpetrello has quit IRC | 21:09 | |
*** johnpur has joined #openstack-meeting | 21:10 | |
*** johnpur has quit IRC | 21:10 | |
*** johnpur has joined #openstack-meeting | 21:11 | |
*** anniec has quit IRC | 21:12 | |
*** anniec_ is now known as anniec | 21:12 | |
*** maurosr has joined #openstack-meeting | 21:13 | |
*** almaisan-away is now known as al-maisan | 21:19 | |
*** adjohn has joined #openstack-meeting | 21:23 | |
*** ryanpetrello has joined #openstack-meeting | 21:25 | |
*** kindaopsdevy_ has joined #openstack-meeting | 21:27 | |
*** al-maisan is now known as almaisan-away | 21:27 | |
*** kindaopsdevy_ has left #openstack-meeting | 21:27 | |
*** AndChat|396416 has joined #openstack-meeting | 21:31 | |
*** ryanpetrello has quit IRC | 21:33 | |
*** troytoman is now known as troytoman-away | 21:33 | |
*** maurosr has quit IRC | 21:34 | |
*** AndChat|396416 has quit IRC | 21:35 | |
*** Dr_Who has quit IRC | 21:39 | |
*** nati_ueno has quit IRC | 21:43 | |
*** jog0 has joined #openstack-meeting | 21:44 | |
*** markmcclain has quit IRC | 21:47 | |
*** sacharya has quit IRC | 21:55 | |
*** dwcramer has quit IRC | 21:57 | |
*** kindaopsdevy__ has joined #openstack-meeting | 22:01 | |
*** kindaopsdevy__ has left #openstack-meeting | 22:01 | |
*** anniec has quit IRC | 22:02 | |
*** anniec has joined #openstack-meeting | 22:03 | |
*** anniec_ has joined #openstack-meeting | 22:04 | |
*** anniec has quit IRC | 22:08 | |
*** anniec_ is now known as anniec | 22:08 | |
*** dolphm has joined #openstack-meeting | 22:09 | |
*** ryanpetrello has joined #openstack-meeting | 22:10 | |
*** ozstacker has quit IRC | 22:11 | |
*** dolphm has quit IRC | 22:13 | |
*** rafaduran has left #openstack-meeting | 22:17 | |
*** s0mik has quit IRC | 22:32 | |
*** ijw1 has joined #openstack-meeting | 22:36 | |
*** ijw has quit IRC | 22:36 | |
*** eglynn has quit IRC | 22:38 | |
*** eglynn has joined #openstack-meeting | 22:38 | |
*** s0mik has joined #openstack-meeting | 22:41 | |
*** ryanpetrello has quit IRC | 22:46 | |
*** anniec has quit IRC | 22:46 | |
*** sandywalsh_ has quit IRC | 22:47 | |
*** nati_ueno has joined #openstack-meeting | 22:48 | |
*** mattray has quit IRC | 22:52 | |
*** anniec has joined #openstack-meeting | 22:58 | |
*** ijw1 has quit IRC | 23:00 | |
*** ijw has joined #openstack-meeting | 23:00 | |
*** ywu has joined #openstack-meeting | 23:01 | |
*** ywu has quit IRC | 23:05 | |
*** littleidea has quit IRC | 23:09 | |
*** littleidea has joined #openstack-meeting | 23:11 | |
*** kindaopsdevy has joined #openstack-meeting | 23:26 | |
*** kindaops_ has joined #openstack-meeting | 23:28 | |
*** kindaops_ has left #openstack-meeting | 23:28 | |
*** EmilienM has quit IRC | 23:29 | |
*** kindaopsdevy has left #openstack-meeting | 23:30 | |
*** matwood has joined #openstack-meeting | 23:39 | |
*** dwcramer has joined #openstack-meeting | 23:40 | |
*** andrewbogott is now known as andrewbogott_afk | 23:40 | |
*** ryanpetrello has joined #openstack-meeting | 23:42 | |
*** ryanpetrello has quit IRC | 23:46 | |
*** s0mik has quit IRC | 23:47 | |
*** ryanpetrello has joined #openstack-meeting | 23:47 | |
*** novas0x2a|laptop has quit IRC | 23:47 | |
*** hemna has quit IRC | 23:49 | |
*** anniec has quit IRC | 23:50 | |
*** anniec_ has joined #openstack-meeting | 23:50 | |
*** s0mik has joined #openstack-meeting | 23:53 | |
*** ryanpetrello has quit IRC | 23:54 | |
*** ryanpetrello has joined #openstack-meeting | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!