*** jamesmcarthur has joined #openstack-meeting-alt | 00:04 | |
*** jamesmcarthur has quit IRC | 00:05 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 00:05 | |
*** lseki has quit IRC | 00:09 | |
*** ttsiouts has quit IRC | 00:12 | |
*** yamamoto has quit IRC | 00:18 | |
*** ijw has joined #openstack-meeting-alt | 00:19 | |
*** ijw has quit IRC | 00:27 | |
*** gyee has quit IRC | 00:38 | |
*** licanwei has quit IRC | 01:01 | |
*** jamesmcarthur has quit IRC | 01:12 | |
*** baojg has joined #openstack-meeting-alt | 01:32 | |
*** yamamoto has joined #openstack-meeting-alt | 01:41 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 01:43 | |
*** jamesmcarthur has quit IRC | 01:44 | |
*** jamesmcarthur_ has joined #openstack-meeting-alt | 01:44 | |
*** apetrich has quit IRC | 01:56 | |
*** yamamoto has quit IRC | 02:03 | |
*** bhavikdbavishi has joined #openstack-meeting-alt | 02:04 | |
*** ttsiouts has joined #openstack-meeting-alt | 02:09 | |
*** bhavikdbavishi has quit IRC | 02:15 | |
*** bhavikdbavishi has joined #openstack-meeting-alt | 02:20 | |
*** iyamahat has quit IRC | 02:24 | |
*** yamahata has quit IRC | 02:26 | |
*** jamesmcarthur_ has quit IRC | 02:33 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 02:34 | |
*** yamamoto has joined #openstack-meeting-alt | 02:39 | |
*** ttsiouts has quit IRC | 02:42 | |
*** yamamoto has quit IRC | 02:51 | |
*** yamamoto has joined #openstack-meeting-alt | 02:54 | |
*** jamesmcarthur has quit IRC | 03:08 | |
*** bhavikdbavishi has quit IRC | 03:10 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 03:11 | |
*** ayoung has joined #openstack-meeting-alt | 03:27 | |
*** jamesmcarthur has quit IRC | 03:34 | |
*** ayoung has quit IRC | 03:42 | |
*** e0ne has joined #openstack-meeting-alt | 03:47 | |
*** lpetrut has joined #openstack-meeting-alt | 03:53 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 04:05 | |
*** lpetrut has quit IRC | 04:18 | |
*** janki has joined #openstack-meeting-alt | 04:38 | |
*** ttsiouts has joined #openstack-meeting-alt | 04:39 | |
*** e0ne has quit IRC | 04:46 | |
*** ttsiouts has quit IRC | 05:12 | |
*** radeks has joined #openstack-meeting-alt | 05:35 | |
*** yamamoto has quit IRC | 05:42 | |
*** bhavikdbavishi has joined #openstack-meeting-alt | 05:44 | |
*** yamamoto has joined #openstack-meeting-alt | 05:46 | |
*** yamamoto has quit IRC | 05:51 | |
*** vishalmanchanda has joined #openstack-meeting-alt | 05:58 | |
*** belmoreira has joined #openstack-meeting-alt | 06:06 | |
*** slaweq has joined #openstack-meeting-alt | 06:06 | |
*** whoami-rajat has joined #openstack-meeting-alt | 06:20 | |
*** yamamoto has joined #openstack-meeting-alt | 06:28 | |
*** yamamoto has quit IRC | 06:29 | |
*** yamamoto has joined #openstack-meeting-alt | 06:30 | |
*** kopecmartin|off is now known as kopecmartin | 06:54 | |
*** apetrich has joined #openstack-meeting-alt | 06:59 | |
*** tesseract has joined #openstack-meeting-alt | 07:09 | |
*** ttsiouts has joined #openstack-meeting-alt | 07:09 | |
*** panda|off is now known as panda | 07:19 | |
*** rdopiera has joined #openstack-meeting-alt | 07:24 | |
*** ttsiouts has quit IRC | 07:42 | |
*** ttsiouts has joined #openstack-meeting-alt | 07:46 | |
*** belmoreira has quit IRC | 07:55 | |
*** belmoreira has joined #openstack-meeting-alt | 07:58 | |
*** e0ne has joined #openstack-meeting-alt | 08:14 | |
*** e0ne has quit IRC | 08:17 | |
*** jtomasek has joined #openstack-meeting-alt | 08:37 | |
*** panda has quit IRC | 08:55 | |
*** panda has joined #openstack-meeting-alt | 08:55 | |
*** ttsiouts has quit IRC | 09:15 | |
*** tssurya has joined #openstack-meeting-alt | 09:18 | |
*** wxy-xiyuan has quit IRC | 09:26 | |
*** belmoreira has quit IRC | 09:33 | |
*** e0ne has joined #openstack-meeting-alt | 09:38 | |
*** e0ne has quit IRC | 09:40 | |
*** yamamoto has quit IRC | 09:45 | |
*** belmoreira has joined #openstack-meeting-alt | 09:48 | |
*** e0ne has joined #openstack-meeting-alt | 09:56 | |
*** tssurya has quit IRC | 09:57 | |
*** e0ne has quit IRC | 09:59 | |
*** raildo has joined #openstack-meeting-alt | 10:00 | |
*** yamamoto has joined #openstack-meeting-alt | 10:06 | |
*** mugsie has quit IRC | 10:25 | |
*** yamamoto has quit IRC | 10:27 | |
*** yamamoto has joined #openstack-meeting-alt | 10:27 | |
*** yamamoto has quit IRC | 10:28 | |
*** ttsiouts has joined #openstack-meeting-alt | 10:31 | |
*** yamamoto has joined #openstack-meeting-alt | 10:32 | |
*** yamamoto has quit IRC | 10:32 | |
*** yamamoto_ has joined #openstack-meeting-alt | 10:32 | |
*** yamamoto_ has quit IRC | 10:33 | |
*** mugsie has joined #openstack-meeting-alt | 10:35 | |
*** mugsie has quit IRC | 10:35 | |
*** mugsie has joined #openstack-meeting-alt | 10:36 | |
*** ttsiouts has quit IRC | 10:36 | |
*** mugsie has quit IRC | 10:38 | |
*** mugsie has joined #openstack-meeting-alt | 10:39 | |
*** tesseract has quit IRC | 10:40 | |
*** tesseract has joined #openstack-meeting-alt | 10:41 | |
*** tssurya has joined #openstack-meeting-alt | 10:42 | |
*** priteau has joined #openstack-meeting-alt | 10:43 | |
*** tesseract has quit IRC | 10:45 | |
*** tesseract has joined #openstack-meeting-alt | 10:45 | |
*** belmoreira has quit IRC | 10:55 | |
*** belmoreira has joined #openstack-meeting-alt | 10:56 | |
*** panda is now known as panda|lunch | 11:02 | |
*** yamamoto has joined #openstack-meeting-alt | 11:08 | |
*** carloss has joined #openstack-meeting-alt | 11:09 | |
*** yamamoto has quit IRC | 11:13 | |
*** belmoreira has quit IRC | 11:15 | |
*** priteau has quit IRC | 11:23 | |
*** yamamoto has joined #openstack-meeting-alt | 11:34 | |
*** belmoreira has joined #openstack-meeting-alt | 11:48 | |
*** rf0lc0 is now known as rfolco|ruck | 11:49 | |
*** janki has quit IRC | 11:54 | |
*** zigo has quit IRC | 11:59 | |
*** yamamoto has quit IRC | 12:02 | |
*** belmoreira has quit IRC | 12:03 | |
*** panda|lunch is now known as panda | 12:06 | |
*** yamamoto has joined #openstack-meeting-alt | 12:07 | |
*** baojg has quit IRC | 12:08 | |
*** iyamahat has joined #openstack-meeting-alt | 12:15 | |
*** jamesmcarthur has quit IRC | 12:21 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 12:21 | |
*** iyamahat has quit IRC | 12:21 | |
*** zigo has joined #openstack-meeting-alt | 12:23 | |
*** jamesmcarthur has quit IRC | 12:30 | |
*** yamamoto has quit IRC | 12:40 | |
*** belmoreira has joined #openstack-meeting-alt | 12:49 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 12:49 | |
*** lseki has joined #openstack-meeting-alt | 12:51 | |
*** jamesmcarthur has quit IRC | 13:05 | |
*** yamamoto has joined #openstack-meeting-alt | 13:06 | |
*** baojg has joined #openstack-meeting-alt | 13:20 | |
*** ttsiouts has joined #openstack-meeting-alt | 13:24 | |
*** lbragstad_ has joined #openstack-meeting-alt | 13:29 | |
*** lbragstad_ is now known as lbragstad | 13:30 | |
*** tesseract has quit IRC | 13:35 | |
*** tesseract has joined #openstack-meeting-alt | 13:35 | |
*** altlogbot_2 has quit IRC | 13:43 | |
*** altlogbot_2 has joined #openstack-meeting-alt | 13:45 | |
*** redrobot has quit IRC | 13:46 | |
*** helenafm has joined #openstack-meeting-alt | 13:47 | |
*** Guest89536 has joined #openstack-meeting-alt | 13:49 | |
*** vhari has joined #openstack-meeting-alt | 13:50 | |
*** Guest89536 is now known as redrobot | 13:55 | |
*** vishalmanchanda has quit IRC | 13:57 | |
*** bhavikdbavishi has quit IRC | 14:02 | |
*** ttsiouts has quit IRC | 14:08 | |
*** ccamacho has joined #openstack-meeting-alt | 14:21 | |
*** ttsiouts has joined #openstack-meeting-alt | 14:23 | |
*** tesseract has quit IRC | 14:24 | |
*** liuyulong has joined #openstack-meeting-alt | 14:25 | |
*** ttsiouts has quit IRC | 14:28 | |
*** yamamoto has quit IRC | 14:35 | |
*** ganso has joined #openstack-meeting-alt | 14:44 | |
*** tesseract has joined #openstack-meeting-alt | 14:50 | |
*** tesseract has quit IRC | 14:51 | |
*** tesseract has joined #openstack-meeting-alt | 14:51 | |
*** hongbin has joined #openstack-meeting-alt | 14:56 | |
*** ttsiouts has joined #openstack-meeting-alt | 14:59 | |
gouthamr | #startmeeting manila | 15:00 |
---|---|---|
openstack | Meeting started Thu May 9 15:00:21 2019 UTC and is due to finish in 60 minutes. The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
*** openstack changes topic to " (Meeting topic: manila)" | 15:00 | |
openstack | The meeting name has been set to 'manila' | 15:00 |
gouthamr | courtesy ping: xyang toabctl bswartz ganso erlon tpsilva vkmc amito jgrosso | 15:00 |
lseki | o/ | 15:00 |
carloss | Hey :) | 15:00 |
ganso | hello | 15:00 |
gouthamr | hello o/ | 15:00 |
*** dviroel has joined #openstack-meeting-alt | 15:00 | |
gouthamr | tbarron is away, and i'm chairing today - my first in a long time so expect some clumsiness :) | 15:00 |
*** tzumainn has joined #openstack-meeting-alt | 15:00 | |
*** tesseract has quit IRC | 15:01 | |
bswartz | .o/ | 15:01 |
gouthamr | we'll wait a couple of minutes for quorum | 15:01 |
vhari | hi | 15:01 |
dviroel | o/ | 15:01 |
amito | o/ | 15:01 |
bswartz | Oh good we can trout slap the chair today | 15:01 |
bswartz | :-) | 15:01 |
gouthamr | :D | 15:01 |
gouthamr | #chair bswartz | 15:01 |
openstack | Current chairs: bswartz gouthamr | 15:01 |
bswartz | Doh! | 15:01 |
gouthamr | now we can | 15:01 |
gouthamr | #topic Announcements | 15:02 |
*** openstack changes topic to "Announcements (Meeting topic: manila)" | 15:02 | |
*** jgrosso has joined #openstack-meeting-alt | 15:02 | |
gouthamr | Welcome back, everyone that traveled to the mile high city | 15:02 |
gouthamr | and thanks everyone that joined us remote :) | 15:03 |
vkmc | o/ | 15:03 |
gouthamr | we had a great PTG and Summit; and it felt good to be back to the old times personally | 15:03 |
*** tesseract has joined #openstack-meeting-alt | 15:03 | |
bswartz | Did it feel like old times with the updated format? | 15:04 |
bswartz | That what I would have hoped for | 15:04 |
*** ttsiouts has quit IRC | 15:04 | |
gouthamr | yes, felt as packed as BCN, which i recall was the last combined conference (Summit + Design Summit) | 15:04 |
gouthamr | the PTG seemed to have a good turn out, comparable to the exclusive event in Denver last cycle, although there were a few new projects represented | 15:05 |
gouthamr | we have a lot to follow up from the PTG ourselves - a day and a half seemed productive and discussions were well paced | 15:06 |
gouthamr | 15:06 | |
gouthamr | in terms of announcements, I proposed the change we discussed at the PTG: A Parking lot for old unimplemented specs | 15:07 |
gouthamr | #LINK https://review.opendev.org/#/c/657894/ | 15:07 |
gouthamr | found a few ones that we might want to move there, please take a look | 15:08 |
vkmc | like the "parking lot" term | 15:08 |
gouthamr | if you have a spec there that needs to be moved to the Train release and not the "unimplemented" section, please note it in review | 15:09 |
vkmc | do we want to clean up those specs? say, change the expected release and asignee fields | 15:09 |
gouthamr | vkmc: yes, the act of moving the specs out of the release specific folder should signify the former | 15:10 |
gouthamr | vkmc: however, i dunno if we should pop out the assignee/s field.. | 15:10 |
gouthamr | vkmc: that is probably a good idea, however - we can use another field that suggests the design author, and leave the assignee field empty | 15:11 |
*** tesseract has quit IRC | 15:11 | |
vkmc | gouthamr, yeah, I mean, in the blueprint you can see who proposed what | 15:11 |
vkmc | so the authorship remains the same | 15:12 |
*** yamamoto has joined #openstack-meeting-alt | 15:12 | |
vkmc | but... having the assignee field filled in might prevent people to take the blueprint and work on it | 15:12 |
gouthamr | true true... if the authors are available to consult, but not code, it might make sense for the new owner to reach out to them | 15:13 |
* gouthamr labels himself design consultant, prints new business cards | 15:13 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 15:13 | |
vkmc | :) | 15:14 |
gouthamr | so PTAL, everyone, and we'll take these discussions to the review | 15:14 |
gouthamr | #topic Summit (29 April 29-May 1) and PTG (May 2-4) summary, action items | 15:15 |
*** openstack changes topic to "Summit (29 April 29-May 1) and PTG (May 2-4) summary, action items (Meeting topic: manila)" | 15:15 | |
gouthamr | #LINK https://etherpad.openstack.org/p/manila-ptg-train | 15:15 |
*** tesseract has joined #openstack-meeting-alt | 15:16 | |
gouthamr | alright, on this one, thanks to note takers, we have a decent summary against each topic and action items listed underneath | 15:16 |
gouthamr | we did not cover all the topics we had proposed, so we'll schedule those topics in our weekly meetings if the proposers can participate | 15:17 |
gouthamr | tbarron is working on an upstream summary, and will post one to the ML (openstack-discuss) soon-ish (TM) | 15:17 |
*** yamamoto has quit IRC | 15:17 | |
gouthamr | if he doesn't, vkmc and I'll channel him and do it anyway :) | 15:18 |
gouthamr | thank you NetApp folks (erlon, dviroel, lseki) for getting us AV support | 15:18 |
*** tesseract has quit IRC | 15:18 | |
gouthamr | and thank you vkmc for hosting the bluejeans session for our remote attendees and for recording the whole thing | 15:18 |
vkmc | ++ | 15:19 |
lseki | :-) | 15:19 |
dviroel | :) | 15:19 |
gouthamr | we'll plan on posting the recording along with the summary to the mailing list | 15:19 |
*** turnerg has joined #openstack-meeting-alt | 15:19 | |
gouthamr | does anyone have anything to say about this $topic? | 15:19 |
* gouthamr hears crickets, sees smiles, moves on :) | 15:20 | |
gouthamr | #topic Train Schedule | 15:20 |
*** openstack changes topic to "Train Schedule (Meeting topic: manila)" | 15:20 | |
*** tesseract has joined #openstack-meeting-alt | 15:20 | |
bswartz | #pun | 15:20 |
*** dtrainor has joined #openstack-meeting-alt | 15:20 | |
gouthamr | wee | 15:20 |
gouthamr | ooh, if you got lucky, you could get your own train whistle at the PTG and blow it on the corridors | 15:21 |
gouthamr | and attract annoyed faces :) | 15:21 |
bswartz | Heh | 15:21 |
bswartz | What is "train" a reference to? | 15:22 |
gouthamr | he asks ^ | 15:22 |
bswartz | Is it the loud train near the PTG hotel? | 15:22 |
ganso | bswartz: the trains in denver close to the hotel | 15:22 |
*** helenafm has left #openstack-meeting-alt | 15:22 | |
bswartz | K | 15:22 |
gouthamr | tbarron flagged off a review for the manila release schedule here | 15:23 |
gouthamr | #LINK https://review.opendev.org/#/c/655667/ | 15:23 |
vkmc | I think it was the only release for OpenStack that was named because everybody got nuts with the naming idea on the denver ptg 2018 feedback session | 15:23 |
gouthamr | it overlays the openstack release schedule with the manila project specific deadlines | 15:23 |
gouthamr | please visit that, and provide your reviews | 15:24 |
gouthamr | there's also our own tracking wiki | 15:24 |
gouthamr | #LINK https://wiki.openstack.org/wiki/Manila/TrainCycle | 15:25 |
gouthamr | it's only beginning to be filled | 15:25 |
gouthamr | PTAL (heh, GitHub PRs you ruined me) | 15:25 |
gouthamr | we'll track our work on that wiki page, so feel free to add your own work items, and status as we proceed through the release | 15:26 |
bswartz | vkmc: I think people couldn't resist the puns | 15:26 |
gouthamr | absolutely, hop on the choochoostack | 15:27 |
vkmc | bswartz, exactly, that made it special :) | 15:27 |
vkmc | haha | 15:27 |
gouthamr | from next week, we'll be the fare enforcement officers on this train, and check your statuses for the work on the wiki | 15:28 |
gouthamr | moving on.. | 15:28 |
gouthamr | #topic Extend share when clients are connected (gouthamr) | 15:28 |
*** openstack changes topic to "Extend share when clients are connected (gouthamr) (Meeting topic: manila)" | 15:28 | |
gouthamr | alright, that's me pinging me | 15:29 |
gouthamr | this could be a known issue for our veteran contributors who at one point or another worked with/on the generic driver | 15:29 |
gouthamr | #LINK https://review.opendev.org/#/c/531568/ | 15:29 |
gouthamr | ^ is a scenario test - a test case suggested by vponomaryov | 15:30 |
bswartz | This is a good one | 15:30 |
* gouthamr misses the dude | 15:30 | |
bswartz | The dude abides | 15:31 |
bswartz | What's the issue here though? | 15:31 |
bswartz | Does it not work? | 15:31 |
gouthamr | so the problem here is the mount becomes stale after issuing an extend API to manila | 15:31 |
gouthamr | the mount inside the nova VM* | 15:32 |
gouthamr | the test logs from a previous CI run are here: | 15:32 |
gouthamr | #LINK http://logs.openstack.org/68/531568/26/check/manila-tempest-dsvm-scenario/1ea651d/logs/testr_results.html.gz | 15:32 |
gouthamr | the error of interest is "dd: failed to open '/mnt/t3': Stale file handle" | 15:32 |
bswartz | Yeah that would be expected for some drivers | 15:33 |
gouthamr | nirg22, who is implementing this test case for us is confused, because it works on the other first party drivers - LVM and ZFSOnLinux | 15:33 |
bswartz | We were not strict about the expectation on the client side, because we have no way to prevent this kind of problem | 15:33 |
bswartz | Sometimes a resize will require a remount, and sometimes it won't | 15:34 |
bswartz | We don't tell the user which situation they're in | 15:34 |
bswartz | They just have to know, or learn empircally | 15:34 |
gouthamr | wont shares from the generic driver always require a remount? | 15:34 |
bswartz | Yes | 15:34 |
bswartz | But the user doesn't know which driver they've got | 15:34 |
gouthamr | true | 15:35 |
gouthamr | i wonder if we can use the "online extend" capability in the cinder API | 15:35 |
gouthamr | although it comes with its own caveats | 15:35 |
bswartz | Does LVM suffer from this at all? | 15:36 |
bswartz | Maybe the answer is to stop using Generic | 15:37 |
*** baojg has quit IRC | 15:37 | |
gouthamr | bswartz: nope, LVM passes the test: http://logs.openstack.org/68/531568/26/check/manila-tempest-minimal-dsvm-lvm/130046f/logs/testr_results.html.gz | 15:37 |
gouthamr | "manila_tempest_tests.tests.scenario.test_share_extend.TestShareExtendNFS" is the test to look for there ^ | 15:37 |
ganso | gouthamr: generic is cinder using LVM underneath, but it happens because here we need to use resize2fs | 15:38 |
ganso | wait a min, LVM also uses resize2fs | 15:38 |
gouthamr | ganso: so the client (generic driver share server) needs to issue a resize2fs? | 15:39 |
gouthamr | ganso: or do you mean this is if we were to use the new online-extension API | 15:39 |
ganso | gouthamr: I'd say it would need to whatever the LVM is doing, because essentially they would work the same in this situation | 15:39 |
bswartz | No | 15:39 |
ganso | gouthamr: s/would need to/would need to do | 15:40 |
bswartz | Cinder is not the same as LVM | 15:40 |
bswartz | The problem is the block device resize, not the filesystem resize | 15:40 |
bswartz | LVs can be resized online | 15:40 |
ganso | gouthamr: what would be the caveat of cinder online extend? | 15:40 |
gouthamr | ganso: the limitations around where it can be used - not all cinder backends support extending attached volumes iirc, and not all nova hypervisors support it either (only libvirt driver has support last i looked, but i could be wrong) | 15:41 |
gouthamr | bswartz: on stopping the use of the generic driver :) - this is the only job where scenario tests are running with DHSS=True | 15:42 |
gouthamr | bswartz: we did take an AI to run scenario tests in the container job... | 15:43 |
ganso | gouthamr: what if we make it configurable? the main importance of the generic driver is in the gate or POC deployments | 15:43 |
bswartz | If we can use cinder online resize we might be able to work around this issue | 15:43 |
bswartz | But the deeper question is what guarantees we make to our end users | 15:43 |
gouthamr | ack | 15:44 |
ganso | gouthamr: if a cloud is using the generic driver in production with a backend that wouldn't work with online extend, just disable it and we fallback to the other problem (if there is no other way to solve the problem) | 15:44 |
bswartz | If we want to guarantee online resizes, then certain drivers may not be able to fulfill the contract | 15:44 |
bswartz | If we don't want to guarantee online resizes, then the test needs to accommodate the drivers that can't, which probably requires communicating some extra information at the API level | 15:44 |
gouthamr | ganso: yes, that would be a good direction to take - unless we want to extract the cinder pools information, and check for the capability atleast from cinder's side and fail faster | 15:44 |
ganso | gouthamr: that is a good idea as well, but the history that I know of online extend capability is a messy one | 15:45 |
ganso | gouthamr: it was enabled by default for everyone | 15:45 |
gouthamr | bswartz: makes sense, a new capability flag that defaults to True? | 15:45 |
ganso | gouthamr: and then the vendors that do not support it need to disable | 15:45 |
bswartz | ganso: We have to make the first decision before we discuss the mechanism for the second | 15:46 |
ganso | gouthamr, bswartz: Why a capability and not a config option just for the generic driver? | 15:46 |
gouthamr | ganso bswartz: ^ yeah, we don't know what other drivers suffer this problem | 15:47 |
bswartz | The question is, can users assume nondisruptive resize or not | 15:47 |
bswartz | If no, then how do they find out if it's disruptive or not | 15:47 |
bswartz | I kinda like assuming nondisruptive | 15:48 |
bswartz | But we should figure which drivers that might break | 15:48 |
*** priteau has joined #openstack-meeting-alt | 15:48 | |
ganso | the scenario test should point that out | 15:48 |
ganso | problem is 3rd parties are not running scenario tests | 15:49 |
gouthamr | +1 the test case looks good right now, and everyone needs to be running it and testing the various backends we have | 15:49 |
ganso | so the only way to obtain this information is mailing list? | 15:49 |
*** belmoreira has quit IRC | 15:49 | |
gouthamr | ganso: yes, sadly, we have one or two AIs about scenario tests from the PTG | 15:49 |
gouthamr | part of the problem is our own inconsistent testing in the first party drivers | 15:50 |
bswartz | Well with this new scenarios test we are de-facto assuming that resizes are nondisruptive | 15:50 |
bswartz | So maybe we just merge it, fix (or kill) the generic driver, and see what other drivers fail the scenario test | 15:50 |
bswartz | Because the status quo is to not tell users if the resize is disruptive | 15:51 |
gouthamr | i wonder if we can try...except and log a warning saying a remount was required to pass the test | 15:51 |
*** kopecmartin is now known as kopecmartin|off | 15:51 | |
ganso | gouthamr: what about the user experience then? | 15:51 |
bswartz | We can later choose to relax the contstraints by communicating about disruptive resizes if we decide we want to continue to allow them | 15:51 |
ganso | gouthamr: and the user's expectations | 15:51 |
bswartz | ganso: My suggestion leaves things unchanged, user-experience-wise, until we decide to relax things. | 15:52 |
ganso | LGTM | 15:52 |
gouthamr | ganso: although i will take an AI to bring this discussion on the ML for the wider audience | 15:53 |
gouthamr | i know that NetApp and Dell/EMC want to run our scenario tests | 15:53 |
gouthamr | would be nice to have all the other drivers do the same | 15:53 |
* gouthamr recalls a bug downstream that vhari pointed out regarding a manila/CIFS scenario test | 15:54 | |
jgrosso | gouthamr I want to run your scenario tests also ! | 15:54 |
*** gyee has joined #openstack-meeting-alt | 15:54 | |
gouthamr | jgrosso++ | 15:55 |
vkmc | jgrosso++ | 15:55 |
vhari | tempest_manila_plugin.tests.scenario.test_shares_scenario.SharesScenarioTest.test_scenario_1 | 15:55 |
*** priteau has quit IRC | 15:56 | |
gouthamr | vhari: that one looks like an internal test.. | 15:56 |
gouthamr | i'm thinking about this one: https://bugzilla.redhat.com/show_bug.cgi?id=1703185 | 15:56 |
openstack | bugzilla.redhat.com bug 1703185 in python-manila-tests-tempest "Tempest manila test fail" [Unspecified,New] - Assigned to rhos-maint | 15:56 |
vhari | here is the CIFS one: manila_tempest_tests.tests.scenario.test_share_basic_ops.TestShareBasicOpsCIFS) | 15:57 |
gouthamr | ack, ty :) | 15:57 |
vhari | :D | 15:57 |
gouthamr | alright, we're almost out of time... this was a great discussion, and we have some AIs | 15:57 |
*** bnemec has quit IRC | 15:58 | |
*** priteau has joined #openstack-meeting-alt | 15:58 | |
gouthamr | #action: suggest next steps for the scenario test regarding the generic driver failure, maintain current user experience/expectations | 15:58 |
*** ijw has joined #openstack-meeting-alt | 15:58 | |
*** bnemec has joined #openstack-meeting-alt | 15:58 | |
jgrosso | Thank you Goutham I don't have anyone new bugs to discuss today ! | 15:58 |
jgrosso | :) | 15:59 |
jgrosso | be prepared next week! | 15:59 |
gouthamr | #action: raise issue on the ML, point to the scenario test - propose a solution to provide a consistent user experience by claiming support (or lack thereof) of online/nondisruptive extensions | 15:59 |
gouthamr | cool, ty jgrosso | 15:59 |
gouthamr | #topic Open Discussion | 15:59 |
*** openstack changes topic to "Open Discussion (Meeting topic: manila)" | 15:59 | |
bswartz | Thx for chairing gouthamr | 15:59 |
jgrosso | gouthamr: welcome | 15:59 |
gouthamr | we have 1 minute :) | 15:59 |
gouthamr | anything anyone wants in the meeting logs before we all retire to #openstack-manila? | 15:59 |
* bswartz reaches for large trout | 16:00 | |
gouthamr | oh ho, we're out of time :) | 16:00 |
gouthamr | #endmeeting | 16:00 |
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 16:00 | |
openstack | Meeting ended Thu May 9 16:00:24 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:00 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/manila/2019/manila.2019-05-09-15.00.html | 16:00 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/manila/2019/manila.2019-05-09-15.00.txt | 16:00 |
openstack | Log: http://eavesdrop.openstack.org/meetings/manila/2019/manila.2019-05-09-15.00.log.html | 16:00 |
*** dviroel has left #openstack-meeting-alt | 16:00 | |
gouthamr | thank you everyone, see you all in #openstack-manila | 16:00 |
priteau | #startmeeting blazar | 16:01 |
* gouthamr reaches for trout shield, activates it, looks around suspiciously | 16:01 | |
openstack | Meeting started Thu May 9 16:01:15 2019 UTC and is due to finish in 60 minutes. The chair is priteau. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
*** openstack changes topic to " (Meeting topic: blazar)" | 16:01 | |
*** tesseract has quit IRC | 16:01 | |
openstack | The meeting name has been set to 'blazar' | 16:01 |
priteau | #topic Roll call | 16:01 |
*** openstack changes topic to "Roll call (Meeting topic: blazar)" | 16:01 | |
priteau | Hi. This is the first Blazar meeting in the new timezone for Americas. | 16:03 |
*** bhavikdbavishi has joined #openstack-meeting-alt | 16:04 | |
turnerg | maybe everybody slept in | 16:05 |
priteau | Hi turnerg. Are you interested in Blazar? | 16:06 |
tzumainn | hi! I haven't really been involved in blazar, but I read some notes from the ptg about possible future ironic integration, and thought that maybe I'd lurk a bit | 16:06 |
turnerg | Yes; we have some GPU's that we're thinking about how to make available on a fair basis | 16:06 |
*** diurnalist has joined #openstack-meeting-alt | 16:06 | |
*** bhavikdbavishi has quit IRC | 16:07 | |
turnerg | Been thinking about Blazar off & on since 2016; come from an HPC background where scheduling is a solved problem +/- error | 16:07 |
*** bhavikdbavishi has joined #openstack-meeting-alt | 16:07 | |
turnerg | For the time being our cloud is not at capacity but, someday, it's gonna happen | 16:08 |
priteau | Hi tzumainn. I remember your message on the mailing list back in February. You are working on the Hardware leasing with Ironic project, is that right? | 16:08 |
priteau | Hi diurnalist | 16:09 |
*** ttsiouts has joined #openstack-meeting-alt | 16:09 | |
tzumainn | priteau, that's correct! I saw some notes from dtantsur in ironic that mentioned updating ironic with owner/lessee fields, and blazar possibly integrating with that | 16:09 |
priteau | I am still expecting another person from University of Chicago to join, they're working through NickServ registration right now. | 16:09 |
priteau | Let's start and they'll read through the logs later ;-) | 16:12 |
priteau | #topic Presentation of participants | 16:12 |
*** openstack changes topic to "Presentation of participants (Meeting topic: blazar)" | 16:12 | |
*** jakecoll has joined #openstack-meeting-alt | 16:13 | |
priteau | Hi jakecoll | 16:13 |
jakecoll | Hey, sorry about that | 16:13 |
priteau | Because this is the first meeting in this timezone, I thought it would be useful for everyone to present themselves and the OpenStack deployment or product they're working on, if any. And of course why they're interested in Blazar. | 16:13 |
priteau | I'll start | 16:13 |
priteau | I am Pierre Riteau, I am the PTL for Blazar in the Train cycle. I have used and developed Blazar since 2015 through the Chameleon testbed: https://www.chameleoncloud.org/ | 16:14 |
priteau | I am not directly involved with Chameleon anymore, but diurnalist and jakecoll so I'll let them talk about it | 16:15 |
priteau | Let's go alphabetically. diurnalist? | 16:15 |
diurnalist | I'm Jason Anderson, DevOps lead for Chameleon. Chameleon is a scientific testbed with ~500 bare metal nodes that we provision with Ironic. Users reserve nodes with Blazar, and we've also been working to extend the idea of reservations to VLAN segments and floating IPs. Blazar is a core part of how Chameleon works; we use it to track allocations on the testbed as well. | 16:17 |
priteau | Actually it will be too slow to do this one by one, everyone can present themselves now | 16:17 |
priteau | Thanks diurnalist | 16:17 |
priteau | Anyone else? jakecoll, turnerg, tzumainn | 16:19 |
turnerg | I'm George Turner, operator with the Jetstream project https://jetstream-cloud.org/ a cloud for science users. We'd like to schedule/allocate GPUs for defined periods of time; something similar for classroom environment. We have worked with some of the admins on Chameleon | 16:19 |
jakecoll | My name is Jake Colleran. I'm Jason Anderson/diurnalist's lackey over on Chameleon. | 16:19 |
*** rdopiera has quit IRC | 16:20 | |
tzumainn | hi! I'm Tzu-Mainn Chen, and I'm interested in the idea of using Blazar to lease ironic nodes; I saw some PTG discussion on the ironic side about how to accommodate that, so I thought I'd peek in here to see what future plans might be | 16:20 |
priteau | turnerg: Nice to see some interest from Jetstream. Do you already have GPUs that users keep allocated for too long? | 16:21 |
turnerg | wanna head that rpoblem off before ti happens ;) we have some gpus on order; they're not GA yet | 16:22 |
priteau | Blazar allows users to define exactly when they want to allocate specific kind of resources, though it needs to be used with a policy to enforce sharing | 16:23 |
priteau | I know Chameleon had to tune its policies because GPU users kept reserving them for long periods | 16:24 |
*** diurnalist_ has joined #openstack-meeting-alt | 16:24 | |
diurnalist_ | I temporarily dropped out; I blame mozilla | 16:25 |
priteau | BTW if you get disconnected, you can read the logs at http://eavesdrop.openstack.org/meetings/blazar/2019/blazar.2019-05-09-16.01.log.txt | 16:25 |
jakecoll | Worth mentioning, we have had problems with users stacking leases, especially gpus. | 16:25 |
diurnalist_ | priteau: thanks | 16:25 |
*** diurnalist has quit IRC | 16:26 | |
priteau | jakecoll: can you share what is the latest policy in use on Chameleon? | 16:26 |
*** diurnalist_ is now known as diurnalist | 16:27 | |
jakecoll | We've developed a policy where users cannot have advanced reservations that back up against each other over 2 days. However, we enforce this with our own scripts that operate outside blazar itself. | 16:27 |
priteau | turnerg: out of the box Blazar doesn't provide any limitations on reservations, so users can reserve GPU nodes even more easily than they can launch instances (which would be subject to quota). Chameleon has developed extensions to limit reservation length, but users were working around that by making multiple advance reservations one after the other. | 16:28 |
priteau | jakecoll: so if a reservation finishes on a Thursday at noon, the user cannot reserve the same resource until Saturday noon? | 16:29 |
*** stephenfin has quit IRC | 16:30 | |
jakecoll | priteau: Yes. Otherwise, the later advanced reservation will be deleted and an email notification sent. | 16:30 |
*** diablo_rojo has joined #openstack-meeting-alt | 16:31 | |
priteau | It would be good to see if some enforcement policies are common enough that they could be integrated in upstream Blazar. | 16:32 |
diurnalist | We have had to introduce some pretty wacky policies due to contention over those highly-prized resources. I'm sure there is a better way of expressing the rule. We considered limiting a user/project to having N leases at a given time. However, this could require that users learn some more of the advanced features of reservations, like the fact that one lease can contain many reservations for different resource types | 16:32 |
priteau | turnerg: How do you envision users would share your GPU resources? Do you have some kind of usage charged against an allocation? | 16:33 |
diurnalist | priteau: correct me if I'm wrong, but Blazar currently has no mechanism to actually charge for usage, correct? it just can handle advanced scheduling of time allotments on a resource. | 16:34 |
*** jamesmcarthur has quit IRC | 16:34 | |
diurnalist | because I've wondered if this is something that has already been discussed, and what changes might be necessary for Chameleon's solution to this to make it upstream. | 16:34 |
turnerg | asorry; dealing with an interrupt | 16:36 |
priteau | diurnalist: That's correct. Unfortunately I am not aware of a standard solution in this space. OpenStack has CloudKitty but I think it only computes usage after the fact and can convert it in $$$ to charge a credit card. | 16:36 |
*** tssurya has quit IRC | 16:38 | |
priteau | Maybe some reservation usage could be expressed like other quotas. Each OpenStack project could have a quota of "seconds/hours/days of advance reservation time" and Blazar could check against it. | 16:40 |
priteau | Keystone is working on unified limits which may be relevant for this: https://docs.openstack.org/keystone/queens/admin/identity-unified-limits.html | 16:41 |
turnerg | We would find it attractive to be able to charge for the tie reserved; Users have ana llocation annd if they burn it then there's at least a cost. We can slice & dice the GPUs down to 1/8 of a GPU; we do it by flavors | 16:43 |
priteau | turnerg: That's what Chameleon does, using a custom allocation backend and extensions to Blazar. Do you already have a similar allocation system in place for your existing cloud? | 16:44 |
turnerg | Reserving without using would be very bad. I could see how smart people could figure that one out | 16:44 |
*** ganso has left #openstack-meeting-alt | 16:44 | |
diurnalist | turnerg: yes. we found that users would set up scripts to automatically just keep making leases in the future for one node. one enterprising user effectively reserved one node for a few months until we spotted it | 16:45 |
turnerg | We have no allocation process other then they use their XSEDE allocation, we go through by hand and disable the project | 16:45 |
diurnalist | turnerg: do XSEDE allocations come in some sort of unit? like a compute hour. or is it more of a "license to compute" | 16:46 |
*** irclogbot_2 has quit IRC | 16:46 | |
*** munimeha1 has joined #openstack-meeting-alt | 16:46 | |
turnerg | hours * CPU_core | 16:46 |
*** irclogbot_1 has joined #openstack-meeting-alt | 16:47 | |
priteau | Are the XSEDE allocations stored in a custom database or is there some common cluster/grid software that handles this? | 16:47 |
turnerg | one SU = ONE core for one Hour. We anticipate using an multipler to make the GPU more expensive | 16:47 |
turnerg | custom | 16:47 |
diurnalist | turnerg: that is exactly what we do. custom multipliers for some thing. we also allow users to reserve GENI slices. it's a VLAN tag but we have a 10x multiplier on those because we only have a few and they're very useful for SDN experimentation. | 16:48 |
turnerg | thinking out loud, if we had some phantom instance start up on the reserved resource and that had a high value charge per time yet consumned no resources on the host, then the u ser could launch thier instance with did real work on the reserved node but that instance would have a zero caahrge rate. this way they'd be charged for the reservation without having to modify our current scripts that just log the time instances are | 16:50 |
turnerg | running | 16:50 |
priteau | turnerg: That's fine for charging reservations that are running, but users who make advance reservations may still reserve more than they should. | 16:51 |
turnerg | ah; thinking.... | 16:51 |
priteau | If Blazar was making REST requests to a customisable endpoint on reservation creation / update, expecting to get a simple yes/no answer (with some details, like how much SUs are left compared to how much would be used), would people be motivated to write a small REST service making the link between Blazar and any custom allocation backend? | 16:52 |
turnerg | I guess the quoda above is a start | 16:52 |
priteau | Time is flying and we're almost finished with the hour. | 16:54 |
priteau | This is a very interesting conversation, I think we should continue it either by email or at the next meeting in two weeks | 16:54 |
diurnalist | :+1: | 16:54 |
priteau | I just want to talk a bit about Ironic | 16:55 |
priteau | #topic Summit and PTG feedback | 16:55 |
*** openstack changes topic to "Summit and PTG feedback (Meeting topic: blazar)" | 16:55 | |
priteau | I wasn't in Denver but I was told that Dmitry (from Ironic) and Tetsuro (from Blazar and Placement) had good discussions about how to make Blazar and Ironic standalone work. | 16:56 |
priteau | There are some details in Dmitry's email to the list: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005951.html | 16:57 |
tzumainn | I read a summary from Dmitry that talked a bit about that, and mentioned that someone on the ironic side that would come up with a blueprint detailing what they'd do | 16:57 |
tzumainn | would there be something similar forthcoming on the blazar side? | 16:57 |
priteau | I haven't had the chance to talk with Tetsuro yet, but I am expecting we would have some specs to match. | 16:58 |
tzumainn | okay, cool! I'd be excited to read that and perhaps offer some feedback | 16:59 |
priteau | There's also a plan for Blazar to work with Nova + Ironic, which we're planning to work on during this release cycle. | 16:59 |
priteau | tzumainn: You are specifically interested in Ironic standalone? | 17:00 |
tzumainn | priteau, both | 17:00 |
tzumainn | is there a spec for the nova + ironic work? | 17:00 |
priteau | Not yet but I'll let you know when it's up for review. | 17:00 |
diurnalist | I'm also interested in the Nova + Ironic spec | 17:01 |
diurnalist | I was pretty confused by this note in the summit notes: | 17:01 |
diurnalist | > #. Blazar creates an allocation in Ironic (not Placement) with the candidate node matching previously picked node and allocation UUID matching the reservation UUID. | 17:01 |
*** stephenfin has joined #openstack-meeting-alt | 17:01 | |
diurnalist | so I'm interested in hearing how Placement will fit in to the Nova + Ironic use-case. | 17:02 |
*** liuyulong has quit IRC | 17:03 | |
diurnalist | also, this note: "To avoid partial allocations, Placement could introduce new API to consume the whole resource provider." - pretty sure this is what resource classes are used for, no? you have CUSTOM_BAREMETAL=1, which can only be 0 or 1, so the node is either 100% or 0% allocated. but I know we're well over time now... | 17:04 |
diurnalist | perhaps something for the mailing list. | 17:04 |
priteau | diurnalist: I haven't looked in detail yet at the proposed standalone workflow. I don't really see how placement can fit with selecting nodes for advance reservations. | 17:04 |
priteau | Let's end here for today. Thanks everyone for joining! | 17:05 |
priteau | #endmeeting | 17:05 |
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 17:05 | |
openstack | Meeting ended Thu May 9 17:05:30 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:05 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/blazar/2019/blazar.2019-05-09-16.01.html | 17:05 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/blazar/2019/blazar.2019-05-09-16.01.txt | 17:05 |
openstack | Log: http://eavesdrop.openstack.org/meetings/blazar/2019/blazar.2019-05-09-16.01.log.html | 17:05 |
priteau | Feel free to join #openstack-blazar if you want to discuss further. | 17:05 |
*** macza has joined #openstack-meeting-alt | 17:12 | |
*** jakecoll has quit IRC | 17:14 | |
*** priteau has quit IRC | 17:16 | |
*** bhavikdbavishi has quit IRC | 17:19 | |
*** yamamoto has joined #openstack-meeting-alt | 17:22 | |
*** diurnalist has quit IRC | 17:29 | |
*** ijw has quit IRC | 17:37 | |
*** hongbin has quit IRC | 17:54 | |
*** ijw has joined #openstack-meeting-alt | 18:00 | |
*** jtomasek has quit IRC | 18:07 | |
*** yamamoto has quit IRC | 18:11 | |
*** jtomasek has joined #openstack-meeting-alt | 18:13 | |
*** jtomasek has quit IRC | 18:16 | |
*** ttsiouts has quit IRC | 18:25 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 18:34 | |
*** igordc has joined #openstack-meeting-alt | 18:35 | |
*** jaypipes has quit IRC | 18:37 | |
*** jamesmcarthur has quit IRC | 18:38 | |
*** ttsiouts has joined #openstack-meeting-alt | 18:40 | |
*** igordc has quit IRC | 18:44 | |
*** ttsiouts has quit IRC | 18:45 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 18:47 | |
*** igordc has joined #openstack-meeting-alt | 18:47 | |
*** yamamoto has joined #openstack-meeting-alt | 18:51 | |
*** iyamahat has joined #openstack-meeting-alt | 18:51 | |
*** yamamoto has quit IRC | 19:02 | |
*** ttsiouts has joined #openstack-meeting-alt | 19:11 | |
*** carloss has left #openstack-meeting-alt | 19:16 | |
*** jamesmcarthur has quit IRC | 19:17 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 19:31 | |
*** jamesmcarthur has quit IRC | 19:39 | |
*** hongbin has joined #openstack-meeting-alt | 19:44 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 19:50 | |
*** vhari has quit IRC | 19:53 | |
*** e0ne has joined #openstack-meeting-alt | 20:09 | |
*** e0ne has quit IRC | 20:10 | |
*** armstrong has joined #openstack-meeting-alt | 20:12 | |
*** ttsiouts has quit IRC | 20:14 | |
*** jamesmcarthur has quit IRC | 20:23 | |
*** ttsiouts has joined #openstack-meeting-alt | 20:29 | |
*** ttsiouts has quit IRC | 20:32 | |
*** ttsiouts has joined #openstack-meeting-alt | 20:33 | |
*** ttsiouts has quit IRC | 20:47 | |
*** turnerg has quit IRC | 20:52 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 20:54 | |
*** jamesmcarthur has quit IRC | 21:05 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 21:17 | |
*** jamesmcarthur has quit IRC | 21:21 | |
*** radeks_ has joined #openstack-meeting-alt | 21:28 | |
*** ttsiouts has joined #openstack-meeting-alt | 21:29 | |
*** radeks has quit IRC | 21:30 | |
*** ttsiouts has quit IRC | 21:34 | |
*** jgrosso has quit IRC | 21:40 | |
*** lbragstad has quit IRC | 21:45 | |
*** diablo_rojo has quit IRC | 21:54 | |
*** ijw has quit IRC | 22:02 | |
*** ijw has joined #openstack-meeting-alt | 22:03 | |
*** hongbin has quit IRC | 22:04 | |
*** hongbin has joined #openstack-meeting-alt | 22:05 | |
*** hongbin has quit IRC | 22:05 | |
*** hongbin has joined #openstack-meeting-alt | 22:06 | |
*** hongbin has quit IRC | 22:06 | |
*** hongbin has joined #openstack-meeting-alt | 22:06 | |
*** ijw has quit IRC | 22:07 | |
*** hongbin has quit IRC | 22:08 | |
*** hongbin has joined #openstack-meeting-alt | 22:08 | |
*** armstrong has quit IRC | 22:21 | |
*** radeks_ has quit IRC | 22:25 | |
*** markmcclain has quit IRC | 22:35 | |
*** markmcclain has joined #openstack-meeting-alt | 22:37 | |
*** raildo has quit IRC | 22:41 | |
*** iyamahat has quit IRC | 22:42 | |
*** ttsiouts has joined #openstack-meeting-alt | 22:46 | |
*** ttsiouts has quit IRC | 22:51 | |
*** slaweq has quit IRC | 22:51 | |
*** munimeha1 has quit IRC | 22:52 | |
*** macza has quit IRC | 22:57 | |
*** slaweq has joined #openstack-meeting-alt | 22:58 | |
*** slaweq has quit IRC | 23:03 | |
*** slaweq has joined #openstack-meeting-alt | 23:11 | |
*** slaweq has quit IRC | 23:16 | |
*** ttsiouts has joined #openstack-meeting-alt | 23:17 | |
*** ttsiouts has quit IRC | 23:21 | |
*** whoami-rajat has quit IRC | 23:34 | |
*** jamesmcarthur has joined #openstack-meeting-alt | 23:36 | |
*** lseki has quit IRC | 23:40 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!