Tuesday, 2014-09-09

sbalukoffSorry, was in a meeting.00:01
sbalukoffPut Barbican on the left and Keystone on the right.00:01
*** jroyall has joined #openstack-lbaas00:01
*** jroyall is now known as joeroyall00:01
sbalukoff(We're not reading manga here, eh. ;) )00:02
*** mlavalle has quit IRC00:05
*** sbfox1 has quit IRC00:08
*** xgerman has quit IRC00:14
rm_youpff00:24
*** vivek-ebay has joined #openstack-lbaas00:24
rm_yousbalukoff: hadn't thought of that :P00:24
rm_youthough I'm probably going to redo the whole thing as a websequencediagram00:24
sbalukoffdougwig: In light of the non-support of eventlet in python 3, should we kill this: https://review.openstack.org/#/c/118827/00:29
dougwigcommented.00:32
*** vivek-ebay has quit IRC00:53
*** joeroyall has left #openstack-lbaas00:55
sbalukoffAnyone interested, feel free to edit as you see fit:  https://wiki.openstack.org/wiki/Octavia/Non-arbitrary_Decisions#What_should_we_call_the_back-end_VM_.2F_container_.2F_machine_.2F_appliance_.2F_thingy.3F01:07
*** vivek-ebay has joined #openstack-lbaas01:37
sbalukoffThis is awesome, and I think we should use it:  https://storyboard.openstack.org/#!/project/list01:43
sbalukoff(Just throwing that out there.)01:43
*** sbfox has joined #openstack-lbaas01:52
*** mestery has quit IRC01:57
*** mestery_ has joined #openstack-lbaas01:57
*** sbalukoff has quit IRC02:10
*** sbfox has quit IRC02:12
*** openstackgerrit has quit IRC02:33
*** openstack has joined #openstack-lbaas03:41
*** sendak.freenode.net sets mode: +ns 03:41
*** sendak.freenode.net sets mode: -o openstack03:46
-sendak.freenode.net- *** Notice -- TS for #openstack-lbaas changed from 1410234100 to 140302124403:46
*** sendak.freenode.net sets mode: +ct-s 03:46
*** vivek-eb_ has joined #openstack-lbaas03:46
*** enikanorov__ has joined #openstack-lbaas03:46
*** mestery_ has joined #openstack-lbaas03:46
*** ptoohill has joined #openstack-lbaas03:46
*** nealph has joined #openstack-lbaas03:46
*** RaginBajin has joined #openstack-lbaas03:46
*** woodster_ has joined #openstack-lbaas03:46
*** rm_work has joined #openstack-lbaas03:46
*** amotoki has joined #openstack-lbaas03:46
*** masteinhauser has joined #openstack-lbaas03:46
*** enikanorov has joined #openstack-lbaas03:46
*** dkehnx1 has joined #openstack-lbaas03:46
*** dougwig has joined #openstack-lbaas03:46
*** jkoelker has joined #openstack-lbaas03:46
*** ctracey has joined #openstack-lbaas03:46
*** pckizer has joined #openstack-lbaas03:46
*** redrobot has joined #openstack-lbaas03:46
*** TrevorV has joined #openstack-lbaas03:46
*** rm_you has joined #openstack-lbaas03:46
*** johnsom_ has joined #openstack-lbaas03:46
*** blogan has joined #openstack-lbaas03:46
*** a2hill has joined #openstack-lbaas03:46
*** sendak.freenode.net changes topic to "https://wiki.openstack.org/wiki/Neutron/LBaaS"03:46
*** sbfox has joined #openstack-lbaas03:48
*** woodster_ has quit IRC03:55
*** jroyall has joined #openstack-lbaas03:59
*** jroyall is now known as joeroyall03:59
*** rm_work has quit IRC04:03
*** joeroyall has quit IRC04:04
*** HenryG has joined #openstack-lbaas04:07
*** rm_work has joined #openstack-lbaas04:08
*** rm_work is now known as rm_work|away04:08
blogansbalukoff: that storyboard looks much better than launchpad, but i think launchpad will suffice now as that storyboard doesn't look too mature04:17
*** vivek-eb_ has quit IRC04:52
*** vivek-ebay has joined #openstack-lbaas04:53
*** sbalukoff has joined #openstack-lbaas05:24
*** vivek-ebay has quit IRC06:08
*** jroyall has joined #openstack-lbaas06:35
*** jroyall is now known as joeroyall06:35
*** joeroyall has quit IRC07:01
*** dkehnx1 has quit IRC07:36
*** dkehnx1 has joined #openstack-lbaas07:37
*** jschwarz has joined #openstack-lbaas07:57
*** sbfox has quit IRC08:21
*** sbalukoff has quit IRC08:23
*** ptoohill-oo has joined #openstack-lbaas09:15
*** ptoohill has quit IRC09:15
*** openstackgerrit has joined #openstack-lbaas09:20
*** amotoki has quit IRC11:57
*** ptoohill-oo has quit IRC12:31
*** ptoohill has joined #openstack-lbaas12:31
*** mestery_ is now known as mestery12:37
*** mestery has quit IRC13:17
*** mestery has joined #openstack-lbaas13:17
*** crc32 has joined #openstack-lbaas13:24
*** mestery has quit IRC13:34
*** vivek-ebay has joined #openstack-lbaas13:34
*** mestery has joined #openstack-lbaas13:34
*** markmcclain has joined #openstack-lbaas13:38
*** vivek-ebay has quit IRC14:07
*** crc32 has quit IRC14:15
*** sbfox has joined #openstack-lbaas14:43
*** TrevorV_ has joined #openstack-lbaas14:52
*** xgerman has joined #openstack-lbaas14:59
*** sbfox has quit IRC15:01
*** markmcclain has quit IRC15:05
*** mlavalle has joined #openstack-lbaas15:11
*** mlavalle has quit IRC15:12
*** mlavalle has joined #openstack-lbaas15:12
dougwigmorning15:17
dougwigis that storyboard what they're talking about replacing launchpad with?15:18
xgermanmorning15:20
xgermanI welcome anything which replaces launchpad15:21
xgermanwonder why they can't get some free Jira like Apache15:21
jschwarzdougwig, ping15:24
jschwarzdougwig, Can you elaborate regarding "Ownership reply"?15:24
dougwigheya.15:24
dougwigjust referring to the nobody question.15:25
jschwarzyes.. I didn't understand the comment15:25
dougwigrunning daemons as a real user is dangerous.  especially one that can get root, like stack.15:25
dougwigfar lesser evil to make that socket writable to all.15:25
jschwarzA discussion worth having15:25
jschwarzThis patch is getting bigger and bigger and it looks like this proposal isn't strong enough and I'll have to drop it eventually15:26
jschwarzdougwig, btw lbaas-agent can get root (runs under stack) and we have no problem with that?15:26
dougwigi think i'm only suggesting a looser chmod on one side, and leaving the nobody on the other?  that should shrink you by one line.15:27
dougwiglbaas-agent isn't internet facing.15:27
*** mikedillion has joined #openstack-lbaas15:27
jschwarzI see your point15:28
dougwigopenstack definitely seems to subscribe to the soft and gooey center model of security, but haproxy isn't in the center.  neutron-server, lbaas-agent, are.15:28
jschwarzI gotta get home which will take me about an hour... will you be here then?15:29
dougwigyes.15:29
jschwarzdougwig, thanks :)15:29
dougwigi'm just waking up.  :)15:29
jschwarz18:29:4315:30
jschwarz:p15:30
jschwarzsee you guys later15:30
*** jschwarz has quit IRC15:30
*** vivek-ebay has joined #openstack-lbaas15:38
*** vivek-ebay has quit IRC15:41
*** woodster_ has joined #openstack-lbaas15:41
*** RaginBajin has quit IRC15:44
*** jroyall has joined #openstack-lbaas15:49
*** jroyall is now known as joeroyall15:49
*** RaginBajin has joined #openstack-lbaas15:56
*** RaginBajin has joined #openstack-lbaas15:57
xgermandougwig I gave blogan a +2; in case you like to +2 and merge16:18
bloganyes +2!16:20
*** rm_work|away is now known as rm_work16:27
dougwigquick -116:28
bloganno!16:28
dougwigi had to show my love.16:28
blogani dont want your love16:28
blogan-1 bc you diagree with the name "amphora"?16:29
blogandisagree16:29
dougwigoh, someone is fishing for a -2.16:29
johnsom_Wasn't being too picky on the TLS stuff as I assumed when it merged they would adjust as needed16:29
bloganjohnsom_: definitely, kind of just followed what neutron lbaas had, but I don't expect we'll be implementing that at first16:30
dougwigquick question on why we're carrying forward the TERMINATED_HTTPS nonsense?  can't we do https + cert here, since we're not carrying already shipped neutron baggage?16:30
*** markmcclain has joined #openstack-lbaas16:30
bloganoh yeah yall wanted just HTTP protocol and then TCP for HTTPS passhtrough, and then HTTPS would be termination?16:30
bloganobviously TCP not only for https passthrough16:31
dougwigright.16:31
dougwigi'd be fine with the review with just nuking that one line.16:31
*** markmcclain has quit IRC16:32
xgermanlet's keep HTTPS -- there is some special stuff we can do as opposed to TCP16:32
blogani still prefer just having the HTTPS protocol and if a user wants to terminate then they provide the tls_container_id16:32
bloganso I'm sensing this will be something we won't have a consensus on right away16:33
*** sbfox has joined #openstack-lbaas16:33
dougwigxgerman: we're not talking about removing it.  just using TCP when it's really a complete pass-through, and using HTTPS when we can do the special stuff.  or i'm fine with blogan's, where it's a pass-through until a cert is defined.16:33
*** markmcclain has joined #openstack-lbaas16:33
*** jorgem has joined #openstack-lbaas16:34
bloganso since I think we all agree we don't like having the TERMINATED_HTTPS protocol, I will remove it and that leaves TCP, HTTP, and HTTPS which are the protocols we need to accomplish what everyone wants16:34
dougwig+116:35
*** jschwarz has joined #openstack-lbaas16:35
jschwarzhola16:35
dougwigjschwarz: heya16:35
*** crc32 has joined #openstack-lbaas16:36
jschwarzdougwig, at the end of the day the issue with the permissions is the least of my worries here16:36
xgermanblogan +116:36
jschwarzdougwig, that whole loop-around because the haproxy don't want to write straight to disk is complicating my code16:37
dougwigjschwarz: i went through the rest, and will +1.  also note that the stats file uses a 666 file to avoid the nobody issue.16:37
jschwarzdougwig, I don't think this is the right solution here16:37
openstackgerritBrandon Logan proposed a change to stackforge/octavia: Initial migration for database structure  https://review.openstack.org/11467116:37
jschwarzbrb16:37
dougwigjschwarz: it's going to be tough getting past the cores until kilo, no matter what.16:38
blogandougwig, xgerman: if yall could +2 again and +A it if you feel it is complete enough16:38
dougwigT minus 22 minutes until hipster overload death.16:38
jschwarzdougwig, I lost hope getting anything before K starts anyway16:39
jschwarzdougwig, Another solution could be implemented in the deployment-tool level, ie. packstack will change the appropriate syslog settings so that will dump to file instead16:40
dougwigor forward to a remote.  that is likely cleaner.16:41
jschwarzThis design is better 'design-wise' but harder to implement16:41
jschwarza remote?16:41
dougwigremote syslog.16:42
jschwarzaye16:42
johnsom_Missed the running typo, but fine16:42
jschwarzthe haproxy guys said that a standing solution they normally offer to clients is simply set-up multiple syslog daemons, one for each haproxy, and that pair will dump to files16:43
*** joeroyall has quit IRC16:54
*** jschwarz has quit IRC16:55
*** jschwarz has joined #openstack-lbaas16:58
*** mestery has quit IRC16:59
*** mestery has joined #openstack-lbaas17:00
xgermanblogan should merge, sorry for the delay...17:00
*** vivek-ebay has joined #openstack-lbaas17:07
*** jschwarz has quit IRC17:16
*** jschwarz has joined #openstack-lbaas17:18
openstackgerritTrevor Vardeman proposed a change to stackforge/octavia: Initial creation of db models, modules, and tests  https://review.openstack.org/11671817:20
*** jschwarz has quit IRC17:22
*** sbfox has quit IRC17:42
*** jschwarz has joined #openstack-lbaas17:42
openstackgerritA change was merged to stackforge/octavia: Initial migration for database structure  https://review.openstack.org/11467117:45
*** sbfox has joined #openstack-lbaas17:53
rm_workdougwig / xgerman / sbalukoff: http://goo.gl/x6oBZu18:10
xgermanok18:10
xgermanlgtm -- so the trust stays for the live of the LB?18:12
*** jschwarz_ has joined #openstack-lbaas18:13
*** jschwarz has quit IRC18:16
rm_workthe Trust stays *forever*18:18
rm_workI should actually indicate that18:18
*** jschwarz_ has quit IRC18:20
xgermanyep..18:20
*** crc32 has quit IRC18:20
bloganshould that be something we clean up after the load balancer is deleted?18:24
rm_workhmm18:26
rm_workThe trust itself isn't really linked to the LB, it's just user_account<-->lbaas_account18:27
rm_workso we could only remove it if they had zero LBs left in the system18:27
bloganso if a user has multiple tls terminated load balancers then it wouldn't create another trust?18:27
rm_workor, zero TLS Terminations18:27
bloganone for both?18:27
rm_workyes18:27
rm_workno additional trust18:27
rm_workI actually just fixed that18:28
rm_worksec18:28
bloganwell still possible to do but less straight forward18:28
rm_workyeah the Trust needs to be stored in the DB on the *tenant* not on the LB18:28
rm_workhttp://goo.gl/zzEDqN18:28
xgermanmakes sense18:28
bloganso we'd need anothert able then18:28
rm_workdo we not already store any tenant info?18:29
bloganstore it based on entities18:29
rm_workor is it literally just the tenant_id in the LB table?18:29
bloganyep18:29
rm_workhmm18:29
bloganneutron may have one18:29
rm_workyeah then we'd need an additional table18:29
bloganof course that will then bring up whether it should be a neutron wide table or lbaas only18:30
rm_workhmm18:30
rm_workwell18:30
rm_workif we're planning to spin out, this is another place we'd want to not mix stuff with neutron18:30
blogangood point18:30
rm_workthough if it's in common utils...18:30
rm_workwhich i think it is... blah18:30
bloganlol18:30
bloganyeah18:30
rm_workthat sucks18:30
xgermanshould be neutron wide ideally18:30
bloganwhen we spin out we'll need to just make sure we have everything we need18:30
rm_workwe're putting all the barbican handling into neutron/common18:31
rm_workso we'll just  need to clone it18:31
rm_workwhen we spin out :/18:31
bloganpretty much18:31
rm_worksooo18:31
rm_workfor now the trustID would need to be stored as part of neutron-core18:31
rm_workwhen we spin out we'll need to add a table for it18:31
bloganwell octavia will most likely need a table for it anyway right?18:31
rm_workyes, octavia also18:31
rm_workerr no18:31
rm_workOctavia won't use trusts18:32
rm_workso it does not18:32
bloganoh thats right18:32
*** RaginBajin has quit IRC18:32
*** mestery has quit IRC18:32
rm_workyou confused me for a sec :P18:32
bloganstoring it all under its service account18:32
rm_workyep18:32
xgermanthat's something we might need to revisit18:32
*** mestery has joined #openstack-lbaas18:32
rm_workprobably18:32
bloganim sure when the time comes to implement, it'll be a good discussion18:33
xgermanyep18:33
rm_workbut for now, this is workable, yes?18:33
bloganyeah18:33
xgerman+118:33
rm_workthough, I am about to have a discussion with some keystone folks, once their meeting is over18:33
bloganand then you'll find out its not workable18:33
rm_workheh18:33
rm_workwell they said it "works"18:33
xgermanjust assume it is :-)18:33
blogandid they use air quotes?18:33
rm_workbut they are concerned about the creation of the Trust18:33
rm_workthey would rather the user create the trust before they deal with us18:34
rm_workwhich I agree is better from a security perspective18:34
rm_workbut often security loses to ease of use :P18:34
bloganahh i see, that is definitely soemthing to think about18:34
xgermanthey just need to learn from Facebook - this app Lb canb access your credit card, etc.18:34
xgermanOk?18:34
rm_workyeah, I was thinking about exactly that from the beginning, but opted to go with this model so the user wouldn't need to figure out all the Trust bs18:35
blogandoing something on the behalf of the user is always somethign to steer away if not needed18:35
rm_workyes, we'll see18:35
bloganit would be odd to tell a user that htey first have to tell keystone to set up a trust with their account to this service lbaas account18:35
bloganif it was an admin doing it, it would be fine, but a common user?18:36
xgermanyep, that's just the opposite what Facebook + Google do18:36
bloganand they do no evil18:36
bloganso we'd be on the side of not evil18:36
xgermanexactly --18:37
bloganbut not exactly good18:37
xgermannah, they can always revoke the trust18:38
*** RaginBajin has joined #openstack-lbaas18:55
*** sbalukoff has joined #openstack-lbaas18:57
*** openstackgerrit has quit IRC19:02
*** jroyall has joined #openstack-lbaas19:13
*** jroyall is now known as joeroyall19:13
*** mlavalle has quit IRC19:17
*** mlavalle has joined #openstack-lbaas19:18
*** vivek-ebay has quit IRC19:18
*** vivek-ebay has joined #openstack-lbaas19:19
*** ptoohill-oo has joined #openstack-lbaas19:33
*** ptoohill has quit IRC19:34
*** openstackgerrit has joined #openstack-lbaas19:41
*** vivek-ebay has quit IRC19:43
*** joeroyall has quit IRC19:57
*** mikedillion has quit IRC20:25
*** sbalukoff1 has joined #openstack-lbaas20:34
*** sbalukoff has quit IRC20:35
*** sbalukoff1 is now known as sbalukoff20:38
*** TrevorV_ has quit IRC20:42
*** RaginBajin has quit IRC20:45
rm_workdougwig / sbalukoff / xgerman / blogan: so, how do we feel about giving the user a template (like, a pre-filled JSON body) and requiring them to hit Keystone and set up the Trust, before they create the TLS Termination?20:55
rm_workinstead of setting up the trust ourselves by hijacking their token during the TLS Termination Create operation?20:56
dougwigi feel pretty yuck about that.20:56
rm_workKeystone people feel very strongly about it20:56
rm_workI tend to agree with you though, dougwig20:56
rm_workso it may come down to us ignoring what people on Keystone tell us :/20:56
dougwiguse AWS or backspace as examples of what people expect from load balancer setup.  that's not anywhere in the mix.20:56
*** mestery has quit IRC20:57
rm_worklol, backspace20:57
dougwigwell, it's a correct "keystone is the center of the universe" approach.  it just makes no sense for end users.20:57
*** mestery has joined #openstack-lbaas20:57
dougwigdamn this dumbed down autocorrect.  :)20:57
rm_workof course, neither is sharing a cert/key from an external repo, so20:57
rm_workthis is all new :P20:57
rm_workyeah... so, one thing I am interested in, is whether "having a thing in Barbican" is going to cost the user money20:58
rm_workif that's billable, then we're adding that cost to them20:58
dougwigwell, you trust certs from your registrar, without setting up trusts.  that's got an easier paradigm/flow.  plus, i'll assume that horizon will make it look semi-seamless, like they do with ssh keys and nova.20:58
rm_workbut that's just a random side-thought, not exactly relevant now20:58
dougwigrm_work: good point, though i'd pay money to get auto-renewals on expiration.20:59
rm_workyeah, probably20:59
rm_workdougwig: yes, me too! but we're not quite there yet :)20:59
rm_workthat requires a few more pieces to fall into place20:59
rm_workbut is definitely the end goal I think20:59
rm_worksince Barbican can now get Certs generated by Symantec21:00
sbalukoffI think what the Keystone people want is too jarring / confusing to the user.21:00
rm_worksbalukoff: agree21:00
*** jorgem has quit IRC21:00
rm_workbut the one complication about "just ignoring them" is they said something about the user's key not always allowing Trust Creation21:00
rm_workso if that is common, it's a problem for us21:00
sbalukoffI imagine we'd end up a with a lot of complaints from people who can't figure out how to add their damned TLS key. :P21:01
rm_worksbalukoff: also agree :P21:01
sbalukoffrm_work: If we encounter that, we return an error.21:01
dougwigi sometimes get the feeling that a lot of folks in openstack are assuming more of the world uses orchestration than is reality.21:01
sbalukoffdougwig: Indeed!21:02
dougwigbecause so many of these integration points are being punted away with heavy-weight alternatives that pretty much *have* to be scripted to make any sense.21:02
sbalukoffAlso folks:  Right now there's not a whole lot on the agenda for tomorrow, though I think we could probably spend most of the meeting just debating using Storyboard for better blueprint / project management.21:02
sbalukoffNevertheless, if y'all have things you want to add, feel free to add them to the agenda.21:02
sbalukoff(I don't think it's worthwhile to send a message reminding people about tomorrow's meeting to the mailing list--  I've done that for several weeks now and it hasn't really changed attendance. So if they haven't figured it out by now...)21:03
dougwigsummon the IRC supporting flock!21:04
sbalukoffdougwig: Er... ?21:04
dougwigthe vote tomorrow.21:04
sbalukoffdougwig: Actually, I've been recruiting heavily among people at Blue Box to attend / start contributing to Octavia. So far, it seems most if not all I've convinced to come are heavily in favor of keeping the meetings in IRC.21:05
dougwigof course they are; it's the clearly superior medium.21:05
sbalukoffI'm also somewhat more amenable to the idea-- to the point where I might just abstain from voting.21:05
dougwig;-)21:05
sbalukoffdougwig: Are you trying to get me to vote against IRC?21:05
dougwigwould i ever troll?21:05
sbalukoffWell, you never use sarcasm.21:06
sbalukoffAnd I'd rather die than use hyperbole.21:06
dougwigboth of which are excellent uses of a text-only medium, too.21:06
sbalukoffHaha21:06
sbalukoffAs a friendly reminder, here's the agenda so far:  https://wiki.openstack.org/wiki/Octavia/Weekly_Meeting_Agenda#Agenda21:07
blogandouwig: backspace == racklink21:16
blogansbalukoff: wait you were convinced to use IRC by your own people? but we couldn't convince you?21:17
sbalukoffblogan: I wasn't convinced to use IRC per se.21:17
sbalukoffBut I'm more on the fence about it.21:17
sbalukoffNow, anyway.21:17
blogani'd like to meet these people that pushed you to be on the fence, do they ahve some kind of mind control?21:18
sbalukoffI forgot my tinfoil hat that day.21:18
sbalukoffAt least one of them will probably be joining us tomorrow.21:18
blogani've got a few questions, api related, but some don't have any real solutions other than adding features to another library21:19
sbalukoffFor the meeting, you mean?21:19
sbalukoffAdd an agenda item!21:19
bloganyaeh21:20
sbalukoffOr are you talking about discussing them here and now in IRC?21:20
blogandoesn't matter, though I think the more people the more likely they ahve experience21:20
bloganyou have any experience with WSME?21:20
sbalukoffNope.21:20
bloganso i guess the one question is that it looks like openstack is drfiting towards a pecan + wsme standard for APIs21:21
bloganjust wanted to make sure everyone is on board with that21:21
bloganalso that we should use jsonschema21:21
sbalukoffCool. Add the agenda item.21:21
blogani have to type?21:22
bloganyuck21:22
sbalukoff...21:22
sbalukoffJust because you're in Texas doesn't mean I don't have ways to smack you.21:22
dougwigi'm on board with pecan/wsme.  i hate json schemas; i think they imply a lazy interface design, and they're ugly.  but i'd live.21:22
blogandougwig: I like it and I don't21:23
dougwigthat's a good summary21:23
bloganthere is some overlap with with WSME can do as well21:23
bloganlol21:23
bloganso if people don't want to use jsonschema, we can use WSME to do some of the things21:24
bloganthough I'm not sure WSME will validate values21:25
bloganstill learning about it21:25
rm_workwell, if you're talking about validation of JSON, take a look at how Barbican does it: https://github.com/openstack/barbican/blob/master/barbican/common/validators.py21:26
bloganive seen it21:26
rm_workwhich method do they use?21:26
rm_workI don't actually know what jsonschema is21:27
bloganjsonschema21:27
rm_workis that a python lib?21:27
rm_workis that what they are using?21:27
bloganand also some customs21:27
rm_workah, yeah it is, I see the import now T_T21:27
dougwigit's a draft RFC for encoding a schema inside json.21:27
rm_workwell, I didn't mind their validator code -- writing it seemed straightforward21:27
rm_workand i believe it works fine21:27
rm_workif I had to vote between that and doing it by hand, I might vote for jsonschema21:28
blogani dont mind using it, or using whatever wsme does as well21:28
dougwigrm_work: until you saw a jsonschema, maybe.21:28
bloganthey're in that link21:28
bloganit makes sense to me21:28
bloganactually very similar to the extension dict in neutron21:29
bloganexcept, it thankfully doesn't also define the resources and methods the plugin should define21:29
blogandougwig: is you're preference to write our own validation?21:29
bloganor just use whatever wsme does and add in what it is lacking21:30
bloganWSME is moving to stackforge so we can definitely make that much better21:30
dougwigmy *preference* is to use a framework like django or rails to automate all this cruft away.  in an openstack context, i'm happy to settle for doing what is the norm within this community.21:31
rm_workdougwig: yeah, I apparently wrote jsonschema validators without even realizing it, and i liked them21:31
blogani know pecan and wsme are becoming the norm, but jsonschema maybe not, though it is being used21:32
bloganobviously barbican is one21:32
bloganthat is21:32
bloganbut they're not using WSME21:32
dougwigit'd be better than no validation or hand validation, for sure.21:32
blogani think improving WSME's validation would be the best way21:33
bloganbut, a stop gap would be to use jsonschema21:33
xgermanI saw barbican using flask..21:37
xgermanor was it uwsgi21:37
xgermanalways confuse things21:37
*** vivek-ebay has joined #openstack-lbaas21:38
xgermanbut we know rmworks opinion to uwsgi :-)21:38
rm_workyeah, uwsgi <_<21:38
rm_workwell, it has problems21:39
rm_workbut21:39
rm_workfor *dev only depolyments* it's not a bad option I guess21:39
dougwigthat's only if it's the edge server.  you can put a reverse proxy in front of it and it works fine.21:40
*** vivek-ebay has quit IRC21:42
rm_workyeah, the uwsgi dev flat out told me it's not intended to ever be exposed directly21:43
xgermanI like uwsgi + flask -- but we should likely follow their standard21:46
rm_workI like Flask a whole lot better than Pecan, but Pecan is the Openstack framework of choice, so I think that's what we're stuck with, for better or for worse21:47
*** vivek-ebay has joined #openstack-lbaas21:50
bloganxgerman: doesn't libra use pecan?21:52
xgermanwell, we use pecan and wsme or something on top21:53
bloganwell right now, for simplicity, im just using wsgiref.simple_server as the server21:53
bloganbut it can be swapped out easily21:54
xgermanbut I like flask better :-)21:54
bloganwell you cant have it!21:54
xgermanwsme also runs on top of flask21:54
bloganwsme runs on top of all the frameworks from what i can tell21:54
bloganlets use falcon21:55
xgermanyeah, wsme seems to bring out the wrost from each framework :-)21:57
xgermanalso we can likely diverge for the REST server on the amp21:58
bloganoh yeah definitely21:59
xgermanjust use them all in some way or another21:59
dougwigjust make it ruby and call it a day.22:00
xgermanRemember Rackspace tried Java22:01
xgermanbut I think you can always use jython22:01
bloganvba22:01
bloganmicrosoft office load balancer22:02
xgermanthat's discontinued. Iron Ruby22:02
sbalukoffWe just went with a simple CGI script as the agent on our devices. Didn't need anything more than that because the back-end API was so simple.22:02
sbalukoff(lighttpd as the webserver)22:02
rm_workooh, let's do that with a php script22:02
sbalukoffIck.22:02
dougwigSOAP with xmlsig.22:02
rm_workhaha22:02
dougwigthat's a thread winner right there.22:03
sbalukoffUgh.22:03
xgermanlet's sue one of those new languages, golang, hack, ...22:03
blogando we have a weekly etherpad to update?22:04
xgermanwe do22:04
xgermansbalukoff??22:04
sbalukoffblogan: Yes, we do22:05
sbalukoffhttps://etherpad.openstack.org/p/octavia-weekly-standup22:05
blogani still have the etherpad, but i'll let you add the template22:05
sbalukoffEr... there was a template on the old one?22:05
bloganwell new week section22:05
sbalukoffAah. Ok.22:06
blogannot exactly a template22:06
sbalukoffSure.22:06
*** vivek-ebay has quit IRC22:14
rm_workhmm, how do I add things to the *Neutron-LBaaS* meeting agenda?22:17
rm_workdo I just talk to dougwig or jorgem?22:17
rm_workor is there a wiki22:17
bloganthere is a wiki, but it hasnt been updated in a while22:18
bloganwell 2 or 3 weeks22:18
rm_workyeah, looked that way22:18
dougwigjust edit the wiki22:18
rm_worknot sure how the last few agendas got set22:18
bloganwasn't much to talk about so it was mostly open discussion22:18
rm_workok, mayhaps I will just do that <_<22:18
bloganbasically there's the obligator "incubator update" then something I probably added just for the hell of it, and then open discussion22:19
rm_workk22:19
dougwigthere's not much to discuss while we're in limbo, and octavia has the focus.22:19
xgermanwe might want to talk design summit again22:19
xgermanand jorgem's election campaign to become PTL of Neutron - kidding...22:20
bloganwhen are those going to be open for registering?22:20
bloganhe woudl lose in a landslide22:20
xgermanthat's before the campaign22:20
bloganhe does play dirty22:20
sbalukoffQuestion for y'all:  With the discussion of pecan / WSME, would you rather see the haproxy amphora agent API proposal I have in mind as stubs in code, or as .rst documentation that will be incorporated into Octavia's official docs?22:20
sbalukoffWaht I mean by that...22:20
dougwigare you trying to get out of actual python coding?22:21
sbalukoffThe former is great if there's an automatic way for documentation to be generated. But I think it's good to have the latter in any case.22:21
dougwig:)22:21
bloganlol22:21
sbalukoffdougwig: If I do the former, it will take me 2-3 x as long, probably. Because, yes, I'm a terrible python coder. :)22:21
blogansbalukoff: is your intention to become a better python coder?22:21
dougwigeither is fine.  i'm sure one of us can transcribe to python quickly.22:22
sbalukoffblogan: Honestly, I'm not sure:  I think there are enough contributors to this project that are light-years ahead of me as far as their python coding ability. Even if I stopped doing anything else for the next three months and just coded python all day I wouldn't catch up to those of y'all who have years of experience in this...22:23
sbalukoffBut I think this (and other Open Source projects) often lack good... I dunno... project management?22:23
sbalukoffI feel like the biggest challenge Octavia has right now isn't my ability to write shitty code, it's that people have a real hard time knowing where to start eating this elephant.22:24
sbalukoffIf that makes sense.22:24
bloganeating elephants makes all the sense to me22:24
sbalukoffI hesistate to call that project management because I think that term carries a lot of baggage.22:24
xgermanI think we need some more architecture/design discipline22:25
bloganits up to you on what you want to do any coding, though I think being the PTL it would help you out in knowing the code22:26
sbalukoffHaha22:26
sbalukoffOf course--  I can read it just fine. It's not too dissimilar from perl or ruby.22:26
xgermanyeah, what I am saying :-)22:26
sbalukoffAnd I try to be thorough on code reviews.22:27
bloganstill I think its beneficial to know how to do it, you could start on it, even if it'd take you 2-3x times longer once it is needed someone could come in and give it a boost22:28
sbalukoffAnyway, maybe it's more a question of:  Am I blocking anyone else's work if I take a while to come up with the haproxy amphora agent API, doing it in code (which, again, I'm hoping will be something from which automatic documentation can be generated, as much as I dislike that term.)22:28
dougwigif you do this one as a doc, because abstract interfaces aren't the best way to learn, you ought to make up for it by writing some unit tests or something.  or else we will all mock you.  yeah, check out that wicked pun.22:28
bloganlol22:28
sbalukoffHaha22:28
sbalukoffRight.22:29
blogansbalukoff = mock.Mock()22:29
sbalukoffBut... not until after I convince you all to go with Storyboard because launchpad sucks.22:29
sbalukoff;)22:29
blogansbalukoff.assertCalledWithAnger()22:29
dougwigabstract interfaces (import abc) + docstrings == a doc, btw.22:29
sbalukoff(though I'm serious about moving to storyboard. It's not the best tool I've seen, but it's way, way better than launchpad.)22:29
xgermanjohnsom says he would have a blueprint by the end of the week for the controller. Biy this wpould eb driver <-> controller?22:29
dougwigwe can replace assertTrue with assertRighteous22:29
xgermanjohnsom_22:30
blogani think the interface is much cleaner, but i'm not sure it really adds any other value22:30
johnsom_I think we need to have .rst specs in the blueprints.  It will help our OpenStack aspirations22:30
sbalukoffdougwig: Right, I just didn't know whether there was an automated way to extract those docstrings and have them populated in the "standard" build documentation.22:30
xgerman+122:30
sbalukoffI guess I could figure that out.22:30
dougwigsbalukoff: indeed there is, though i've never done it.  but the python interface docs are built that way.22:30
johnsom_I said I would have a start on the controller blueprint this week.22:30
bloganthe problem with having specs right now is that we'd need to draft them up, then review the specs, then implement the code, then review the code22:31
xgermanthat's how software dev works22:31
sbalukoffblogan: +122:31
johnsom_Yes, which is a demonstration of the openstack process and we have our act together...22:31
bloganyes the openstack process on projects that are mature22:32
sbalukoffAnd it makes things move approximatly 1/10th the speed while we are still bootstrapping.22:32
bloganwhich i agree we should do once this is in a mature enough state22:32
sbalukoffYep.22:32
xgermanwell, we want to be an OpenStack project so we need to demonstrate their pocesses. What's the alternative? Drinking a bottle of tequial and writing LBaaS?22:33
johnsom_I don't see how we can get to a mature state without a good set of reviewed specifications22:33
xgerman+122:33
dougwigas a group, we spend a *lot* of cycles worrying about being in openstack.  seemingly more than we spend building something that we can put in openstack.  is the current bootstrapping leaving anyone feeling like they don't know what's going on, or that we're not iterating in the right direction?22:33
sbalukoffxgerman: I see what you did there!22:33
sbalukoffdougwig: +122:34
sbalukoffI feel like we're at the "write some goddamn code already" stage in many ways.22:34
johnsom_dougwig: yes, I don't feel like we have a solid direction22:34
sbalukoffdougwig: Yes, that's my impression as well.22:34
sbalukoffI think this could be helped by fleshing out the current blueprints.22:35
*** jroyall has joined #openstack-lbaas22:35
*** jroyall is now known as joeroyall22:35
sbalukoffBut I'd like to move to a better system before we do that (ie. Storyboard)22:35
johnsom_Why do blueprints at all if we aren't going to build specs behind them and review?22:35
sbalukoffAlso, I'm starting to be convinced that storyboard is the "future" of how blueprinting is going to work for OpenStack anyway... so I'd rather get on it now rather than lose a bunch of history in launchpad by doing a transition later.22:36
dougwigjohnsom_: blueprints are a way for the manager to make a schedule/milestones.  specs are to get devs talking before coding.  i'm feeling like things are early enough that the code reviews are simple enough to be the rallying point for discussion.  i take it that you disagree?22:37
sbalukoffjohnsom_: I think blueprints at this stage are a good way to show logical divisions in work domain-- so that people can work independently in many cases without worrying about stepping on each others toes.22:37
bloganplus we agreed that when you pick up a blueprint in launchpad, you'd add more details to it explaining at a high level of how to accomplish the implementation22:38
sbalukoffblogan: Except I don't think people know what details to fill out.22:38
sbalukoffblogan: I think we're going to have to do that for people so they know what the blueprint actually entails.22:38
bloganno they don't, but do people know what details to fill out in a blueprint spec?22:38
xgermanyep, we should establish good processes/docs22:39
sbalukoffWe're eating an elephant, but even knowing how to eat an elephant leg seems overwhelming.22:39
johnsom_blogan: yes, I think that is what I am arguing for, is filling out the specs behind those blueprints.  It will help with the integration points in the code.22:39
sbalukoffAnd again, the launchpad system really doesn't help. :P22:39
bloganno it doesn't, but even the blueprint spec process is pretty new to openstack anyway and so most projects (if any) were not boostrapped by doing specs22:39
sbalukoffthat's true.22:40
bloganspecs being similar to the neutron-specs22:40
sbalukoffThey were mostly bootstrapped by a small group of devs working together who shared a common vision and just sat down and wrote code.22:40
blogannow giving as many details int he description/work items in launchpad section is great, and bring it up in the weekly meetings22:40
sbalukoffI feel like with the v0.5 discussion, hopefully people have enough of that common vision.22:40
johnsom_I have spent some time looking at the blueprints for "mature" projects to get a feel for how they are used.  I am coming up to speed on them and plan to post some specs in the next week or so22:40
dougwigis the issue here that we need specs, or that we need to get the direction of each component less abstract/more defined before some are comfortable running with it?  (and a spec is one way of accomplishing that.)22:41
bloganjohnsom_: specs on blueprints you are doing?22:41
sbalukoffBut I'm discovering with people I'm recruiting to work on this internally, that they still need more specifics than "write controller" to get started even knowing how to break that up into smaller tasks.22:41
blogansbalukoff: oh thats definitely a problem, and really I didn't know exactly what that woudl entail with the controller either, though I knew the controller has the most dependencies22:42
johnsom_blogan: I signed up for base-image and controller22:42
sbalukoffdougwig: I think the problem is that each of these blueprints need better definitions, probably from me or blogan (or others here who have a good sense of how things fit together.)22:42
sbalukoffjohnsom_: I suspect that's too much work for one person.22:42
dougwigjohnsom_: heh, both??  :)22:42
johnsom_base-image is pretty straight forward22:43
sbalukoffConsidering that's probably *most* of the code that will be written. ;)22:43
blogansbalukoff: that was the point of me dong them right now, to get it started and then as things became more clear (or unclear) then more details would be added, blueprints would be split into multiple ones22:43
sbalukoffblogan: Sure, but I think we should probably fill in the bits we can see already. This will help those who don't see those bits yet.22:43
bloganperfectly fine by me22:44
sbalukoff(while being open to additions or deletions from those bits)22:44
johnsom_sbalukoff: we are spinning up a team, so I will have help22:44
sbalukoffOk, again, this could be done using launchpad, but I think storyboard is much better suited to the task.22:44
sbalukoffjohnsom_: Then I'd rather have visibility into that team. Get your team-mates to join here and in the meetings, eh!22:44
blogansbalukoff: im just concerned with the adoption of storyboard, doesn't seem like its being adopted at a high rate22:45
dougwigpushing storyboard before even infra is using it for real seems....   like some serious dislike for launchpad.22:45
xgermanalso to add that johnsom_ is a god22:45
bloganand i don't see any extra added value right now other than it being a cleaner UI22:45
sbalukoffblogan: Full disclosure, I've got the guy who works on refstack (David Lenwell) helping me out on this. :D22:46
xgermanalso we are a big team so we need process22:46
xgerman(we = Octavia people)22:46
johnsom_sbalukoff: you will see that Al has already signed up for some work.22:46
sbalukoffAlso, storyboard is run by the openstack-infra guys.22:46
johnsom_xgerman: you are funny22:46
sbalukoffAnd is much more responsive for bugfixes / feature additions than launchpad. :D22:46
sbalukoffblogan: You can split blueprints into tasks.22:47
sbalukoffAnd you can comment on blueprints22:47
bloganwhat if it is never adopted because openstack has a knack for adopting new technologies?22:47
xgermanwell, let's not argue tools22:47
dougwigwell, sbalukoff is going to have to use those tools for organizing more than the rest of us, so i'm fine with him picking his poison.22:47
bloganslowly22:47
blogani really don't care iether way, just voicing concerns22:48
johnsom_It seems like we have a start in launchpad, maybe we should let storyboard cook a bit and switch on the next round22:49
xgermanmy main concern is that we need some process to get design documents -- we are far too many people to just code and it will work out22:49
sbalukoffGot it. I can't predict the future, but it certainly seems to me that launchpad is obtuse enough, and poorly designed enough, that something like storyboard is likely to supersede it.22:49
bloganadding comments to blueprints is actually a huge feature, if anyone can add comments22:49
bloganthat way we can actually get feedback on the blueprint iteself22:49
sbalukoffblogan: That's the first thing I latched onto.22:50
sbalukoffAlso, when you split a blueprint up into tasks, you can assign those tasks to different people easily.22:50
sbalukoffSo, it helps foster more people contributing in parallel.22:50
sbalukoffEach of those tasks has a status...22:50
sbalukoffSo you can see, pretty quickly, where things are at right now with the blueprint.22:51
sbalukoff(Also, a pretty extensive history)22:51
bloganxgerman: i agree with you there, but I still think too much process in the bootstrapping phase will slow this down to a snail's pace, look how long it took us to agree on the name of the device hosting the load balancer22:51
sbalukoffI don't think launchpad provides any of this. And if it does, then I predict its buried somewhere difficult to find.22:51
sbalukoffxgerman: I don't have a problem with people writing specs especially around APIs and interfaces between components...22:52
sbalukoffI think that's a really good idea.22:52
rm_youoh man, you guys generated like.... half of my visible scrollback, in the 30 minutes i was driving home >_<22:52
sbalukoffBut for other things, sometimes just having some code to discuss can move us forward fastr.22:52
xgermanlol22:52
sbalukoffYou know me: I'm a huge fan of documentation. :)22:52
sbalukoffrm_you: Then our nefarious plan worked!22:52
xgermanwell, we have some fundamental decisions to make which will hunt us for many years -- so slower speed and getting it right is better for me right now22:53
sbalukoffxgerman: I might point out that that's a bit of a false dichotomy. :/22:53
johnsom_I just see broken pages (dashboard), pages that spin loading, and the lack of subscribe (unless I missed it) features....22:54
bloganxgerman: i don't think it will haunt us for many years because we can easily change it since it is just bootstrapping22:54
dougwigwe have warring software methodologies going on here.  spec like mad up front and hope you get it right before coding, or write some code, knowing some will be crap, and iterate.22:54
sbalukoffjohnsom_: What browser are you using? It's all working fine for me (using Chrome on a Mac)22:54
dougwigaka waterfall vs agile22:54
johnsom_Chrome22:55
sbalukoffdougwig: I think we need a happy-medium between both.22:55
sbalukoffIf that makes sense.22:55
bloganagilefall22:55
xgermanI am ok with just coding the diretcoy structure, coding the controller not so much22:55
rm_youIt looked fine to me (also Chrome/OSX) but I wasn't quite sure what the point was22:56
rm_youI guess I need to see Storyboard in action22:56
sbalukoffdougwig: That is to say, if you're not sure how to do something, discuss it, spec it up, etc. But if you know what you want to do, write the code and see what people think.22:56
rm_you(for a project I understand somewhat)22:56
sbalukoffrm_you: Poke around in the refstack project.22:56
johnsom_I definitely see the benefit to the history and easier task assignment.  I'm just not sure how cooked it is and wish it had the subscribe feature.22:57
sbalukoffFWIW, storyboard encourages using specs. :)22:57
xgermansbalukoff, I have a clear idea that I don't want a driver in the controller so I sahould just code it that way?22:57
bloganxgerman: thats been discussed22:57
johnsom_It is also missing the link to specs22:57
bloganxgerman: so not an accurate counter22:57
sbalukoffxgerman: Right, and then we'd shoot you down in the code review. ;)  Especially because we have been discussing this and will be voting on it.22:57
sbalukoffxgerman: So yes, those kinds of decisions are worth having a discussion about before code.22:58
xgermanblogan, I just discussed it because I didn't know I could just write it and didn't need consensus...22:58
sbalukoffxgerman: But if you are better at expressiong your idea through code, I think that's valid.22:58
sbalukoffxgerman: That's where the 'iteration' bit comes in.22:58
johnsom_sbalukoff: where is the spec tie-in with storyboard, I am missing it22:58
xgermanI am just saying we need some spec before we write a ton of code we will -222:59
sbalukoffjohnsom_: Oh, looks like it's usually added just as a first task. So yes, I guess that's up to whoever is running the project. (example: https://storyboard.openstack.org/#!/story/105 )23:00
dougwigxgerman: i'm not sure any of the reviews should be a "ton" of code yet... maybe that's our disconnect?23:00
blogansbalukoff: lol so it really doesn't encourage specs, its whomever is creating the tasks23:00
sbalukoffblogan: Hmmm... David seems to imply that it did, though, like you, I'm not seeing obviously how it does.23:01
dougwigi don't have a problem with specs.  i only don't like them when writing the spec takes as long as writing the code, which is usually true of smaller bites.  nor do i mind a -2 in such a situation.  if you're writing a big thing and not sending any of it to gerrit until the end, then a spec would be a damn fine idea; but i'd also question that approach.23:01
johnsom_sbalukoff: hmm, I was looking on the openstack-infra/storyboard project and it doesn't have that.  In fact, task 1 is missing altogether.23:02
xgermandougwig, agreed. But I also see value in parcticing to write specs with small parts23:02
xgermanespecially since we are settling on new tools and most of us are new to them23:02
xgermanBut I am ok with a spec just being the abstract class IHMO23:03
sbalukoffjohnsom_: Right. So I retract my earlier statement: I'm also somewhat unfamiliar with this tool (though, again, I already like it a lot more than launchpad), and I don't see exactly how it encourages the writing of specs.23:03
*** sbfox has quit IRC23:04
dougwigxgerman: i don't have a problem with that, as long as we're being fluid about it and not just adding a required heavy-weight process just because.  there are things that i'd want to spec out (like the entire controller) just to get my head straight and get some feedback quicker, and there are other things (like the neutron interface), where it'll just be23:04
dougwigeasier to throw some code into gerrit and start getting comments there.23:04
dougwigxgerman: just saw the abstract class comment; right, agreed.23:04
xgermanok, glad. sbalukoff seomtimes makes things sound very one sided so I had to object :-)23:05
sbalukoffxgerman: Objecting with me for the sake of objecting with me gets us nowhere, eh. :P23:06
sbalukoffxgerman: If you're being serious, eh.23:06
dougwigxgerman: i'd actually prefer we use the specs directory in our repo for the occasional spec.  the review process is more formal.23:06
sbalukoffThat's what the specs directory is for!23:07
xgermangreat. I like to see more specs unless for very simple things -- not just the things we "think" we know how it's done23:07
dougwighas this just been 1000 lines of irc violent agreement?  i feel like this started with a nugget of discontent from johnsom_, and we didn't actually change anything.23:07
xgermanit's is human nature to overestimate23:07
sbalukoffdougwig: I used this discontent to push my super-secret agenda to get us all using storyboard instead of launchpad.23:08
xgermandougwig we do a lot of that. Maybe that's why we shoul ditrch irc an d have a permanent google hangout23:08
sbalukoff(In reality, it was appropriate to the topic.)23:08
sbalukoff(Since the launchpad blueprint system is pretty horrible.)23:09
bloganwhile we're at violent agreements, Amphora!23:09
sbalukoffxgerman: HAHA! Oh, man, let's get the holy wars started, eh!23:09
johnsom_Wow23:09
* dougwig cries.23:10
bloganalright I'm out, ill be on later I'm sure23:10
sbalukoffblogan: Too late! I already documented that non-arbitrary arbitrary decision in the wiki, and as we all know, the wiki is immutable.23:10
blogansbalukoff: it's immutable becasue I don't want to touch wiki markup23:10
sbalukoffblogan: Doesn't it have a WYSWIG edtor java app or something attached?23:11
sbalukoffEr...23:11
sbalukoffWYSIWYG23:11
xgermanI can added Wikis. Blogan just tell me what you need and I will hack it there with big fomts and all caps23:12
sbalukoffxgerman: Don't forget to make the text blink.23:13
xgermanyep23:13
dougwigand red.  gotta be red.23:17
sbalukoffOoh! Cycle the colors through all the colors of the rainbow. Rapidly.23:17
sbalukoffDoes anyone here remember Butt Fugly Design?23:17
xgermandisco lights?23:18
johnsom_Can we have it start playing Rick Astley music when the page loads?23:19
sbalukoffI used to love to gawk at that site:  100% W3C standards compliant HTML code, and completely unsuable with nausea-inducing design.23:19
rm_youI remember that this is the website of the professor at my college that *taught the web design course*: http://www.cs.trinity.edu/~thicks/23:19
rm_youI REALLY hoped it was an attempt at humor23:19
rm_youbut I am afraid it wasn't23:19
sbalukoffrm_you: HAHA!23:19
rm_youthem gifs...23:19
rm_youoh man and the javascript dropdown navigation that used frames and could end up in a loop...23:20
sbalukoffAnimated gifs are the future of the internet.23:20
sbalukoffGet with the 1990's, man!23:20
sbalukoffStill better than frontpage.23:21
johnsom_rm_you: The "Dr. Web" is choice!23:21
rm_youuhhh23:21
rm_youso23:21
rm_youview-source:http://www.cs.trinity.edu/~thicks/Main-Pages/Home.html23:21
rm_youit IS frontpage23:21
johnsom_<meta name="GENERATOR" content="Microsoft FrontPage 6.0">23:21
rm_you<meta name="GENERATOR" content="Microsoft FrontPage 6.0">23:22
rm_youyep23:22
rm_youhehe23:22
johnsom_I just had to look!23:22
rm_yousame23:22
sbalukoffHAHA!23:22
xgermanlol23:22
sbalukoffOh man, I had no idea. XD23:22
*** joeroyall has quit IRC23:24
rm_youit's ok, it's hidden, because FRAMES23:24
rm_youbrb23:25
*** mlavalle has quit IRC23:47

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!