21:04:18 #startmeeting reddwarf 21:04:19 Meeting started Tue Jun 4 21:04:18 2013 UTC. The chair is vipul. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:04:20 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:04:22 The meeting name has been set to 'reddwarf' 21:04:31 #topic action items 21:04:37 #link http://eavesdrop.openstack.org/meetings/reddwarf/2013/reddwarf.2013-05-21-21.04.html 21:04:56 I missed this meeting, but looks like we didn't how a whole to actioned 21:05:03 ugh can't type 21:05:07 whole lot actioned 21:05:21 esmute: any update on log archiving rdjenkins? 21:05:31 vipul: Wow that is a seriously show list of action items 21:05:44 *short 21:05:58 grapex: we're all done 21:06:04 :) 21:06:17 Great meeting everyone! 21:06:26 A new record! 21:06:41 vipul: Havent got on it yet :-( 21:06:57 #action esmute to work with SlickNik to get rdjenkins logs archived 21:07:01 hey 21:07:07 did I miss it? 21:07:08 straggler 21:07:09 I guess i will have to give that task back to SlickNik 21:07:31 #topic Next meeting time 21:07:34 #link https://wiki.openstack.org/wiki/Meetings/RedDwarfMeeting 21:07:36 agenda ^^ 21:07:51 Ok, it seems like we want to discuss an alternate meeting time? 21:08:10 so we're solid that we're meeting at 2pdt/4cdt? 21:08:14 WEWT! 21:08:32 are we solid on that time? 21:08:40 I don't remember what times were discussed 21:08:44 are folks ok with this time? 21:08:44 11/1 21:08:48 or we we want to change it 21:08:50 Originally we wanted to move the time to earlier 21:08:54 For everyone in CDT 21:08:55 1/3 21:08:56 yeah 21:08:59 that's what i want to know, there was some confusion 21:09:00 that would be nice 21:09:03 yeah I think juice had it 11/1 21:09:04 but the issue is hub_cap has conflicts with that an OpenStack meetings 21:09:04 heyo 21:09:13 oh 21:09:18 who's hub_cap? 21:09:19 #link https://wiki.openstack.org/wiki/Meetings 21:09:21 imma create a doodle 21:09:27 that's about the only time slots that would realistically work 21:09:27 http://www.doodle.com/ 21:09:38 do we wanna pick another day? 21:09:40 #action hub_cap to create a doodle for meeting times (http://www.doodle.com/) 21:09:44 yes vipul 21:09:48 ill doodle up some times 21:09:54 nice 21:10:01 doodle away 21:10:29 #topic Backup Validation 21:10:34 we have stand ups at 10:30 pst 21:10:43 ok juice i wont put it around that time 21:10:45 backup validation did we kill this one earlier 21:10:52 I'm fine with 5am/7am 21:10:57 :) 21:11:04 you might be the only one here :) 21:11:13 backup validation - store disk util in reddwarf 21:11:32 validate volume size is equal to or greater than that value stored in the backup db 21:11:36 juice: do we store zipped size currently or acutal? 21:11:39 …on instance create 21:11:52 we will need to store actual size 21:12:04 not sure what value storing zipped size gets us 21:12:09 ok i thought there was something in metadata about the size today 21:12:29 right we need to store actual 21:12:29 vipul: ive updated the wiki plz refresh at your convienence 21:12:35 there probably is but we will capture the actual size separately 21:12:39 hub_cap: done 21:12:45 <3<3 21:12:49 juice: ok sounds good 21:13:03 so do we consider the backups-api patch good to merge then? 21:13:08 since this shouldn't hold that up 21:13:12 its' already merged 21:13:17 this is an enhancement 21:13:19 w00t.. even better 21:13:27 a new blue print created by mr hub_cap 21:13:38 ok cool 21:13:39 moving on then 21:13:44 the backup api documentation should be merged 21:13:54 #topic Notifications - Exists event 21:13:56 I don't see a reason throwing out the bathwater with the baby 21:14:12 juice: you want to discuss updates on this one? 21:14:25 :D im the blueprint ghostwriter 21:14:42 yes - just want to clarify on the previous topic that the documentation as is 21:14:50 is good enough for now 21:15:07 and when we add the validation we will put a note about the error in the documentation 21:15:19 juice: yes, that is the plan 21:15:27 thanks robertmyers 21:15:31 so onto exists 21:15:45 We are working on getting this merged 21:15:57 I had a few good comments from grapex and cp16net 21:16:03 those are in the latest patch set 21:16:09 hub_cap: refresh agenda when you get a chance 21:16:21 we are doing some testing here in our pre-prod env 21:16:22 coo 21:16:53 not sure if there are any outstanding questions/feedback on that one 21:17:20 anyone? anyone? (in my ferris bueller teacher voice) 21:17:45 ok looks like we're good on that 21:17:50 #topic Heat Update 21:17:55 hey thats me 21:17:58 go for it 21:18:10 so heat will be a very nice add, from what i gather 21:18:33 im currently hacking away at a custom template for deb based installs that will set passwd and eventaully install guest 21:18:56 cdb-scd 21:18:58 it was a bit rough to get started but i understand how heat works now 21:19:01 im out, peace 21:19:10 l8r 21:19:24 grapex: confused 21:19:31 bai 21:19:32 and heat is in a place that we can use it now, it pretty good as it stands 21:19:41 vipul: grapex likes saying random shit 21:20:00 vipul: Hai guys my Mac is stuttering like the '80's Cinemax mascot 21:20:02 lol i figured as much 21:20:04 so thats all i have now. ill have a template for real in a day or 2 21:20:16 grapex: max headroom? 21:20:18 hub_cap: so does that make our image minimal? 21:20:32 vipul: it makes us able to run any uec image 21:20:41 so the imagebuilder stuff becomes a moot point i think 21:20:54 we may want to still do that since install on the fly will slow boot 21:20:59 i might be speaking too soon since im using the imagebuilder's image to upload 21:21:11 well it downloads a uec image vipul 21:21:25 so we can just do the same thing it does, cache 1x and use 21:21:37 but again im not 100% sure yet 21:21:39 more to come in the future 21:21:51 :- 21:21:53 :-D 21:21:59 and the template can sub out the mysql version / package too 21:22:00 we shall wait 21:22:06 "I'll be done in 2 days... or maybe not at all" 21:22:07 so it should be pretty extensible 21:22:13 imsplitbit: GOTO hell 21:22:15 :) 21:22:16 lol 21:22:30 that is all 21:22:50 alright 21:22:57 #topic API Validation Update 21:23:02 juice: is this you? 21:23:12 little progress on this 21:23:21 have all the schema's defined 21:23:32 working on the code to slide it in the pipeline as a filter 21:23:35 so we're going with json-schema to validate? 21:23:41 yes sir 21:23:44 nice 21:23:47 noice 21:23:51 pass in a dict of the request and it validates 21:24:00 I have tests for the schemas and responses 21:24:18 it's pretty tight framework but then it takes a bit of practice to get the schema just right 21:24:21 are we putting them in the reddwarf project? 21:24:41 cuz i think we had xsds that were in reddwarf-int? 21:24:45 it has a few limitations/quirks but it follows the json schema spec quite closely 21:24:58 we should probably put them in reddwarf proper 21:25:02 there are currently in reddwarf 21:25:22 what does putting them in reddwarf-integration buy us? 21:25:37 nothing.. it makes it harder to manage 21:25:41 or rather, why would we put them there? to generate documentation or something 21:25:47 i think i saw schemas there 21:25:58 reddwarf proper 21:26:00 anyone want to elaborate on this comment? "Many groups are doing this now" 21:26:03 period 21:26:08 yes vipul i wrote it 21:26:18 there was talk about 2 projects adopting things 21:26:24 some are doing json schema 21:26:27 but some others are using wsme 21:26:28 hub_cap was trying to peer pressure me :) 21:26:34 :D 21:26:38 smoke it kid everyone is doing it :) 21:26:42 so eventually there will be some oslo stuff 21:26:46 juice: ok! 21:27:03 but we might end up moving away from json schema is what it seems, possibly 21:27:07 since pecan/wsme can do it 21:27:13 but thatll require a complete wsgi rewrite 21:27:16 so we're monving to pecan/wsme? 21:27:17 so we shou move forward for now 21:27:18 pecan? 21:27:21 wsme? 21:27:30 juice: pecans fall from the wsme tree 21:27:38 oh yes - of course 21:27:54 #link https://pypi.python.org/pypi/WSME 21:28:01 but id say we wait for those 21:28:03 is there a mandate? 21:28:09 #link https://pypi.python.org/pypi/pecan/0.3.0 21:28:13 no but thre is a movement 21:28:25 we are the _only_ project using oslo.wsgi 21:28:31 this seems small enough of an effort for now, I don't see any reason to hold up and wait 21:28:32 no way.. 21:28:33 nova? 21:28:38 so it never caught on... and everyone is doing it their own way 21:28:40 nope vipul 21:28:46 markmc mentioned that 21:28:54 and it's more or less orthogonal to the the rest of the project being a filter 21:29:03 ya juice keep moving w/ it 21:29:07 we need good validation 21:29:18 so much for being good citizens and using oslo 21:29:19 just saying we shold watch what other groups are doing too 21:29:25 heh right vipul? 21:29:52 is pecan support going ot be in oslo then? 21:30:01 probably eventually 21:30:01 or we just ditch oslo for the web layer? 21:30:14 i honestly dont know at this point 21:30:16 we wait 21:30:21 oslo.wsgi works for us 21:30:29 ill hack it up when the time comes 21:30:39 k 21:30:59 #topic Validation of API input in Services vs Models 21:31:43 what's this one about? 21:31:48 I was going to ask the same 21:31:51 is this about the database name validation that merged last week? 21:31:56 related to validating the db-user ? 21:31:59 it merged? 21:32:02 i put it on b4 it had merged 21:32:03 i think so 21:32:07 but yes it was about esmute's stuff 21:32:17 i didnt want it to sit for more time 21:32:20 do we have a guideline on these things? 21:32:25 you +2'ed hub_cap 21:32:34 I have a related question when we are done with this 21:32:42 o/ 21:32:43 well good esmute! 21:32:47 haha 21:32:56 o? 21:32:57 so some things that touch the db make sense to be validated at the db layer 21:33:06 since the underlying db may be different and there may be different rules 21:33:24 example? 21:33:26 right, we may not have the same rules for, for example usernames, should we switch db flavors 21:33:36 database name is the example 21:33:40 sorry db-user name 21:33:45 oh i was thnking the infra db 21:33:49 maybe some future impl doesn't allow db names that start with underscores 21:33:49 and was confused 21:34:07 im good now 21:34:10 yep what datsun180b said 21:34:31 jsonschema can do pattern validation 21:34:42 or regex validation 21:34:42 on the other hand, maybe our api has restrictions that aren't forbidden in the impl 21:34:56 datsun180b: which is definitely the case.. since we have a backdoor 21:35:00 exactly 21:35:00 yes datsun180b both of your cases are valid 21:35:39 i think esmute mentioned a use case where not-validating and creating a user through the backdoor breaks other apis 21:35:51 i know the case 21:35:58 me too :) 21:36:02 me too 21:36:14 create a db that fails the validation and then try to list dbs, and you'll get 400s for a simple db list 21:36:14 please enlighten us :D 21:36:21 so validation is still done in the models.. at least for DB stuff 21:36:52 400 being "bad request" 21:37:09 so what was done was to have teh validation in not all the db requests.... 21:37:14 for example listing DBs. 21:37:16 datsun180b: but that should be fixed with esmute's patch? (list dbs at least) 21:37:21 ideally 21:38:05 esmute: but doesn't this still break like the grant api? 21:38:13 you can list the users, but you can't do much more with them 21:38:24 we should pass this off for now and think about it 21:38:28 oh was that not part of the changes? 21:38:33 you can list DB with bogus names that were created via the backdoor (root user) 21:38:34 cuz this is gonna bite us for _every_ type of db in the future 21:38:43 but you cannot delete them via the api 21:38:43 we'd better bite first 21:39:14 because when you try to deletethem, the validation code will prevent you 21:39:14 datsun180b: exactly! 21:39:34 i think our primary concern with validation is to fight exploitation, and our second is making sure that our resources can be referred to properly and uniquely with our url schema 21:39:46 just so that we are all in the same page, we are talking about https://review.openstack.org/#/c/28850/ 21:40:05 esmute: i didnt +2 that! ;) 21:40:26 oops. Sorry hub_cap.. I mistook you with grapex 21:40:38 esmute: grapex is my better half 21:40:41 oh i don't think i approved that one either, i wanted tests 21:40:50 esmute: we all do that 21:40:54 they're basically twins 21:40:54 there are tests there 21:41:00 datsun180b: so what about just url-encoding when we do a list? i think we need to rethink the validation 21:41:11 i hadn't seen them since, my last comment was against patch 2 21:41:11 imsplitbit: im danny devito, and grapex is ahhhnald 21:41:17 Guys am I hearing I didn't minus 2 enough? I tried my best! 21:41:26 what i can do is add a new test that makes sure it skips the validation when it should 21:41:48 esmute: i think a test needs to create a bad instance with a backdoor before all of the other tests run 21:41:58 esmute: if none of them are interrupted we don't have a problem 21:42:01 hub_cap: Because we all know recessive genes are worse somehow. :) 21:42:08 HA 21:42:29 i mean 'db' and probably also 'user' instead of 'instance' 21:42:58 datsun180b: Ok. we need to have the agent mocked for that , no? 21:43:24 yeah, that test can be mocked pretty easily 21:43:32 shall we move this convo and move on? seems like we know we can write a test and keep vigilant of it 21:43:40 #agree 21:43:44 we need to discuss this more 21:43:50 #agree w #agree 21:43:51 ok 21:44:00 #topic Create Schema vs Database 21:44:14 huh? whats this? IR confused 21:44:15 something to consider w/future version of our API 21:44:27 ohhhhhhh u mean the api 21:44:33 not all database types refer to database the way mysql does 21:44:41 like the name SchemaController? 21:44:54 i think vipul means from the customer facing perspective, ya? 21:44:58 is that what it's really named? 21:45:02 hub_cap: yes, customer facing api 21:45:03 and the fact that it doesn't really create schemas so much as databases 21:45:11 instances/id/schema 21:45:25 then again i'm no dba 21:45:54 well specifically oracle.. it's referred to as schema 21:45:55 so heres the deal. most people i know who _arent_ dbas or who dont work w/ krow say database 21:46:18 :-) 21:46:19 http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5004.htm 21:46:21 and when you create one in mysql in particular, "create database" is the way to go 21:46:44 i know how much as anecdotal evidence helps an argument 21:46:51 heh datsun180b 21:46:52 http://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_6014.htm 21:46:54 :D 21:46:57 oddly enough the mysql workbench used to refer to creating database as creating schema 21:47:20 Use the CREATE SCHEMA statement to create multiple tables and views and perform multiple grants in your own schema in a single transaction. 21:47:31 well what would you know, all you have over me is extensive experience working with databases...oh 21:47:33 will we be doing that? if so we shoudl use schema stuff 21:48:24 imho we are doing database stuff, not schema stuff 21:48:26 Yea i am not sure what the right way is... just got some feedback that this was confusing to some 21:48:27 also mysql information_schema refers to dbs as schema 21:48:41 right, performance_schema, etc 21:48:46 exactly 21:48:51 http://docs.oracle.com/cd/B19306_01/server.102/b14231/create.htm#i1008985 21:49:27 my whole point is since i have no formal idea of what i'm talking about so i vote however imsplitbit does 21:49:28 but then again im totally _not_ a dba 21:49:35 datsun180b: #agree 21:49:36 because i defer to him 21:49:58 and maybe we route both urls to the same thing :D 21:50:12 but something to consider for v2 or v3 or whenever 21:50:46 vipul: id prefer not to, since in things like oracle create schema is drastically different than create database 21:50:56 im not saying we _dont_ do schema stuff 21:51:10 id prefer to not call it create schema if we arent doing anthing more than createing a database tho 21:51:20 fair enough 21:51:25 maybe it's an additional set of calls 21:51:31 that really creates a schema instead 21:51:37 hub_cap: this is a much bigger discussion 21:51:45 yes it is imsplitbit 21:51:50 fwiw, mysql docs say 21:51:50 CREATE DATABASE creates a database with the given name. To use this statement, you need the CREATE privilege for the database. CREATE SCHEMA is a synonym for CREATE DATABASE as of MySQL 5.0.2. 21:52:02 but again thats _just_ for mysql :) 21:52:09 hub_cap: thats because mysql was designed to be simplified 21:52:16 a schema is a database 21:52:20 but it is beyond that 21:52:22 maybe the distinction should be left to the impl and it just happens to be that mysql does that for us 21:52:25 it is a definition of constraints 21:52:38 sure but its something our users need to understand 21:52:43 in the mysql context it means its a folder where table files get put 21:52:51 right 21:52:58 i'd like to hope the understand the difference about the time they pick the impl they're using 21:53:10 datsun180b: simple is better :) 21:53:14 schema is more "cross platform" :-) 21:53:17 vipul: see the can u opened ;) 21:53:26 is it imsplitbit? 21:53:26 most DBMS' refer to them as schema 21:53:46 hub_cap: yea we should probably table this since it's not going to be decided here 21:53:54 imsplitbit: def 21:53:56 well the ones that require you to wear your big boy pants 21:53:57 vipul: double def 21:54:03 imsplitbit: HAH 21:54:15 select count(*) from nice_things where we_can_have_one = 1; 21:54:17 mysql is kinda "my first dbms" 21:54:22 0 rows returned 21:54:23 imsplitbit: i've heard it be called schema in oracle for the most part 21:54:44 wait see i'm no dba, it would return one row, count = 0 21:55:14 vipul: this is interesting and non authoritative 21:55:14 ok onto the next thing? 21:55:14 Now that we have a database we can create schemas inside the database. A schema is defined as a user that owns data such as tables, views, indexes, and so forth. If a user has no data of their own and just connects and queries for information then they are not considered a schema. This is the difference between a schema and a user. Also you may be familiar with other database systems and will notice that what Oracle calls a schema the other 21:55:25 yes plz :) 21:55:34 is there a #motion-to-table 21:55:40 #move-on 21:55:41 #tabled 21:55:45 #help 21:55:52 #topic Incubation Status 21:55:54 #plzgodhelp 21:55:58 lol 21:56:02 ummmmmmmm really? 21:56:02 this one's for you hub_cap 21:56:09 ok we are um, incubated 21:56:11 done 21:56:11 hub_cap i think you're hashtagging by mistake 21:56:14 what does it mean?? 21:56:21 #doublerainbow 21:56:38 do we have an update on when we move to openstack github 21:56:49 or when we can leverage openstack-ci for gate? 21:56:50 yes i know exactly when we move 21:56:55 when we get a name 21:57:01 mark collier is _still_ on vacation 21:57:12 so we wont get any traction till he gets back on that 21:57:25 kk 21:57:49 is there a list of things that we need to do prior to next summit? 21:57:56 or is this process pretty open-ended 21:58:04 really just heat 21:58:08 thats the only thing they want to see done 21:58:12 thats my biggest prio 21:58:35 ok cool 21:58:57 #topic Open Discussion 21:59:09 anything else to discuss? 21:59:28 replication api ideas 21:59:43 can someone give me the quick run down of class type/module and responsibility 21:59:44 I've started spit balling on the replicationa and clustering concepts wiki page 21:59:53 go ahead imsplitbit 21:59:55 imsplitbit: sorry i've been out how's that been going 22:00:05 I'd love to hear from anyone and everyone on what I've got 22:00:11