23:00:49 #startmeeting milk 23:00:50 Meeting started Mon Jan 13 23:00:49 2014 UTC and is due to finish in 60 minutes. The chair is sarob. Information about MeetBot at http://wiki.debian.org/MeetBot. 23:00:51 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 23:00:53 The meeting name has been set to 'milk' 23:01:29 meeting agenda is posted on https://wiki.openstack.org/wiki/Meetings/milk 23:01:59 roll call, who do we have with us today? 23:02:11 Joshua Kolden 23:02:28 Keyvan Fatehi, developer @ DigitalFilm Tree 23:02:40 Guillaume Aubuchon, CTO @ DigitalFilm Tree 23:03:20 okay, lets go 23:03:43 #topic finalize weekly meeting time 3pm PST 2200 UTC on #openstack-meeting 23:04:21 #vote 3pm PST as meeting time 23:06:01 Hmm, suboptmial for me in the middle of the day, I'd prefer beginning of the day, lunch time, or end of day, but I'll make whatever work. 23:06:02 do we have agreement to keep this as the meeting time going forward? 23:06:19 im good with 12pm or 3pm 23:06:35 it's OK for me but my cohort, Jeanre is in south africa and 3pm is a bit late for him, as it's 1 AM his time 23:07:06 so 12pm pst would be 10pm local? 23:07:33 yeah that works for him, he usually signs off at like 1PM 23:08:00 1PM Pacific 23:08:22 okay so we have 3 votes for 12pm pst mondays 23:08:57 #action milk-dev meeting time on #openstack-meeting 12pm PST going forward 23:09:27 SOunds good to me 23:09:27 #topic create mailing list milk-dev@etcusc.org 23:10:02 so this is a google group with thread history here https://groups.google.com/a/etcusc.org/forum/#!forum/milk-dev 23:10:25 any problems with how this is working so far? 23:10:43 i like it, nice job with the Wiki and all this stuff :) 23:11:22 one happy customer, yeah 23:11:47 how about the rest of yous? 23:12:26 I think it will work, I like that it keeps record 23:12:27 Yep, looks good to me. I haven't tried to edit the wiki or anything, but looks like a good way to start at leaste. 23:13:10 ah, right for editing the wiki, you will need to create a launchpad account 23:13:19 painless, free and easy 23:13:43 k 23:13:55 anyone can edit the wiki and history is maintained for booboos 23:14:29 #topic share existing metadata API with group as starting point 23:15:03 who should I work with to get this up on the milk wiki? 23:15:53 Where are we thinking to host the code? Github? 23:15:58 is there any interest iun creating a google+ circle for this group and then do a weekly hangout? 23:16:06 Github 23:16:16 or rather reference the existing API so we can work on the milk api 23:16:33 github roger that 23:16:38 WHat existing API are you talking about? 23:17:11 you offered to start with the metadata api as example 23:17:14 Are you talking about doeye, the metadata APi we were working on? 23:17:19 GAUBUCHON: i see some stubbing of that here, https://wiki.openstack.org/wiki/Milk i imagine this is where we'll fill out more detail pre-writing code? 23:17:35 yup 23:17:58 is the source on github, me cant find it 23:18:10 Yes we can gladly contribute that as well as code for additional functionality. 23:18:19 1 sec ill open it 23:18:26 oh, coolo 23:18:51 we should limit the number of functions supported in the prototype 23:19:04 so we wont want to forklift 23:19:33 https://github.com/DFTi/doe-eye 23:19:33 but im open to what we can get done in the time we have 23:19:35 We are opening that right now with MIT licensing 23:19:49 got it 23:19:56 i need to add a license and ... it is written in rails >.< 23:20:49 im open to what the group wants to build the prototype on 23:21:05 Ah, rails. We talked about python, but I'm ok with rails too. However most client side tools in post are python. 23:21:23 And it is lacking docs, but Jeanre and I should probably try to extract the good out into the Wiki, separated from the rails app 23:21:23 yeah, most are python 23:21:31 okay 23:21:40 yeah lets pull the knowledge / value out of it 23:21:45 and then make a decision later 23:22:04 we can focus on the api functions that we think will be good to start from 23:22:16 structure in milk api 23:22:41 id like to do it in Python, considering i dont use it much 23:22:42 We can prototype and document using something like http://apiary.io/. And simply create a blueprint doc. 23:22:48 we can still use wsme, pecan a python api jump 23:22:50 start 23:23:05 Then server tech isn't important. 23:23:06 right i like that 23:23:59 so we have three basic goals for the prototype 23:24:14 #info Ingesting data from the many changes throughout the media pipeline 23:24:38 #info Metadata from the data stored and the pipeline activities 23:24:57 #info URL/URN marker identification for assets 23:26:08 does doe-eye structure lend to the structure we came up with in dec 23:26:16 i found some docs 23:26:17 https://github.com/DFTi/doe-eye/blob/master/documentation/api.md 23:26:53 Do you guys have good coverage with rspec? We can also use that for docs. 23:27:02 not sure what you guys came up with in December, my reference point is Guillaume's original document that Jeanre and I used for doe-eye 23:27:17 although Jeanre worked on doe-eye while i did node.js stuff on our other projects 23:27:20 #link https://wiki.openstack.org/wiki/Milk#To_build 23:28:12 #link https://wiki.openstack.org/w/images/thumb/7/74/Etcc-design-meeting-17dec2013.jpg/120px-Etcc-design-meeting-17dec2013.jpg 23:28:22 MrJoshua: https://github.com/DFTi/doe-eye/tree/master/spec/requests/api looks pretty specced out at the API level 23:28:36 It has some of the fundamentals 23:28:52 #link https://wiki.openstack.org/w/images/7/74/Etcc-design-meeting-17dec2013.jpg 23:29:38 reviewing 23:30:27 okay we can start here 23:32:16 Yeah, this looks in some ways similar to what we have for C4. We abstract things like 'projects' to 'groups' and make lots of different things (including projects) out of groups, and we differentiate 'assets' from other types of data / documents. 23:32:45 we need to define some actions for the IO and metadata IO APIs 23:33:07 DFT guys take metadata as homework 23:33:29 and joshua? 23:33:44 up to you guys 23:34:02 what do you mean take metadata as homework 23:34:15 we can hold an extended hangout to work out the actions 23:34:56 what actions will the metadata api be capable of 23:35:22 when the metadata client queries 23:37:06 from what i understood from the original docs, vendors are just POSTing data to the API, letting the app know what doc type it is, and then post-processing occurs thereafter to make the potentially unstructured data structured 23:37:45 i know there's a comprehensive listing of the vision behind doe-eye , but i am still trying to figure out if that's what we're seeking to build here, or if milk is a superset or a subset or what 23:39:03 as prototype it should be simple, collaboration project 23:39:25 not trying to improve on doe-eye or other projects 23:39:57 rather build a simple prototype with the three goals in mind 23:41:10 I know that we can contribute some of what we have already done. We can contribute for the NAB prototype without rewriting code that has already been written. 23:41:21 if we can get the prototype working, then in may 2014 we start breaking off into groups like metadata, transport, security, archive 23:41:35 and continue the development work 23:41:55 guabuchon sounds good 23:42:09 that makes sense sarob, i like the idea of the 3 goals 23:42:33 keyvan: excellent 23:42:44 i wrote them in a piratepad for my own benefit 23:42:48 http://piratepad.net/z9DdDDotma 23:43:04 as i know these 3 are like bullet points with many subpoints and discussions that can happen beneath them 23:43:16 sure 23:43:17 i know we wont get to them now, but i like the idea of these as the starting point 23:43:40 now the 1st goal, ingesting data etc , is definitely what we were trying to do... any vendor could send us any data 23:43:59 yup 23:44:04 then metadata and pipeline, to me, sounds like the next step we were to undertake which was to have workers try ot decipher/organize/structure the data 23:44:08 So I guess we need to know what the devision of labor is going to be and how we want to proceed 23:44:15 so we prototype a couple of simple ones 23:44:16 and the URL/URN sounds like the basic request of giving everything a UUID 23:44:31 It seems to be where talking mostly about a database gateway, with some interface standardization. Perhaps tied in with karma so that it can interface with any vender database? 23:44:36 keyvan: yup you got it 23:44:37 #link https://github.com/InformationIntegrationGroup/Web-Karma 23:45:06 MrJoshua: ya karma is cool, were you at the ETC meeting when they demo'd it ? 23:45:13 So if we break that down we already have basic ingest, we already have code that assigns a UUID, let's put that all together to prototype quickly 23:45:18 nope but i have heard some 23:45:21 yes 23:45:33 MrJoshua: i was the loudmouth at the back being overly appreciative of it :P 23:46:00 haha, it is cool and seems like what we need. 23:46:05 so you guys want to take a stab at the milk wiki 23:46:14 yes 23:46:27 edit freely brother 23:46:41 I have to jet to antoher call, but we will take a hack through the wiki this week. 23:46:48 here's my proposal for uuid: `openssl dgst -sha256 ingestfile` ;) 23:46:50 roger 23:47:28 #topic two additional core reviewers 23:47:29 lol yeah for these insanely large files 23:48:14 sarob (me) as core reviewer, who else wants the responsibility? 23:48:21 to share 23:48:32 i volunteer Gullaume 23:48:35 he accepts 23:48:46 he left early so its his 23:48:48 I'm happy to help, as I we talked about on the phone. 23:48:55 righto 23:49:32 when do you think we get to the writing code part ? im a bit excited to take a break from ruby and node.js and learn pro python with unit tests, etc 23:49:57 #action gaubuchon mrjoshua are nominated to core reviewers 23:50:15 fireworks and popcorn fiies 23:50:23 keyvan: haha. You can just start writing anything you like. 23:50:44 well, i want to know how to write in the style of openstack developers 23:50:48 wellllll, wsme and pecan need to be integrated into milk 23:50:49 One thing, however, is I strongly encourage us to develop in a TDD style. 23:50:56 i feel like a bit of an outsider since i dont prefer python 23:51:11 and i dont know the tools around doing it properly with tests 23:51:23 yeah MrJoshua definitely 23:51:44 #link https://github.com/stackforge/wsme 23:52:09 #link https://github.com/stackforge/pecan 23:52:27 sarob these hash commands are going into some doc that we'll get at the end of the meeting? 23:52:36 yup 23:52:45 meetbot tracks them 23:52:49 cooolo 23:52:57 and thanks i didnt know about wsme, i would have gone and used CherryPy which is the last tool i used; no tests on that particular app either :P ended up porting it to ruby/sinatra 23:53:15 keyvan: you up for checking these out? 23:53:35 yea definitly 23:53:37 they are potential web services creation standards for openstack 23:53:54 the ceilometer project worked on them 23:54:23 you could whack the code and post a patch with dummy para 23:54:42 would get you used to the gerrit review process 23:55:02 #link http://docs.openstack.org/training-guides/content/operator-getting-started-lab.html 23:55:21 i wrote this up for the openstack contribution process 23:55:41 ah ok perfect. yeah that owuld be cool to make use of the gerrit thing 23:55:50 #topic timeline 23:56:08 do we have a CI infrastructure? 23:56:18 yes sir, we do 23:56:34 we are using the stackforge subset of openstack ci 23:57:24 #link https://wiki.openstack.org/wiki/InfraTeam 23:57:26 hm so its something im unfamiliar with -- well if i get a project up and going i'll ping the group on Google Groups 23:58:02 ah, wicked 23:58:12 we already have everything setup 23:58:40 milk repo is under the stackforge parent on githb 23:58:44 github 23:59:04 oh yeah i just found it 23:59:22 I need to get up to speed on it too. 23:59:23 so i guess it just needs some general direction, and then some code ... 23:59:26 march 2014 is our target r2 release 23:59:31 MrJoshua: empty repo basically https://github.com/stackforge/milk 00:00:08 so late feb should be bug fixing 00:00:29 empty repo 00:00:29 Yeah thanks, I'll try to get an apiary blueprint started, since I have some experance there. It can connect right into the github repo. 00:00:31 right 00:00:41 okay i have to wind down 00:00:50 lets continue on mailing list 00:00:59 cool thanks guys, look forward to helping out 00:01:00 Cool. I have to go too. 00:01:01 roger and out people 00:01:07 #endmeeting