17:00:49 <docaedo> #startmeeting app-catalog
17:00:50 <openstack> Meeting started Thu Nov  5 17:00:49 2015 UTC and is due to finish in 60 minutes.  The chair is docaedo. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:54 <openstack> The meeting name has been set to 'app_catalog'
17:01:15 <docaedo> Courtesy ping kfox1111_ kzaitsev_mb kzaitsev_ws reed j^2 ativelkov
17:01:24 <docaedo> #topic rollcall
17:01:26 <drwahl> o/
17:01:28 <docaedo> o/
17:01:31 <kzaitsev_mb> o/
17:01:52 <kzaitsev_mb> have you guys recovered from jetlag? )
17:02:11 <drwahl> barely
17:02:12 <docaedo> I had a miracle this time, no jet lag when I got home
17:02:23 <docaedo> never got over jet lag in tokyo though :/
17:02:36 <docaedo> Todays agenda:
17:02:38 <docaedo> #link https://wiki.openstack.org/wiki/Meetings/app-catalog#Proposed_Agenda_for_November_5th.2C_2015_.281700_UTC.29
17:02:52 <drwahl> i was in tokyo just long enough to get used to it, then the next day headed home. so i feel like i've been jetlagged for like 2 weeks straight
17:03:05 <docaedo> hoo that sucks!
17:03:37 <drwahl> caffiene is making life tolerable
17:04:18 <docaedo> it's magic sometimes isn't it?
17:04:19 <kzaitsev_mb> I hadn't been lagged in Tokyo, but had on my way here — I've never been getting up that early =)
17:04:39 <docaedo> haha
17:05:15 <docaedo> light attendance, but might as well roll on :)
17:05:18 <docaedo> #topic Tokyo Summit update (docaedo)
17:05:37 <docaedo> I think we had several really good sessions and conversations in Tokyo
17:05:58 <docaedo> Big thanks to kzaitsev_mb for all your efforts to bring everyone together and build bridges!
17:06:35 <docaedo> Also same goes for kfox1111_ - both of you made a lot of connections and facilitated great conversations
17:07:14 <docaedo> I also wrote an email that touched on what I thought were the major things that came up in Tokyo that people were asking for/interested in
17:07:16 <kzaitsev_mb> Well, after all that was the reason we travelled half the word =) To meet each other =)
17:07:17 <docaedo> #link http://lists.openstack.org/pipermail/openstack-dev/2015-November/078217.html
17:07:52 <docaedo> Rather than recap the whole email, I'll just shorten to say we need to work on trusted content upload/download (signed content for guaranteed integrity)
17:08:25 <docaedo> also trusted providers - for instance RAX or Dreamhost would like to add assets to the catalog and easily filter down to just those for a view they show to their customers
17:08:28 <j_king> i'd say it's more for verification, as in most package repos
17:08:38 <docaedo> indicating that stuff is supported by them
17:09:52 <docaedo> (but then also share the other contents for customers as an "unsupported" app/component)
17:10:04 <drwahl> we (dreamhost) would also like filters for things like image format (i.e. raw vs qcow2)
17:10:39 <docaedo> drwahl: that's super easy, and will be part of the API implementation for sure
17:11:03 <drwahl> \o/
17:11:31 <docaedo> also apologies for using you as an example - I know you're not committed to this, but I like to point you guys out as a good example of a big public cloud running OpenStack :)
17:11:54 <docaedo> (and of course, I'm hoping we can make this something that you are excited to expose to your users!)
17:11:55 <drwahl> i am/we are honored
17:12:07 <docaedo> :)
17:12:43 <docaedo> Last big point is all the API work we have ahead of us, and that it's an absolute requirement both for making the data easier to ingest/sort/expose, as well as make it much easier to add and maintain content
17:13:18 <docaedo> for posterity, sharing two of the etherpads that had direct or at least tangentially related info:
17:13:21 <docaedo> #link https://etherpad.openstack.org/p/TYO-app-catalog
17:13:24 <docaedo> #link https://etherpad.openstack.org/p/murano-mitaka-contributors-meetup
17:13:45 <docaedo> Anyone else have updates from app-catalog work in Tokyo?
17:14:08 <kzaitsev_mb> The conversation turned out to be quite productive on Friday, pity you only managed to get to the 2d part.
17:14:47 <j_king> unfortunately I was flying home on fri
17:14:53 <docaedo> kzaitsev_mb: yeah, I know :/  kfox1111_ got me caught up and filled me in on most/all of it
17:15:32 <kzaitsev_mb> I was keeping meeting minutes, so I probably should give an update on that to the ML =)
17:16:33 <docaedo> kzaitsev_mb: ah yes, if you have stuff to share that would be great - thanks by the way for keeping the minutes, I meant to thank you for that on my message
17:17:20 <docaedo> ok moving on...
17:17:28 <docaedo> #topic status updates
17:17:52 <docaedo> since I don't think much has happened since tokyo, let's check this:
17:17:59 <docaedo> jet lag recovery status:
17:18:05 * docaedo fully recovered
17:18:13 * drwahl is 90% recovered
17:18:38 * j_king fully recovered
17:19:21 <docaedo> haha
17:19:34 <docaedo> #topic Alternating times/regions for IRC meeting
17:20:12 <docaedo> lifeless made a really good point during the fishbowl - this particular meeting time is not compatible with australia/asia
17:20:17 <j_king> consolidating notes/decisions/etc -- how we do?
17:20:45 <j_king> etherpad per meeting?
17:20:50 <docaedo> j_king: great question :) what I've seen so far is
17:20:57 <docaedo> yes, etherpad is one big piece
17:21:26 <kzaitsev_mb> one thing I heard from glance team about the idea — was that they had a very negative experience with the thing, cause the team was split in two with two separate groups of people discussing different things.
17:21:30 <docaedo> and main thing would be that the chair has to have at least read the absorbed the log of the other meeting, and basically present a quick recap
17:21:35 <kzaitsev_mb> I believe though that it would not apply to app-cat
17:21:47 <kzaitsev_mb> just something we should be aware of
17:22:05 <docaedo> kzaitsev_mb: it could actually, so good to bring it up to be sure we stay aware
17:22:20 <kzaitsev_mb> I'm attending horizon meetings and they alternate time weekly, and seem to be ok with that
17:22:34 <docaedo> I think alternating weekly is the way to go
17:23:00 <docaedo> I'm really not sure the alternate time one would have any attendees other than Tom Fifield (he's usually in australia)
17:23:13 <docaedo> so this might be more a matter of putting it out there for a while to see who can attend
17:24:05 <docaedo> looking at my own schedule though, I don't really have a time slot that works well with australia, as I usually put my kids to bed, and am not up too much later than that (and would rather talk to my wife for a while after the kids are in bed rather than you lot ;) )
17:24:34 <drwahl> i second ^^
17:24:51 <docaedo> but if we can get someone to chair a meeting time that works well for austrlia/asia, figure we're safe to set it up and see if we start seeing participation
17:26:33 <docaedo> Thanks for the feedback on this - we'll structure it alternating weeks, and will be mindful of splitting efforts/conversations
17:27:13 <docaedo> regarding etherpad, I'm mostly on board with that, but I like having the agenda and previous meetings on the wiki (it's permanent, vs. ethereal)
17:27:39 <docaedo> figure since the meeting is being logged, it's easy to refer back to the conversation that was missed if/when it is missed
17:28:36 <docaedo> any other thoughts/opinions on that? Otherwise I'll move forward with trying to find a chair and a time slot, and this time slot will become every other week (once we've set up the second meeting)
17:29:23 <j_king> we can consolidate the etherpad over a cycle of meetings into a wiki page?
17:30:16 <docaedo> j_king: that's a pretty good plan
17:31:54 <docaedo> ok I think we're good then :) moving on
17:31:59 <docaedo> #topic Next steps for API work
17:32:35 <docaedo> I'm not comfortable planning too much here without kfox1111_ in attendance,
17:33:16 <docaedo> but can say the demo we saw from ativelkov showed glance artifact (glare) covers much of what we need from an API and in a way we can work with
17:33:46 <docaedo> my only concerns are the other requirements, and I'll argue strongly against it if we get into implementation details and find we need to run our own keystone on the web server
17:33:55 <docaedo> and we need to run our own barbican to sign things
17:34:19 <docaedo> (and we need to run mistral to manage the image upload workflow or something - since there's much debate right now around how to handle uploads in the future)
17:34:22 <docaedo> etc.
17:34:30 <kzaitsev_mb> I believe that shouldn't be the case =)
17:34:50 <docaedo> BUT as long as glare provides an excellent and sane abstraction between the set of assets, and the DB that info live in, I think we will be good to go
17:35:17 <docaedo> kzaitsev_mb: I do not think it will go that direction either - just stating that's a thing I worry about
17:35:58 <kzaitsev_mb> ativelkov is also away for 1.5 more weeks — travelling around japan.
17:36:40 <docaedo> oh nice! I hope he enjoys it. I love Japan, hopefully will have a chance to go back and travel around more (and bring the family next time :) )
17:37:16 <docaedo> I think it's safe to say next steps will be experimenting with implementation of glare, and thinking through how we'll deploy and maintain down the road
17:37:33 <kzaitsev_mb> I believe the next step with the plugin demo should be to commit the code to some app-catalog repository for review
17:37:55 <kzaitsev_mb> don't think we should do a separate repo for that
17:38:30 <docaedo> kzaitsev_mb: yeah I don't want to fork glare stuff, so just need to figure out exactly how we pull it in
17:39:49 <kzaitsev_mb> right now the code is on github, I believe.
17:39:57 <kzaitsev_mb> Gotta give it a bit of a thought
17:40:25 <docaedo> OK we can continue the conversation around glare implementation then on the regular channel and the mailing list over the next two weeks, then when ativelkov gets back form vacation we can get into more details too
17:40:29 <kzaitsev_mb> I mean — the plugin is supposed to be a python package afaiu
17:40:47 <kzaitsev_mb> but in any case — I believe it should be ok to just have it in a separate directory
17:41:19 <docaedo> yep - this should be something we can stand up with pip install pretty easily
17:42:03 <docaedo> and probably on the server install from debian package :)
17:42:44 <docaedo> Ok think we can move on to open discussion so there's time for more stuff (just in case anyone has anything to discuss that is)
17:42:47 <docaedo> #topic Open discussion
17:43:49 <drwahl> so, i've been poking around the app-catalog stuff that is on https://apps.openstack.org/
17:43:55 <drwahl> and most of the glance images are qcow2
17:44:00 <drwahl> which doesn't work with ceph
17:44:14 <drwahl> (at least, not until kilo. then glance can auto-convert)
17:44:16 <kzaitsev_mb> oh, I'd like to think a bit about automating asset uploads/updates. We talked on it a bit, in Tokyo, but less than I hoped.
17:44:26 <kzaitsev_mb> qcow2 — yep, about 90%
17:44:44 <drwahl> so essentially 90% of the app catalog is useless to those using ceph on the backend
17:45:16 <drwahl> i don't have an answer/solution to this (beyond saying upgrade glance to kilo for auto-convert), but it creates a poor user/operator experience
17:45:24 <docaedo> yes, the issue of converting images on demand has come up
17:45:53 <docaedo> I'm in favor of implementing a mechanism that converts (when possible) on request, and caches the converted image
17:46:30 <drwahl> the other thought i had bouncing around my head was having an "app" in the catalog (at least for glance), be more like a container
17:46:37 <drwahl> and the container could have qcow2 or raw
17:46:42 <drwahl> and the user select which they want/need
17:47:12 <docaedo> yep, that's about what I was thinking - the additional format would just become part of the asset
17:47:33 <kzaitsev_mb> drwahl: doesn't feel generic enough
17:47:36 <docaedo> in that you have the metadata of the asset describing everything, and then one or more links to different available formats
17:47:48 <kzaitsev_mb> although..
17:48:08 <kzaitsev_mb> every asset seem to have it's own format of a kind
17:48:29 <docaedo> but for glance images, it's pretty easy to do this
17:48:31 <drwahl> that does put the onus on the uploader to provide multiple formats, or some job on the backend to create multiple formats for the app catalog
17:48:38 <j_king> yeah... do we want like 5 entries for every format that a node.js "app" can take?
17:48:40 <docaedo> i.e. upload qcow, and then convert that, and keep both files
17:48:46 <kzaitsev_mb> docaedo: drwahl: well, glance images already have format
17:49:00 <kzaitsev_mb> you can even sort based on format in their schemas
17:49:16 <kzaitsev_mb> or am I getting the idea wrong?
17:49:26 <docaedo> right to me this use case only works for glance stuff, where you're talking about a base image that can be retrieved in different formats
17:49:54 <docaedo> (think of it as a directory, and you can download in tar.gz format or .zip format, but the source for both those files is the same)
17:49:55 <kzaitsev_mb> oh, you mean — you want to have 1 asset with mutiple BLOBs based on the format
17:50:01 <drwahl> ya
17:50:02 <docaedo> kzaitsev_mb: exactly
17:50:45 <drwahl> ideally, you wouldn't need to download *all* the blobs for an asset, just the one you actually want (since these can be kinda huge files)
17:51:02 <kzaitsev_mb> That indeed makes sense for glance
17:51:08 <docaedo> I would not want to put the conversion effort on the user though, because few people will do that, and we can easily store multiple blobs per asset
17:51:45 <docaedo> (and convert them either automatically upon upload, or else on demand, though the conversion process is not exactly quick...)
17:51:49 <drwahl> where are these assets being stored? would auto-creating raw/qcow/etc. be overly burdensome?
17:52:12 <drwahl> not just auto-creating, but also storing multiple copies essentially
17:52:20 <drwahl> (one for each format)
17:52:50 <kzaitsev_mb> drwahl: wonder if it would be a good idea to convert on-demand..
17:52:56 <kzaitsev_mb> might take a lot of time though
17:52:56 <docaedo> the actual binary bits will end up in S3 somewhere, provided by one of the many generous people who give up free compute and storage to OpenStack infra
17:53:17 <drwahl> we (dreamhost) might even be able to provide some storage resources
17:53:36 <kzaitsev_mb> so probably it should happen after initial upload
17:53:37 <docaedo> TBH drwahl this could be backed by dreamobjects ;)
17:53:44 <drwahl> that's what i was thinking :)
17:54:05 <kfox1111_> fyi, my wife's not feeling well, so I'm teleworking today. I may have to hop off randomly to chase after a 2 year old.
17:54:33 <docaedo> kfox1111_: no prob, hope your wife feels better soon and enjoy chasing the kid :)
17:54:44 <kfox1111_> thx. :)
17:55:01 <kzaitsev_mb> kfox1111_: =)
17:55:39 <docaedo> drwahl: we will have to keep this conversation going, I think there's a good opportunity here not just for the app catalog but maybe even for openstack infra.  All the cool kids are getting in on hosting some bits of the big OpenStack gating system :D
17:56:52 <kzaitsev_mb> with final couple – I'd still want to get back to upload/update automation.
17:57:03 <kzaitsev_mb> we've talked about it a bit.
17:57:33 <drwahl> docaedo: sounds good. i'll chat with the required people on my side about what we can do to help out with this effort
17:58:31 <kzaitsev_mb> docaedo: what bits do we need to implement to get that going? I guess api, auth/users system and a more sane storage should do that.
17:58:46 <docaedo> drwahl: excellent - I think the infra team would be excited to have another cloud to gate stuff through, especially one that is a pretty honest OpenStack cloud, I can facilitate that from the openstack side
17:58:59 <docaedo> kzaitsev_mb: yeah, a legit API is required
17:59:00 <kzaitsev_mb> am I missing anything?
17:59:28 <docaedo> kzaitsev_mb: means authentication, and backed by an API that is aware of versions of assets, authentication and ownership, etc.
18:00:23 <docaedo> kzaitsev_mb: at this point I think we need to see how quickly we can ingest/implement glare for that - otherwise we're going to spend a lot of time solving the same problem in a minimal way just to throw it out when we switch to glare (assuming it works in practice at least as well as the demo)
18:00:50 <docaedo> oh .. out of time :(
18:01:03 <docaedo> OK let's continue the conversation on the #openstack-app-catalog
18:01:03 <kzaitsev_mb> yep, I'd want to get it right on the 1st try, not implement another workaround
18:01:07 <docaedo> thanks everyone!
18:01:12 <kfox1111> might be a good thing to see if alexander can prototype.
18:01:26 <docaedo> #endmeeting