17:00:43 <docaedo> #startmeeting app-catalog
17:00:43 <openstack> Meeting started Thu Dec 17 17:00:43 2015 UTC and is due to finish in 60 minutes.  The chair is docaedo. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:45 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:47 <openstack> The meeting name has been set to 'app_catalog'
17:00:55 <docaedo> Todays agenda can be found here:
17:00:57 <docaedo> #link https://wiki.openstack.org/wiki/Meetings/app-catalog#Proposed_Agenda_for_December_17th.2C_2015_.281700_UTC.29
17:01:04 <docaedo> #topic rollcall
17:01:07 <docaedo> o/
17:01:09 <ativelkov> o/
17:01:10 <olaph> o/
17:01:29 <docaedo> hey look at that, it's enough for a meeting :D
17:02:00 <docaedo> #topic Status updates
17:02:29 <docaedo> I've been working on dead link checker (and some related stuff)
17:02:31 <docaedo> I'd like feedback on the "skip assets with active: false" review, I think it's good to go:
17:02:34 <docaedo> #link https://review.openstack.org/#/c/257663/
17:02:39 <docaedo> This will be used with the dead link checker (to be discussed later in this meeting)
17:02:42 <docaedo> But happy to discuss the method (thanks kfox1111 for pointing out it should be handled in python rather than JS!)
17:02:44 <docaedo> I want to start using this because the dead-link checker is ready (details later), and we could have stale links being automatically discovered in a week.
17:04:00 <docaedo> any other status updates? ativelkov maybe there's something new re: glare?
17:04:28 <ativelkov> Yeah, there are some updates on glare
17:04:56 <ativelkov> So, the summit's decision on separating the process for Glance and Glare is getting implemented
17:05:13 <ativelkov> The spec has been approved and the actual patches are ready and on review right now
17:05:35 <ativelkov> This will allow to run glare without a burden of having the rest of Glance deployed, which should be nice for app-catalog
17:05:43 <docaedo> nice! I've been following, would join the weekly meeting but it happens at a time I can't make :(
17:05:54 <docaedo> ativelkov: absolutely, I'm excited about it!
17:06:23 <ativelkov> Also, we are working towards API stabilization (addressing API-WG and DefCore concerns ets). The spec is on review
17:06:59 <ativelkov> Also, I've got a design for the pluggable DB backends (The feature we discussed with kfox1111 on the summit). I'll submit the spec in a day or two
17:07:28 <ativelkov> And last but not least: we are planning a video call in the nearest days to discuss glare roadmap in voice
17:07:46 <ativelkov> All are welcome to join
17:08:16 <docaedo> please share the details on the ML and maybe on #openstack-app-catalog because I'd like to join and others might as well
17:08:23 <ativelkov> We just need to sync with flaper87 and nikhil to find out the timeslot convienient for all
17:08:26 <ativelkov> Yeah, sure.
17:08:48 <docaedo> thx
17:08:49 <ativelkov> The problem is that the holiday season is approaching and it may be hard to find a slot equally convinient for all. But we'll do our best
17:09:17 <docaedo> yeah, we are rapidly running out of time to get things done in 2015 :)
17:09:46 <ativelkov> some links:
17:09:54 <ativelkov> #link https://review.openstack.org/#/c/254710/  - the api refactoring spec
17:10:28 <ativelkov> #link https://review.openstack.org/#/q/topic:bp/move-v3-to-glare - patches to move the glare api to separate process / keystone endpoint
17:11:01 <ativelkov> Feel free to review and leave comments if you have time :)
17:11:47 <docaedo> ativelkov: thanks, I will
17:12:20 <docaedo> I'll hold for one minute to see if there are any other status updates before moving on...
17:13:38 <docaedo> #topic Dead link check script
17:13:53 <docaedo> This script goes through the assets.yaml file, grabs the URLs and does a header request (with redirect follow).
17:13:56 <docaedo> If it doesn't get a 200 response, it adds "active: false" to the attributes section.
17:13:58 <docaedo> #link http://paste.openstack.org/show/482152
17:14:00 <docaedo> This is the output from the script, which found two dead links!
17:14:03 <docaedo> #link http://paste.openstack.org/show/482151
17:14:14 <docaedo> You'll note one difference from the active assets.yaml we have - the "---" at the top is gone, as that's no longer necessary due to there being only "assets" in the yaml, and no other record types.
17:14:20 <docaedo> It is adapted from the project normalizer script infra uses. It's coupled with a shell script that clones the repo, and if the check_catalog.py script found any dead links, it submits a review to update assets.yaml.
17:14:30 <docaedo> I'll propose this in app-catalog/tools and will coordinate with infra to sort out what user actually submits the PR (probably catalog-bot). With any luck we could have automated dead-url checks live in less than a week!
17:14:37 <olaph> sweet
17:14:57 <docaedo> I want to use this same script to add a mechanism for keeping a frequently changing asset up to date. The use case would be something like the Debian weekly testing image
17:15:00 <docaedo> #link http://cdimage.debian.org/cdimage/openstack/testing/
17:15:07 <docaedo> The asset entry would include something like "desired-hash-url: http://cdimage.debian.org/cdimage/openstack/testing/MD5SUMS", and the script could see if the current hash entry in assets.yaml matches.
17:15:09 <docaedo> If it's different, it updates the entry with the hash and an "updated: <date>" entry in attributes section.
17:15:11 <docaedo> The tricky part will be making this abstract enough to work for more than just debian weekly.
17:15:13 <docaedo> What do you think?
17:16:23 <olaph> makes sense to me...
17:16:47 <ativelkov> Well, this allows to ensure that the image is not modified on its way, yes
17:17:19 <docaedo> ativelkov: well yes, but more than that, will allow us to put some frequently changing things in the catalog without having to manually update them all the time
17:17:48 <ativelkov> Yes, I understand that. But I believe we should provide some notice to the users
17:17:54 <docaedo> ativelkov: when we have glare for the backend, this will probably change, because it will become reasonable for someone to just use the API to authenticate and update a record they own
17:18:19 <ativelkov> Something like "The asset has been changed by its owner since the last time we've checked that"
17:18:57 <docaedo> oh I was actually thinking the entry in the catalog would just get updated to include the new/latest hash when it changes, along with an entry for updated date
17:19:37 <docaedo> that way any time someone comes back to find the latest debian (or coreos, or any of the others that frequently change images)
17:19:51 <docaedo> they see what the latest is, along with a verifiable hash, and the date the record was updated
17:20:08 <docaedo> vs. right now for instance, we just point to the qcow entry and don't list the hash
17:20:09 <ativelkov> Ah yeah, so we need two timestamps then
17:20:22 <ativelkov> Or may be even three
17:20:48 <ativelkov> date when the entry is added to catalog, the date of the last update and the date of the last (manual) verification of the asset
17:21:18 <ativelkov> so the clients may choose not to download unverified assets
17:21:46 <docaedo> I would agree with added and updated, but not last verified (because the intention is that this will be automatically verified every day, triggering a review if anything changes - would not be useful to update every record with that date every day)
17:22:13 <docaedo> Keep in mind too - this is a stop-gap thing until we implement glare backend, when many of these mechanisms will change
17:22:27 <ativelkov> yeah, you are right, I think
17:22:37 <ativelkov> And manual verification does not scale for sure
17:22:49 <docaedo> My intention with this is to take the least-effort path to keeping assets "fresh" until we have a legitimate backend in place :)
17:23:16 <docaedo> I am done with dead-link checker except for coordination with infra, and can probably sort out the "hash checking" stuff before end of year
17:23:44 <docaedo> then next up probably work on some UI refreshes I have in mind while coordinating with you and others on glare implementation
17:24:35 <ativelkov> cool
17:24:45 <docaedo> As for manual in my short-term plan, the only manual step will be reviewing what the bot submits when it finds a change. That's low effort for me or others, but also provides a nice safety valve in case the bot goes crazy we can just ignore what it submits
17:25:18 <ativelkov> yeah, that sounds the right thing to have
17:25:46 <docaedo> sweet, agreement all around :)
17:26:55 <docaedo> as for the meeting agenda, I left "discuss glare outline" up, but I'm good with the draft and had it there in case kfox1111 and kzaitsev_ws were here (they might have had some points of debate)
17:27:26 <docaedo> So since they're not here, unless olaph or ativelkov want to chat about it?
17:27:28 <docaedo> #link https://etherpad.openstack.org/p/app-cat-glare
17:28:14 <olaph> I'm good
17:28:31 <ativelkov> I did review kzaitsev_ws's etherpad, it was quite aligned with what we are doing in glare
17:28:53 <docaedo> yep, I like the plan there too
17:28:58 <docaedo> once again - agreement all around :)
17:29:06 <docaedo> so moving on...
17:29:14 <docaedo> #topic Open discussion
17:29:48 <docaedo> Anyone have any open discussion points to openly discuss?
17:30:08 * docaedo stops typing for 1-2 minutes
17:30:17 <kfox1111> hi
17:30:29 <docaedo> oh hey!
17:31:12 <kfox1111> I added a few things to the etherpad, but still need to find a few minutes to finish reading. theres quite a bit in there.
17:32:18 <docaedo> kfox1111: are you implying Community App Catalog is not your only full-time job? What, are you trying to run a huge cloud or something too??? ;)
17:32:39 <kfox1111> ;)
17:33:00 <docaedo> Considering the scale of the effort, I'd say we leave it on the agenda and continue to discuss in January
17:33:14 <kfox1111> +1
17:33:17 <docaedo> (and of course, on #openstack-app-catalog between now and then)
17:34:10 <ativelkov> Agree.
17:35:55 <docaedo> ok - anything else for open discussion? kfox1111, anything you wanted to touch on that we talked about earlier? Much scrollback for you I think...
17:38:49 <kfox1111> things look good to me.
17:38:57 <docaedo> sounds like we're done for today then...
17:39:11 <kfox1111> yeah, keeping the sums up to date is seperate from prompting users there is updated stuff.
17:39:21 <kfox1111> hopefully versioning plays in somehow there too.
17:40:47 <docaedo> in the short term, I don't see any reasonable way to do versions in a manually-updated assets.yaml file, but will HAVE to have versions with glare
17:41:06 <docaedo> I should say .. versions could be done
17:41:15 <kfox1111> I'm curious if we can do some kind of web scraper plugin thingy for a few different things.
17:41:21 <kfox1111> like, ubuntu's cloud images page.
17:41:45 <kfox1111> they update them constantly, and have a lot available. being able to run a cron job that generates bits of assets.yaml with all the versions would be very nice.
17:42:05 <kfox1111> I think fedora does something similar.
17:42:28 <kfox1111> something to think about.
17:42:40 <docaedo> definitely worth thinking about
17:43:04 <kfox1111> (another thing unioning the yaml could help with)
17:43:20 <kfox1111> gtg. talk to you later.
17:43:31 <docaedo> ok thanks for dropping by :)
17:43:42 <docaedo> I think we're done - thanks everyone for joining!
17:44:16 <docaedo> #endmeeting