14:05:05 <markwash> #startmeeting glance
14:05:05 <openstack> Meeting started Thu Jun  5 14:05:05 2014 UTC and is due to finish in 60 minutes.  The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:05:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:05:08 <openstack> The meeting name has been set to 'glance'
14:05:49 <markwash> quick update from me for project stuff
14:06:17 <markwash> There are some notes on the email list and in the openstack/governance repo for adopting a mission statement for the Images program
14:06:54 <markwash> actually there's two changes: 1) adopt a mission statement that reflects our old mission 2) change the mssion statement to reflect the catalog / api artifacts stuff
14:07:23 <markwash> #link http://lists.openstack.org/pipermail/openstack-dev/2014-June/036767.html
14:07:38 <markwash> that's it from me, any thoughts folks wanna share?
14:08:30 <rosmaita> i put coments on the reviews
14:09:04 <arnaud__> markwash, do we want to rename everywhere we mention Image Service?
14:09:06 <jokke_> I replied to the mail
14:09:48 <markwash> jokke_: yeah I saw your question. honestly I'm okay with squashing them but I think the process-y folks are going to prefer we adopt one officially before we change it
14:09:54 <markwash> since it is a change to adopt the catalog approach
14:10:14 <markwash> arnaud__: like, s/image/catalog/?
14:10:22 <arnaud__> yep
14:10:33 <markwash> arnaud__: I have the same question
14:10:42 <ewindisch_> I like the two-step as it clarifies and records the change
14:10:45 <arnaud__> I mean much of the documentation is refering to image service
14:10:54 <jokke_> markwash: that's perfectly understandable ... just wanted to hear what's the story behind it as it kind of seemed to be bit silly couple out there ;)
14:11:06 <nikhil__> what do we do about logic around data?
14:11:49 <markwash> I was thinking of putting a comment in my own review to the effect that we should consider changing the program name from images. . but I bet there is a lot to change as you say so I wanna get some feedback from the infra/tc type folks
14:12:17 <markwash> nikhil__: can you elaborate a little on your question?
14:12:57 <nikhil__> currently, Images service seems to indicate most (if not everthing to do with images)
14:13:13 <nikhil__> including properties and image data ownership
14:13:37 <rosmaita> so if we call it "Artifact Repository Service", that covers that we don't actually create artifacts, we just store, catalog, and make avaialable
14:13:40 <nikhil__> if we rename the program to catalog, wondering if there would be immediate side effects
14:14:02 <markwash> nikhil__: I think with artifacts / the catalog, much of that data stuff stays the same, since images are just a type of artifact
14:14:30 <nikhil__> (like people creating other tiny little services to deal with it)
14:15:13 <rosmaita> i agree that glance is not simply a catalog
14:15:41 <markwash> rosmaita: oh I see. so artifact repository service is an alternative to catalog
14:15:49 <rosmaita> markwash: exactly
14:16:10 <rosmaita> i think "repository" implies particular types of services
14:16:21 <nikhil__> yeah
14:16:28 <rosmaita> cataloging, discovery, delivery, storage
14:16:37 <rosmaita> basically, what glance does
14:16:43 <markwash> ah I see
14:16:43 <TravT> what about the idea of the metadata catalog API?  is metadata a kind of artifact?
14:16:53 <nikhil__> that seems to cover a broader area of services
14:16:59 <markwash> having just gotten up, I hadn't yet seen the comments on the second patch
14:17:00 <rosmaita> no artifact == data + metadata
14:17:22 <rosmaita> An artifact is a (data, description) association in which the description has a well-defined discoverable type and the data is immutable once the artifact is made available.
14:18:08 <rosmaita> TravT: https://docs.google.com/document/d/1tOTsIytVWtXGUaT2Ia4V5PWq4CiTfZPDn6rpRm5In7U/edit?pli=1
14:18:26 <TravT> rosmaita: have you seen any of our discussion related to Graffiti?  at the summit, multiple people asked us to propose a metadata catalog become available from Glance as well.
14:19:07 <markwash> TravT we probably need to slightly refine our terminology around the glance artifact and the graffiti metadata catalog
14:19:33 <markwash> because the graffiti stuff does not really fit into the glance catalog, rather it is a worthy addition on the side of it
14:19:54 <ewindisch_> my concern with the current proposals for the artifacts, as I understand them, is that it still doesn’t address - say - Nova being able to access artifacts that live outside of the Image program
14:20:00 <rosmaita> TravT: i see grafitti as tagging artifacts + instances + workloads, etc
14:20:52 <TravT> rosmaita: yes it is for tagging the various things, but a key component of it having a catalog of available metadata.  Much of it very relevant to images and artifacts.
14:20:54 <rosmaita> ewindisch_: the artifacts will be stored in glance, nova can get them using glance api as it does currently for images
14:21:10 <markwash> ewindisch_: I think artifacts would like to expand to be able to serve anything that Nova could use (i.e. including stuff like aufs images. .
14:21:23 <markwash> ewindisch_: but I don't think we're proposing that Nova *only* be able to talk to glance
14:21:43 <lakshmiS> rosmaita: That sounds good about the idea of graffiti tagging the artifacts too...
14:21:47 <ewindisch_> markwash: but that’s the only way to “be core” and the only thing that passes the refstack tests
14:22:04 <TravT> with a catalog available, we can then make it available from horizon.
14:22:18 <markwash> ewindisch_: interesting
14:22:38 <rosmaita> TravT: i think the grafitti catalog would be a different level of abstraction
14:22:45 <markwash> TravT: its just going to make it harder for us if we call the graffiti metadata index a catalog as well :-)
14:22:59 <arnaud__> ewindisch_, a full project doesn't need to be core afaik, nova will not be core completely
14:23:06 <arnaud__> it can be the same for glance
14:23:29 <markwash> anyway we should pick different terms at some point
14:23:47 <rosmaita> +1 on good & different terms!
14:24:02 <TravT> markwash: different terms are okay with me.
14:24:53 <TravT> but are you still okay with it being proposed as part of the program?
14:25:16 <rosmaita> TravT: not clear on what you're asking?
14:25:24 <markwash> TravT: to me, it would fit in as a helper component to using the artifacts repository
14:25:41 <markwash> TravT: and the fact that it helps with metadata elsewhere in the cloud is just gravy
14:26:07 <rosmaita> TravT: i see artifacts metadata as developer-oriented, while grafitti tags are user-oriented (at the horizon level)
14:26:56 <msundara> the metadata drives behavior so it is sort of developer-oriented too.
14:27:47 <markwash> msundara: yes, but also the goal with artifact type schemas is to move as much behavior-affecting attributes *out* of metadata
14:28:09 <markwash> excuse me: "out of *user-metadata*"
14:28:31 <msundara> markwash: yes, we talked about that last time and graffiti is only for data taht is outside the syntax of the original resource.
14:28:51 <markwash> okay this kinda morphed into a graffiti discussion
14:29:09 <TravT> yes, we can have separate discussion.
14:29:21 <markwash> but I think maybe we should move on. . I appreciate all the feedback here and in the reviews for the mission changes and will update them this morning
14:30:10 <markwash> today we have jokke_ here so let's hit his questions :-)
14:30:19 <jokke_> :)
14:30:22 <markwash> #topic testing approaches (jokke_)
14:30:51 <markwash> #link https://blueprints.launchpad.net/glance/+spec/fake-keystone-for-testing
14:30:56 <jokke_> so, the noauth clearly is causing some grey hair in our testing
14:31:51 <markwash> I see you linked in a pretty gross sounding bug as well
14:31:57 <markwash> #link https://bugs.launchpad.net/glance/+bug/1308413
14:32:00 <uvirtbot> Launchpad bug 1308413 in ossa "Missing x-tenant-id header to registry will return list of all images  while using v2 api with registry" [Undecided,Won't fix]
14:32:07 <jokke_> I was planning to build a small fake keystone that would do just "token verification" and "pass auth" so we would get keystone like behaviour for the functional tests
14:33:12 <markwash> so my only concern is the amount of slowness around functional tests
14:33:19 <jokke_> markwash: the problem there is that as decided the v2 does not pass the tenant ID to registry anymore like v1 did
14:33:34 <markwash> I  agree it seems we've missed some coverage and we need to take some action to fix that
14:33:46 <jokke_> which broke pretty much all functional tests when trying to extend them to be used with registry
14:33:55 <markwash> oh wow
14:34:19 <markwash> jokke_: so the thing that is important to me is that the we kinda follow the pyramid approach
14:34:28 <markwash> a huge base of unit tests to get most of the coverage
14:34:40 <markwash> a medium set of integration tests that are still pretty fast
14:35:00 <markwash> and then a smaller sliver of slow functional tests that ensure everything isn't secretly broken when you string it together
14:35:23 <markwash> it sounds to me like we could have caught this just by adding real keystone testing to the functional level (with registry db turned on)
14:35:27 <markwash> does that sound accurate?
14:35:35 <jokke_> markwash: I do understand and like that approach
14:35:43 <markwash> well real -> less fake than noauth
14:36:00 <jokke_> My biggest concern is that we support only keystone auth, but those bits are not really tested anywhere
14:36:42 <markwash> jokke_: I would be pretty happy with your fake keystone approach if it wasn't really expanding the granularity of coverage at the functional level
14:36:49 <jokke_> and I do not like the approach either that we would monkeypatch the code we are testing to do something else what it's doing in production
14:37:05 <arnaud__> jokke_, what about tempest? did you try? with v2/registry?
14:37:51 <jokke_> arnaud__: to be honest, I haven't got that far yet. I have had full time job trying to get my head around glance so far ;)
14:38:11 <arnaud__> ok, no worries :)
14:39:03 <jokke_> markwash: also a point here. If we want to make keystone a default auth in glance config, that would not work if we do not have "keystone" available during the testing
14:39:18 <markwash> jokke_: so how about it, we can add a fake keystone test component to the functional level to do better than noauth, but not really change the granularity of the functional tests (or interfere with the goal of making those functional tests a bit coarser actually)
14:39:51 <jokke_> markwash: I'm perfectly fine with that, actually quite fond of it
14:40:00 <markwash> okay cool
14:40:23 <markwash> jokke_: it sounds like you might also want to add a run of the tests that configures the system to use the registrydb for v2
14:40:24 <jokke_> markwash: My plan was not increase testing too much on fnctional level, but rather just make the tests reflect more real life
14:41:18 <markwash> well, this makes sense to me then. . . other thoughts or should we move on to your question about log levels?
14:41:24 <jokke_> I just stumbled upon that fact when I tried to run the existing tests against the configuration where I enabled registry instead of sqlalchemy
14:42:02 <jokke_> the registry gets spun up currently, but nothing actually touches it
14:42:52 <markwash> gotcha
14:43:23 <markwash> #topic debug log levels that should be info
14:44:19 <markwash> jokke_: to make a long story short, I bet folks will be happy with the log changes you're proposing
14:44:35 <markwash> :-)
14:45:06 <jokke_> Yes, so while looking a bug filed agaist our stores and how they rise InvalidStoreUri exception and trying to find my way around translations with exceptions and debug logging I noticed that most of those messages should be logged higher level. Is there a reason why we are logging pretty much everything on debug?
14:45:47 <markwash> we at one time had way too much stuff at a higher level, like logging 400s as warnings
14:46:02 <jokke_> I think this was raised from the ops side as well as complaint that they effectively need to run their openstacks on debug as they will get next to nothing otherwise, which is not nice
14:46:22 <markwash> but I don't think we've done a recent audit of our log stuff (at least not that I remember / or am aware of)
14:46:53 <jokke_> ok ... so from one far end to another :)
14:47:23 <markwash> jokke_: so i think the message here is "fire at will" with any improvements to bring us more in line with the logging standards
14:47:37 <markwash> and for everybody, probably it is good to review the following link just to keep log levels in mind
14:47:56 <markwash> #link https://wiki.openstack.org/wiki/LoggingStandards
14:47:56 <jokke_> IMHO we should log our exceptions either on info or like the invalid store uris on warn ... after all that's something that really affects how the configuration behaves
14:48:48 <jokke_> markwash: ok ... I look into those lines where I walked on to mine and start looking those bit deeper when I have time for it
14:48:49 <markwash> it depends on the exception I guess
14:49:18 <markwash> some exceptions are just part of the regular code path, so unfortunately there is not a clear way to distinguish other than just the semantics of the given function
14:49:30 <jokke_> markwash: I agree, but if we cannot access store because there is typo in uri, that shouldn't need debugging before that gets told to ops ;P
14:49:34 <markwash> +1
14:49:35 <markwash> :-)
14:49:43 <markwash> okay lets talk catalog
14:49:47 <markwash> #topic catalog status
14:50:02 <markwash> gokrokve: ativelkov: got some updates for us?
14:50:16 <gokrokve> Hi
14:50:28 <gokrokve> Alex is here so he can do status update.
14:50:52 <ativelkov> hi
14:51:25 <ativelkov> I am going to submit the catalog specs to glance-specs in the nearest days
14:51:44 <ativelkov> Got the first draft of common metadata definition and the API
14:51:58 <ativelkov> will finalize and submit to gerrit soon
14:52:33 <ativelkov> This will include only the first batch of features - the ones which we agreed to implmenet in J
14:52:52 <gokrokve> Yes. Once we have specs I will join review & development process too :-)
14:52:52 <markwash> ativelkov: great, thanks! we will be on the lookout for it in glance-specs
14:53:15 <ativelkov> More advanced stuff (dynamic references etc) will be left for future
14:53:15 <gokrokve> Randall also wanted to join the development.
14:53:30 <arnaud__> (btw, glance-specs review link https://review.openstack.org/#/q/status:open+project:openstack/glance-specs,n,z)
14:53:31 <markwash> btw reminder everybody: glance-specs is live and is now the entry point for new feature proposals
14:54:20 <markwash> okay, just a few minutes to catch the late entry to our agenda
14:54:29 <markwash> #topic swift multi-container work
14:54:46 <markwash> this is something iccha and sri were working on but since Trove stole iccha away from us. . .
14:54:52 <markwash> rosmaita: is this your item?
14:55:16 <rosmaita> thought we discussed last week?
14:55:34 <rosmaita> we agreed that we need to get glance.store done first
14:55:38 <rosmaita> (i think)
14:55:56 <markwash> ah perhaps it was someone elses item
14:56:21 <nikhil__> markwash: ah it's mine
14:56:32 <nikhil__> markwash: just wanted to quickly confirm who all are interested in that MP
14:56:49 <markwash> nikhil__: okay cool, last week we said what rosmaita said
14:57:00 <nikhil__> am trying to get the ncessary changes in soon-ish to get it tested functionally in a dev environment with a live swift install
14:57:09 <markwash> great
14:57:46 <nikhil__> have some comments posted and if all interested parties can jump in to resolve the questions either way, it would help a ton!
14:58:13 <markwash> #link https://review.openstack.org/#/c/34801/
14:58:22 <markwash> all right, few minutes of open discussion
14:58:26 <markwash> #topic open discussion
14:58:29 <ashwini> mid cycle meetup
14:58:29 <markwash> everybody shout at once!
14:58:35 <jokke_> One more thing regarding last week, I updated https://wiki.openstack.org/wiki/Bug_Tags could someone who has access to it update the official tags?
14:58:46 <ashwini> july 28- aug 1,
14:59:06 <ashwini> location - VmWare office, CA
14:59:12 <ashwini> done deal?
14:59:35 <arnaud__> I want to make sure we know how many people
14:59:37 <arnaud__> can make it
14:59:38 <markwash> sounds good to me, I may have to leave a little early on august 1 in order to pick up someone from the airport
14:59:39 <arnaud__> to CA
15:00:00 <ashwini> arnaud__: all of rackspace team will
15:00:26 <markwash> okay, we're out of time! we need to make way
15:00:29 <nikhil__> ashwini: we have 2 more joining, so arnaud__ knows the updated count
15:00:34 <markwash> thanks everyone
15:00:37 <arnaud__> thanks
15:00:38 <markwash> #endmeeting