17:00:03 <gibi> #startmeeting nova notification
17:00:04 <openstack> Meeting started Tue May  3 17:00:03 2016 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:07 <openstack> The meeting name has been set to 'nova_notification'
17:00:17 <gibi> hi!
17:03:02 <gibi> hm
17:03:53 <cdent> o/
17:04:01 <gibi> hi cdent!
17:04:04 <gibi> I'm not alone! :)
17:04:13 <syjulian> hi guys
17:04:18 <cdent> sorry, i'm so deep in a hair nest of debugging that I keep losing track of the time
17:04:18 <gibi> hi syjulian!
17:04:34 <gibi> cdent: no worries
17:05:23 <gibi> ok I guess we start and other can join later
17:05:54 <gibi> I happy that we can restart this work on notifications
17:06:21 <cdent> ++
17:06:25 <gibi> #topic outstanding reviews
17:06:55 <gibi> so we have a main spec about transforming the first batch of legacy notifications
17:07:09 <gibi> #link https://review.openstack.org/#/c/286675/
17:07:28 <gibi> I think we are close to get it merged
17:07:51 <gibi> cdent: want to talk about your comment about the verbosity of the spec?
17:08:44 <cdent> It wasn't specifically directed at that spec, but at nova specs in general. It feels like there's a dangerous trend. That just happened to be the one I was looking at when the need to write that down overtook me.
17:09:17 <gibi> cdent: I share your worries even if I see some reasons behind the size of the spec
17:09:19 <cdent> I get why you did it in that spec, and it makes good sense. I just fear that that level of detail could become a habit
17:09:40 <gibi> cdent: +1
17:09:54 <gibi> cdent: what about stating an ML thread about this trend?
17:10:11 <cdent> I probably will, but I'm struggling to come up with a way to clearly articulate the problem.
17:10:19 <cdent> I'll keep thinking
17:10:44 <gibi> cdent: you can share your raw version of the mail privatly with me and I can comment if that helps
17:11:06 <cdent> that's great, thank you, I'll see what I can put together soon
17:11:13 * cdent adds to this list
17:11:18 <gibi> great!
17:11:39 <gibi> anyhow I will ping couple of cores about the spec to get it merged
17:12:03 <cdent> I think the work specified is great, will be glad to get that moving
17:12:05 <sjmc7> hey gibi - this is steve, we met on friday. i’m lurking as an interested party here
17:12:23 <cdent> Did you see my one question about a reminder on how to deal when both notifications are set?
17:12:24 <gibi> sjmc7: hi steve! great that you joined!
17:12:57 <gibi> cdent: yes, I saw, basically we are emitting the versioned notifications on a separate topic 'versioned_notifications'
17:13:11 <gibi> cdent: this was implemented in Mitaka
17:13:30 <gibi> cdent: so consumer can diferentiate by topic name
17:13:34 <cdent> ah cool, excellent
17:13:41 <cdent> I figured it was covered
17:14:03 <gibi> cdent: I added a small clarification about it in the last path set
17:14:29 <cdent> nice
17:14:58 <gibi> any other comment on the transformation spec?
17:16:13 <gibi> my other business is the JSON schema generation spec for oslo.versionedobjects
17:16:21 <gibi> #link https://review.openstack.org/#/c/311194/
17:16:29 <sjmc7> yeah, we’ve got some interest in that, too, as a consumer
17:16:41 <gibi> sjmc7: I can imagine :)
17:17:22 <cdent> very glad to see that one
17:17:26 <gibi> I think that spec doesn't have outstanding comment from nova side but I have to get feedback to oslo guys
17:17:36 <sjmc7> yeah. how many different versions do you expect to have floating around in a given deployment?
17:17:38 <gibi> s/to/from
17:18:11 <gibi> sjmc7: I think one nova deployemnt means a single version of an notification (event_type)
17:18:46 <gibi> sjmc7: as the version is defined by the nova code
17:18:47 <sjmc7> ok. i’m a little nervous at having to try to cope with loads of different versions for a given release, even if we have schema available
17:19:19 <gibi> sjmc7: what can happen is that if we have a notification that is emitted from nova-compute and deployer does a rolling upgrade
17:19:36 <sjmc7> ah, ok, that makes sense
17:19:37 <gibi> sjmc7: then old nova-computes might emit the old version
17:19:49 <sjmc7> yeah. ok, that’s soemthing we’ll try to bear in mind
17:20:02 <gibi> sjmc7: I think as soon as we have to need for the first version bump we have to understand this situation better
17:20:14 <sjmc7> in terms of the schema generator - that’d be something we can either run or request from nova and get a static schema at configuration time?
17:20:58 <gibi> sjmc7: I think I can add a small script to nova/tools to generate the schema
17:21:09 <gibi> sjmc7: in a similar way as config.samples can be generated
17:21:15 <gibi> sjmc7: that would be OK for you?
17:21:16 <sjmc7> yeah, that would make sense
17:21:55 <gibi> great
17:22:26 <gibi> any other question / comment on the schema spec?
17:23:00 <gibi> great
17:23:20 <gibi> I think we have a bunch of specs up on review that wants to add new notifications
17:23:28 <gibi> I tried to gathered them on the etherpad
17:23:37 <gibi> #link https://etherpad.openstack.org/p/nova-versioned-notifications
17:24:15 <gibi> I'm trying to keep reviewing them
17:24:42 <gibi> but I have some backlog from last week
17:25:12 <gibi> if somebody wants to speak about them then it is a good time :)
17:26:07 <gibi> syjulian: I saw that you published your spec about adding new fields to the instance.create notification
17:26:19 <gibi> #link https://review.openstack.org/#/c/312169/
17:26:55 <syjulian> yeah. it's pretty much adding that scheduler_hints field to the create notification once it's transformed
17:27:24 <syjulian> does that go in the etherpad above?
17:27:32 <gibi> I added it to the etherpad just now :)
17:27:37 <syjulian> cool
17:27:59 <gibi> I will try to review it tomorrow
17:28:17 <syjulian> awesome!
17:28:39 <gibi> takashin has a patch of transforming the instance.create notification so your addition will go after it
17:29:08 <gibi> #link https://review.openstack.org/#/c/279416/
17:29:47 <gibi> we don't have instance.create on the list in the transformation spec but the agreement on the midcycle was that only the first couple of transformation needs a spec
17:29:54 <gibi> so I think it won't be a problem
17:30:15 <sjmc7> we’ve also got some stuff we’d like to add; i’ll put up a spec for that dependent on the initial one
17:30:41 <gibi> sjmc7: just out of curiosity, which notification you want to extend?
17:31:08 <sjmc7> at this time, the ones related to servers
17:31:42 <gibi> sjmc7: OK, ping me when the spec is up and I will review it
17:32:50 <sjmc7> thanks. our ideal is that all events that are output that change a server include the full set of information about that server (so the .create notification payload is the same or very simiilar to the .update one)
17:33:31 <sjmc7> which is mostly the case right now, although some of the extensions (like volume or interface attach) are a bit different
17:34:46 <gibi> sjmc7: I see. I like the idea to make those streamlined. Have you considered the amount of data you will get?
17:35:11 <gibi> sjmc7: ceilometer guys had problems handling that in the past
17:35:26 <sjmc7> yeah.. for us though it’s much easier for us to receive a full document and index it versus figuring out what changed
17:35:40 <sjmc7> yeah, i know. i’m not sure whether that’s because of the way they were handling it
17:35:45 <sjmc7> storing in mysql, for instance
17:36:05 <cdent> ceilometer has many mistakes
17:36:14 <sjmc7> :)
17:36:16 <cdent> in both the datastore and the processing pipeline
17:36:19 <cdent> it's much better now
17:36:26 <cdent> although still some issues
17:37:07 <cdent> I tend, however, to think that including all the data in every notification is kind of...weird
17:37:15 <sjmc7> yeah, i can see that
17:37:35 <cdent> there's a part of me that thinks they should just be an event type and a resource identifier
17:37:36 <cdent> and that's it
17:37:46 <sjmc7> then we have to hit the API?
17:37:53 <cdent> I agree that can make things really difficult for assembling info later, though
17:37:56 <sjmc7> we’re really trying to move away from that
17:38:00 <cdent> yes, I know
17:38:11 <cdent> but I think that's only because the API is poor
17:38:32 <sjmc7> we are able to handle partial updates, it’s just more tricky. i can certainly understand not wanting to include more than is necessary, although it might be simpler from the backend’s perspective
17:38:35 <cdent> if it were good at resolving "what's the info on this id" then it would make moderately more sense
17:38:51 <cdent> I agree that if some info is sent, it may as well be all of it
17:39:44 <sjmc7> we are actually hitting the API now on some requests
17:40:22 <sjmc7> well, let’s get v1 of the notifications done and go from there :)
17:40:35 <gibi> I feel we cannot solve the API problem soon so we might want to add those extra fields to support searchlight
17:40:52 <gibi> and in the meantime start making the API better
17:41:06 <cdent> :)
17:41:16 <sjmc7> yeah.. part of why we’re doing this is to have a way of making horizon (or other consumers) experience better without huge demands on each service API
17:41:52 <gibi> sjmc7: I think the searchlight use case is valid
17:42:21 <gibi> anything else on sjmc7's spec?
17:43:22 <gibi> OK. Then let's go back a bit to the schema generation spec as harlowja pinged me on the oslo channel minutes ago
17:43:31 <harlowja> \o
17:43:38 <gibi> o/
17:43:45 * harlowja posted a small comment on that spec
17:43:52 <gibi> harlowja: thanks
17:44:00 <gibi> harlowja: so you suggest to use yaml instead of json
17:44:01 <harlowja> also i found that json-schema.org appears dead :-/
17:44:13 <harlowja> well i just want to clarify, the json schema is that IETF schema right
17:44:15 <harlowja> ?
17:44:26 <harlowja> (the one with the dead website, lol)
17:44:40 <gibi> harlowja: yes
17:44:42 <harlowja> k
17:44:46 <gibi> harlowja: with the dead page :D
17:44:52 <gibi> harlowja: it worked last week :D
17:44:55 <harlowja> :-P
17:45:09 <harlowja> ok, then i think json is fine, since making that yaml would be weird
17:45:41 <harlowja> u guys are confident that json schema will be able to handle the intricacies of versioned objects?
17:46:10 <gibi> at least we have some experience in nova with the API request validation, there the json schema works well
17:46:13 <harlowja> k
17:46:37 <gibi> I also wrote jsonschema for to validate a small query language in celiometer in the past
17:46:37 <cdent> it's perfectly fine to write jsonschem out as yaml
17:47:01 <harlowja> cdent right, i was just thinking that if these are going to be eventually stored as files, then yaml might be nice (due to comments)
17:47:28 <harlowja> but that might be a little far off (?)
17:47:49 <gibi> on nova side we don't want to store them just generate them on demand
17:47:50 <cdent> i'm trying to locate something, related
17:47:56 <harlowja> gibi ok
17:48:15 <cdent> here we go: https://github.com/brantlk/service-catalog-schema/blob/master/schemas/v2.yaml
17:48:20 <gibi> harlowja: it does not mean notification consumers will not store it
17:48:40 <gibi> cdent: this look nice
17:48:46 <harlowja> cdent ya, i guess thats jsonschema in yaml :-P
17:48:53 <cdent> ya
17:49:55 <harlowja> cool, will up to u all
17:50:09 <gibi> I'm not agains use yaml this way it is just a bit wierd that we produce the notifications as json and use yaml to describe the schema
17:51:00 <gibi> anyhow this is not a big change to add later if somebody need it
17:51:22 <gibi> harlowja: thanks for bringing this up and thanks for the review
17:51:31 <harlowja> np
17:51:33 <gibi> harlowja: I will respond
17:51:58 <rlrossit> gibi: I have arrived :). Just very very late
17:52:07 <gibi> rlrossit: no worries, logging is on :)
17:52:36 <gibi> anything else on the schema generation?
17:53:37 <gibi> anything else on any other open spec about notifications?
17:54:12 <gibi> OK then lets move on
17:54:14 <gibi> #topic open discussion
17:54:21 <gibi> I have one simple topic
17:54:46 <gibi> I will be on vacation next week so do we have a volunteer charing the next meeting?
17:55:48 <cdent> do we have the numbers to warrant having the meeting next week if you're not here?
17:55:53 * cdent looks at rlrossit
17:55:57 <rlrossit> agrees with cdent
17:56:01 * rlrossit doesn't want to volunteer
17:56:16 <gibi> :)
17:56:38 <rlrossit> I still need to change when I eat lunch on Tuesdays...
17:56:46 <cdent> I'd say skip, use email and gerrit if something comes up, and just look forward to massive happenings the following week
17:56:49 <gibi> rlrossit: sorry about that
17:57:02 <gibi> cdent: OK, let's do that
17:57:05 <rlrossit> gibi: blame my friends who are resistant to change
17:57:43 <gibi> rlrossit: but during the winter it will be good again due to not having a summer time
17:58:03 <gibi> #agreed skipping next week meetin, use email and gerrit instead
17:58:37 <gibi> any other thing to discuss?
17:58:45 <gibi> we still have 90 seconds left :D
17:59:40 * gibi will wait until the hour as it is so close
17:59:54 <gibi> great
17:59:57 <gibi> thank you all!
18:00:06 <gibi> see you around!
18:00:12 <gibi> #endmeeting