16:00:33 <krotscheck> #startmeeting StoryBoard
16:00:35 <openstack> Meeting started Mon Feb 16 16:00:33 2015 UTC and is due to finish in 60 minutes.  The chair is krotscheck. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:39 <openstack> The meeting name has been set to 'storyboard'
16:00:42 <krotscheck> Good morning everyone!
16:00:46 <yolanda> hi
16:00:49 <NikitaKonovalov> o/
16:00:53 <krotscheck> Agenda: https://wiki.openstack.org/wiki/StoryBoard#Agenda
16:01:18 <krotscheck> Everyone say hi to gulic!
16:01:33 <yolanda> welcome gulic! :)
16:01:40 <gulic> hi :)
16:01:47 <NikitaKonovalov> hi
16:01:47 <nibalizer> gulic: welcome!
16:02:26 <krotscheck> We’ll let you do formal introduction thigns in the main channel, that way people not at the meeting can get to know more about you :)
16:02:41 <krotscheck> #topic Actions from Last Week
16:02:56 <krotscheck> rcarrillocruz: I saw your notification coverage change merge.
16:03:12 <krotscheck> So that’s done
16:03:25 <krotscheck> #topic Actions from last week: krotscheck)
16:03:59 <krotscheck> Working on auth token verification made me incredibly angry at how our database wasn’t managing timestamps properly.
16:04:01 <krotscheck> So I fixed that.
16:04:15 <yolanda> did that change land?
16:04:20 <krotscheck> And then I fixed the timestamp auth check.
16:04:27 <krotscheck> I don’t think so, checking
16:04:35 <krotscheck> https://review.openstack.org/#/c/155083/
16:05:25 <krotscheck> Turned out that while it was “understood” that all datetimes were in UTC, the datetime() object never had a timezone attached to it, which created lots of boilerplate around trying to align and guess timezones.
16:05:33 <krotscheck> So that got fixed.
16:05:41 <yolanda> ok, i need to review it
16:05:47 <NikitaKonovalov> great
16:05:51 <NikitaKonovalov> I'll also review
16:05:59 <krotscheck> Story about pagination controls: https://storyboard.openstack.org/#!/story/11
16:06:13 <krotscheck> Story on autocomplete: https://storyboard.openstack.org/#!/story/2000163
16:06:21 <krotscheck> Fix cron: Didn’t get around to it.
16:06:39 <krotscheck> #topic Actions from last week ( NikitaKonovalov )
16:06:52 <NikitaKonovalov> ok, I've been working on sessions
16:07:05 <krotscheck> I saw! That’s neat!
16:07:05 <NikitaKonovalov> and finally there is a patch with a session per request approach
16:07:20 <krotscheck> NikitaKonovalov: I don’t suppose you had the chance to do a before/after benchmark?
16:07:41 <NikitaKonovalov> there is also a comment about using a specific pecan hook
16:07:49 <NikitaKonovalov> looks like it will be handy
16:07:53 <krotscheck> TransactionHook, yes.
16:07:55 <NikitaKonovalov> as for testing, not yet
16:08:05 <NikitaKonovalov> but willdo
16:08:13 <krotscheck> You don’t ened to, I’m just curious.
16:08:34 <krotscheck> The fact that we can use the ORM linking inside storyboard now is enough for me :)
16:08:38 <NikitaKonovalov> I'm also curious, that's why still will do
16:08:43 <krotscheck> NikitaKonovalov: You got it.
16:08:55 <krotscheck> NikitaKonovalov: I’m guessing you didn’t have time to work on the accesstoken index?
16:09:09 <NikitaKonovalov> yep, no indexes still
16:09:24 <krotscheck> No worries. Anyone want to take that off NikitaKonovalov’s plate?
16:09:30 <NikitaKonovalov> but I'll switch to them after the session is done
16:09:41 <NikitaKonovalov> or someone can take that
16:10:02 <krotscheck> Well, it’s a super easy thing for someone who wants to get ATC status before the convention passes go out.
16:10:07 <yolanda> what needs to be done there?
16:10:09 <NikitaKonovalov> btw we can mark it as a "low hanging fruit"
16:10:26 <krotscheck> yolanda: We need the mysql accesstoken table to index on the actual token column
16:10:28 <NikitaKonovalov> yolanda: there should be a migration to add index
16:10:39 <NikitaKonovalov> 1 file change
16:10:49 <yolanda> ok, i have my ATC badge but i can take it :)
16:10:54 <yolanda> do you have a story about it?
16:11:11 <NikitaKonovalov> let's see
16:11:44 <yolanda> np, or i just create one
16:12:22 <NikitaKonovalov> https://storyboard.openstack.org/#!/story/2000165
16:12:22 <krotscheck> Cool, let’s move on.
16:12:25 <NikitaKonovalov> there it is
16:12:30 <yolanda> thx
16:12:31 <krotscheck> #topic Urgent Items: Foreign Keys!
16:12:36 <krotscheck> NikitaKonovalov: the floor is yours.
16:12:46 <NikitaKonovalov> we've got a problem
16:13:07 <NikitaKonovalov> it is possible to create a comment for a story that does not exist
16:13:34 <NikitaKonovalov> which means that sqlalchemy declaration for a foreign key is not enough to enforce it
16:13:48 <NikitaKonovalov> without having a real foreign key in the db
16:14:01 <ricky1> o/
16:14:05 <NikitaKonovalov> and some time ago we droppped most of those
16:14:07 <krotscheck> That seems like a bug in SQLAlchemy
16:14:41 <NikitaKonovalov> krotscheck: probably
16:14:59 <jeblair> how would you create reference to it if you can't instantiate a story?
16:15:01 <NikitaKonovalov> I can try to roll back to 0.8 sqla and see how it works there
16:15:26 <NikitaKonovalov> jeblair: when creating a comment you do not need to instanciate a Story
16:15:40 <NikitaKonovalov> you just set story_id="whatever" and it works
16:16:00 <yolanda> so just manipulating the javascript should be enough?
16:16:07 <NikitaKonovalov> yep
16:16:20 <jeblair> that sounds like an input validation problem... normally i would expect an app to not only verify that the story existed, but also that the user had access to it, etc....
16:16:54 <krotscheck> Ditto on that.
16:17:30 <NikitaKonovalov> we have options here, we may have our own validation for all foreign keys
16:17:34 <krotscheck> Honestly I don’t know enough about how SQLA handles FK columns itself internally to handle things like that, so without investigation I can’t really come up with an opinion on this.
16:17:42 <NikitaKonovalov> and it can be effective with new session management
16:17:44 <krotscheck> I recognize that it’s an important issue that needs to be addressed though
16:18:05 <NikitaKonovalov> or get the constraints back on db backend level
16:18:38 <jeblair> a pattern i usually use is for this sort of thing is: story.addComment(...);  that makes it really hard to add a comment without a story
16:19:31 <NikitaKonovalov> jeblair: what we now have looks more like session.add(Comment(story_id=xyz, text="a new comment"))
16:21:09 <NikitaKonovalov> anyway I'll check older sqla versions first
16:21:37 <NikitaKonovalov> but most probably we need a good validation layer for that
16:22:02 <krotscheck> I feel like SQLA should be providing that, and if it’s not we should ping their channel for some guidance.
16:22:52 <NikitaKonovalov> krotscheck: agree, otherwise what's use of declaring constraints
16:22:52 <krotscheck> But I’ll leave it to NikitaKonovalov to figure it out.
16:22:58 <krotscheck> NikitaKonovalov: Exactly
16:22:58 <NikitaKonovalov> fine
16:23:04 <jeblair> related to the pattern i use...
16:23:12 <jeblair> i never set the id field directly either
16:23:23 <jeblair> i always associate two objects with each other using one of their relationships
16:23:45 <jeblair> which requires you always have the existing object loaded
16:24:16 <jeblair> so never: comment.story_id = story.id; but rather: comment.story = story; or: story.comments.append(comment)
16:25:02 <jeblair> (the story.addComment() method would do: comment=Comment(...); session.add(comment); self.comments.append(comment))
16:25:32 <NikitaKonovalov> jeblair: I like that approach
16:25:57 <NikitaKonovalov> looks like it's time for a huge db_api refactoring
16:26:09 * ricky1 hopes to find some time to read up a SQLA book on Safari
16:26:24 <jeblair> #link http://git.openstack.org/cgit/stackforge/gertty/tree/gertty/db.py
16:26:25 <krotscheck> I feel like NikitaKonovalov’s the best qualified for that, since he’s doing something very similar in the python API library
16:26:43 <jeblair> that ^ shows the approach and mostly isn't terrible.  maybe just a little.  :)
16:27:35 <NikitaKonovalov> or I can ask aripinen to do that when he finishes refresh tokens and other stuff
16:27:38 <krotscheck> jeblair: We can improve on it :)
16:28:21 <krotscheck> NikitaKonovalov: Ok, so it can be either yours or his task, but if we’re going to dramatically change how the DB api works, do you think we’ll need a spec to set some general guidelines on how we do references?
16:28:59 <NikitaKonovalov> krotscheck: yes I can make a spec for that
16:29:17 <ricky1> given the wide scope, i agree a spec would be good
16:29:45 <krotscheck> Well, do we need one?
16:29:53 <krotscheck> Ooookay, n/m. Spec it is.
16:30:03 <krotscheck> #action NikitaKonovalov Write a spec for DB APi refactor.
16:30:38 <NikitaKonovalov> fine
16:30:49 <krotscheck> Cool, moving on.
16:31:04 <krotscheck> No user feedback in the agenda, so we’ll skip that.
16:31:13 <krotscheck> #topic Discussion: Google Summer of Code.
16:31:41 <krotscheck> So, the OpenStack call for GSoC proposals just went out.
16:31:44 * NikitaKonovalov needs to read through that
16:31:54 <yolanda> yep, met oo
16:32:11 <krotscheck> And I figure StoryBoard could be a good candidate, if we can find a nice, independent feature chunk that we could put forward as a proposal.
16:32:54 <yolanda> oh , that sounds good
16:32:57 <krotscheck> Does anyone have any ideas?
16:33:08 <ricky1> yolanda: you already spoke about HP Women Scholarship program
16:33:19 <ricky1> not GSOC, but related in terms of getting someone to do SB stuff...
16:33:20 <yolanda> yes, gulic is going to take the file attachments feature
16:33:24 <krotscheck> ricky1: Yep, meet gulic  :D
16:33:31 <yolanda> i talked with ttx and guli about that
16:33:32 <krotscheck> oooOOOoooo
16:33:44 <ricky1> excellent!
16:33:52 <yolanda> guli=gulic.. i hate my keyboard
16:34:05 * krotscheck likes file attachments.
16:34:32 <NikitaKonovalov> file attachmets are also worth having a spec I think
16:34:39 <krotscheck> Yep.
16:34:53 <yolanda> yes, i was talking with ttx about that, we can work with gulic and she can define it
16:35:00 <yolanda> so ttx is not present today, he told he was flying
16:35:15 <krotscheck> That suggests a GSOC topic. We could build up some of the toolking which the UX team asked for into storyboard as a GSoC project.
16:35:17 <yolanda> but we can talk more about it this week
16:35:36 * krotscheck tries to steer the conversation back to the meeting topic.
16:36:11 <NikitaKonovalov> krotscheck: sounds good
16:36:33 <jeblair> krotscheck: that sounds like a great idea
16:37:16 <krotscheck> I agree. Does anyone else have feature groupings? I’d like to put this forward to the lists in general to see if anyone else has ideas.
16:37:38 * NikitaKonovalov needs to think more on that
16:38:10 <krotscheck> Alright, we’ll keep it on the agenda for next week. Everyone ponder it.
16:38:21 <yolanda> yes, i need to think about it as well
16:39:20 <krotscheck> #topic InProgress(krotscheck)
16:39:43 <krotscheck> What I didn’t spend on datetime last week I spent on two things.
16:40:16 <krotscheck> The first was to fix all the problems in the webclient that made yolanda jump through ridiculous hoops to make user preferences work.
16:40:36 <yolanda> ep, my hoops weren't ridiculous :)
16:40:38 <krotscheck> That more or less starts here: https://review.openstack.org/#/c/155943/
16:40:42 <yolanda> or they were ...
16:40:48 <krotscheck> yolanda: Yes, but they could be SO MUCH EASIER.
16:41:08 <yolanda> i like what you've done there
16:41:37 <krotscheck> The whole recursive link between user -> session -> resourcefactory -> user was ridiculous and unecessary.
16:42:18 <krotscheck> The other thing I worked on was email, which jeblair had raised some concerns with but I’m not really ready to discuss yet.
16:42:31 <krotscheck> But I should be ready by next week, so I’ll put that on the agenda.
16:43:08 <krotscheck> rcarrilocruz is not here, we’ll skip him.
16:43:11 <ricky1> yeah
16:43:12 <ricky1> !
16:43:16 <krotscheck> Oh!
16:43:30 <ricky1> bouncer is giving me grief, rcarrillocruz is in zombie...
16:43:33 <ricky1> so
16:43:33 <krotscheck> Gotcha
16:43:36 <ricky1> streaming!
16:43:42 <krotscheck> #topic InProgress(ricky1)
16:43:46 <ricky1> i figured the reconnect thing
16:43:57 <ricky1> https://review.openstack.org/#/c/143304/13/storyboard/streaming/consumer.py
16:44:22 <ricky1> now, we create a queue when the websocket is opened and we set the attribute x-expires to 300000
16:44:33 <ricky1> that means if there's no consumer consuming it, it will be dead
16:44:50 <ricky1> and put a 'reconnect' command to the SB WS protocol
16:45:01 <ricky1> let's say our client connects to Tornado
16:45:07 <ricky1> and subscribes to things like tasks or stories
16:45:13 <ricky1> then suddenly connection drops
16:45:25 <ricky1> the queue still is around keeping storing events
16:45:55 <ricky1> then the client can do in the next 5 minutes a 'reconnect <string>' and that gets bound to the previous queue, making all the undelivered messages to be sent on the WS ...
16:46:17 <ricky1> the string thing i'm not sure what convention to follow, right now i'm thinking that we sent to the client the name of the rabbitmq queue name on connect
16:46:24 <ricky1> so they can use later as the reconnect argument
16:47:01 <ricky1> following that i will tackle the 'replay' command: to get events from an arbitrary date range, that will not go thru RabbitMQ, it will just be a DB lookup
16:47:05 <krotscheck> ricky1: When a client first makes a connection, can we 303 them to a different url that only goes to their queue?
16:47:37 <krotscheck> i.e. Connect /v1/socket   -> yay but it’s over here -> 303 /v1/socket/124lfdkskajlsfalkf
16:47:46 <ricky1> websockets only use HTTP on handshake
16:47:55 <ricky1> after that , you can't really use 303 or other HTTP verbs
16:48:34 <krotscheck> So we can’t redirect during the handshake? Suck.
16:48:42 <ricky1> niet
16:49:27 <krotscheck> Ok, so do you have a solution, or do you want us to brainstorm something?
16:50:11 <ricky1> the idea i had to prevent users to reconnect to queues that were not opened by themselves is to keep a dict/shelve of queue-names-connect strings
16:50:25 <ricky1> i.e. you connect to SB Tornado, authenticate ok with your bearer token
16:50:39 <ricky1> we send the client a string like 'queue-234234:connectstring'
16:50:48 <ricky1> and that pair can be used later on by the client to reconnect
16:50:57 <ricky1> what you think?
16:51:17 <krotscheck> That seems reasonably sane. Are there similar examples out there?
16:51:50 <ricky1> not that i've found tbh, most rabbitmq->tornado examples use cookies to persist data
16:51:57 <ricky1> that's also something we could pursue
16:52:09 <ricky1> to put that 'session' on a cookie and client can send on reconnect?
16:53:02 <krotscheck> Also an option.
16:53:23 <ricky1> otoh, doing cookies kind of leaves clients to always be browsers
16:53:41 <ricky1> i'd like to have this feature to be browsers, CLI clients, etc
16:53:42 <krotscheck> Yeah, I think that disqualifies it.
16:54:48 <krotscheck> We need to move on to others, how about yo umove this to discussion for next week and present a list of options?
16:54:54 <ricky1> sure
16:54:57 <ricky1> makes sense
16:55:02 <krotscheck> #topic InProgress(yolanda)
16:55:04 <krotscheck> Yi!
16:55:06 <ricky1> thx
16:55:06 <krotscheck> Hi!
16:55:08 <yolanda> so...
16:55:09 <krotscheck> You have 5 minutes :)
16:55:11 <yolanda> integration tests
16:55:15 <yolanda> i started working on that
16:55:26 <yolanda> i create a tooling to start and stop backend for testing
16:55:34 <yolanda> https://review.openstack.org/155975
16:55:51 <ricky1> weee
16:55:53 <ricky1> nice
16:55:53 <yolanda> and most important, i proposed some initial job https://review.openstack.org/156044
16:56:00 <yolanda> but i need jeblair feedback on that, about zuul-cloner
16:56:18 <krotscheck> NICE
16:56:23 <jeblair> yolanda: i should be able to do that tomorrow!
16:56:31 <yolanda> ok, thx, i left some comments on the change
16:56:40 <yolanda> Ajaeger also reviewed the change
16:56:56 <yolanda> i set the tests as non-voting but seems that for gate, all tests need to pass
16:57:12 <yolanda> so we either rely on check-* at the moment, or we use some experimental pipeline
16:57:20 <yolanda> what do you think?
16:57:36 <jeblair> yolanda: just put the nonvoting job in check, then when you are ready to make it voting, add it to the gate in the same change
16:57:39 <yolanda> maybe just a check and we ensure that everything makes sense?
16:57:47 <yolanda> ok
16:58:13 <yolanda> and krotscheck, i created some initial POC for the integration tests, but i'm sure you can give better advice :)
16:58:20 <yolanda> https://review.openstack.org/156217
16:58:34 <krotscheck> On it!
16:58:40 <yolanda> basically using protractor to test real api requests
16:58:51 * krotscheck loves all the words in that sentence.
16:59:28 <yolanda> i was reading a lot about TDD, karma and protractor this weekend on Safari
16:59:30 <krotscheck> Alright, we’re ready to wrap up.
16:59:32 <yolanda> i really love that subscription
16:59:35 <yolanda> ok, latest one
16:59:41 <yolanda> https://review.openstack.org/155242
16:59:47 <yolanda> this needs another +2 ^
17:00:06 <krotscheck> NikitaKonovalov: you’re on deck for htat.
17:00:11 <krotscheck> I’ve already _2’d it :)
17:00:13 <krotscheck> Thanks everyone!
17:00:14 <krotscheck> #endmeeting StoryBoard