| *** fvollero|gone has quit IRC | 00:04 | |
| *** jcru has quit IRC | 00:05 | |
| *** jamieh has quit IRC | 00:14 | |
| *** kgriffs is now known as kgriffs_afk | 00:18 | |
| *** kgriffs_afk is now known as kgriffs | 00:18 | |
| *** kgriffs is now known as kgriffs_afk | 00:32 | |
| *** kgriffs_afk is now known as kgriffs | 00:32 | |
| *** vkmc has quit IRC | 00:34 | |
| *** kgriffs is now known as kgriffs_afk | 00:40 | |
| *** nosnos has joined #openstack-marconi | 01:04 | |
| *** asalkeld has joined #openstack-marconi | 01:19 | |
| *** amitgandhi has joined #openstack-marconi | 01:22 | |
| *** asalkeld has quit IRC | 01:39 | |
| *** oz_akan_ has joined #openstack-marconi | 02:26 | |
| *** malini_afk has left #openstack-marconi | 02:44 | |
| *** oz_akan_ has quit IRC | 02:46 | |
| *** oz_akan_ has joined #openstack-marconi | 02:48 | |
| *** kgriffs_afk is now known as kgriffs | 02:50 | |
| *** kgriffs is now known as kgriffs_afk | 03:02 | |
| *** kgriffs_afk is now known as kgriffs | 03:03 | |
| *** kgriffs is now known as kgriffs_afk | 03:11 | |
| *** kgriffs_afk is now known as kgriffs | 03:11 | |
| *** oz_akan_ has quit IRC | 03:11 | |
| *** amitgandhi has quit IRC | 03:21 | |
| *** amitgandhi has joined #openstack-marconi | 03:22 | |
| *** amitgandhi has quit IRC | 03:26 | |
| *** kgriffs is now known as kgriffs_afk | 04:02 | |
| *** kgriffs_afk is now known as kgriffs | 04:02 | |
| *** kgriffs is now known as kgriffs_afk | 04:08 | |
| *** cpallares has quit IRC | 04:12 | |
| *** gleicon____ has joined #openstack-marconi | 04:16 | |
| *** amitgandhi has joined #openstack-marconi | 04:22 | |
| *** gleicon___ has quit IRC | 04:23 | |
| *** gleicon____ is now known as gleicon___ | 04:23 | |
| *** amitgandhi has quit IRC | 04:27 | |
| *** amitgandhi has joined #openstack-marconi | 05:23 | |
| *** amitgandhi has quit IRC | 05:28 | |
| *** amitgandhi has joined #openstack-marconi | 06:24 | |
| *** amitgandhi has quit IRC | 06:31 | |
| *** amitgandhi has joined #openstack-marconi | 07:27 | |
| *** kgriffs_afk is now known as kgriffs | 07:31 | |
| *** amitgandhi has quit IRC | 07:32 | |
| *** kgriffs is now known as kgriffs_afk | 07:41 | |
| *** westmaas_ is now known as westmaas | 08:06 | |
| *** kgriffs_afk is now known as kgriffs | 08:21 | |
| *** ani has joined #openstack-marconi | 08:24 | |
| *** amitgandhi has joined #openstack-marconi | 08:27 | |
| ani | \aniuskad | 08:29 |
|---|---|---|
| *** ani has quit IRC | 08:29 | |
| *** amitgandhi has quit IRC | 08:32 | |
| *** kgriffs is now known as kgriffs_afk | 08:54 | |
| *** kgriffs_afk is now known as kgriffs | 08:54 | |
| *** yassine has joined #openstack-marconi | 08:57 | |
| *** jamieh has joined #openstack-marconi | 08:59 | |
| *** kgriffs is now known as kgriffs_afk | 09:03 | |
| *** amitgandhi has joined #openstack-marconi | 09:28 | |
| *** amitgandhi has quit IRC | 09:33 | |
| *** kgriffs_afk is now known as kgriffs | 09:41 | |
| *** nosnos has quit IRC | 09:47 | |
| *** kgriffs is now known as kgriffs_afk | 10:03 | |
| *** amitgandhi has joined #openstack-marconi | 10:29 | |
| *** amitgandhi has quit IRC | 10:34 | |
| *** amitgandhi has joined #openstack-marconi | 11:29 | |
| *** amitgandhi has quit IRC | 11:34 | |
| *** vkmc has joined #openstack-marconi | 12:04 | |
| *** vkmc has quit IRC | 12:04 | |
| *** vkmc has joined #openstack-marconi | 12:04 | |
| *** amitgandhi has joined #openstack-marconi | 12:30 | |
| *** malini has joined #openstack-marconi | 12:32 | |
| *** amitgandhi has quit IRC | 12:35 | |
| *** tedross has joined #openstack-marconi | 12:55 | |
| *** malini is now known as malini_afk | 13:08 | |
| *** oz_akan_ has joined #openstack-marconi | 14:03 | |
| *** oz_akan_ has quit IRC | 14:04 | |
| *** oz_akan_ has joined #openstack-marconi | 14:05 | |
| *** alcabrera has joined #openstack-marconi | 14:06 | |
| *** amitgandhi has joined #openstack-marconi | 14:06 | |
| *** tedross has quit IRC | 14:07 | |
| *** tedross has joined #openstack-marconi | 14:07 | |
| *** alcabrera has quit IRC | 14:15 | |
| *** amitgandhi has quit IRC | 14:17 | |
| *** amitgandhi has joined #openstack-marconi | 14:18 | |
| *** amitgandhi has quit IRC | 14:22 | |
| *** malini_afk is now known as malini | 14:22 | |
| *** jcru has joined #openstack-marconi | 14:32 | |
| *** amitgandhi has joined #openstack-marconi | 14:33 | |
| *** amitgandhi has quit IRC | 14:39 | |
| *** amitgandhi has joined #openstack-marconi | 14:40 | |
| *** kgriffs_afk is now known as kgriffs | 15:07 | |
| *** alcabrera has joined #openstack-marconi | 15:28 | |
| *** kgriffs is now known as kgriffs_afk | 15:30 | |
| *** kgriffs_afk is now known as kgriffs | 15:30 | |
| *** alcabrera has quit IRC | 15:53 | |
| *** jcru has quit IRC | 15:54 | |
| *** amitgandhi has quit IRC | 16:03 | |
| *** amitgandhi has joined #openstack-marconi | 16:04 | |
| *** alcabrera has joined #openstack-marconi | 16:09 | |
| *** amitgandhi has quit IRC | 16:09 | |
| alcabrera | kgriffs: ping | 16:14 |
| kgriffs | o/ | 16:14 |
| alcabrera | How goes the openstack summit? :D | 16:14 |
| kgriffs | great! | 16:14 |
| kgriffs | ...except for the jet lag. :p | 16:14 |
| alcabrera | heh. :P | 16:14 |
| alcabrera | I need your input on a sharding patch, if you have a spare moment. (5 min?) | 16:15 |
| kgriffs | sure | 16:15 |
| alcabrera | L417: https://review.openstack.org/#/c/54945/3/marconi/queues/storage/sharding.py | 16:15 |
| alcabrera | why was the 'if not exists(...)' check reverted? | 16:16 |
| alcabrera | s/reverted/removed | 16:16 |
| * kgriffs is thinking | 16:16 | |
| alcabrera | The queue re-registration problem cropped up today during testing. ;) | 16:17 |
| kgriffs | TBH, I don't remember there being a good reason | 16:18 |
| kgriffs | it may have been accidental | 16:18 |
| alcabrera | kk - just wanted to make sure. | 16:19 |
| alcabrera | I was going to fix things up today and wanted to make sure I wasn't missing something. | 16:19 |
| kgriffs | hmm | 16:19 |
| kgriffs | so, I was just looking at L163, where that register is called | 16:23 |
| kgriffs | and yeah, it doesn't check for exists there | 16:23 |
| kgriffs | seems like someone should! | 16:23 |
| *** amitgandhi has joined #openstack-marconi | 16:24 | |
| *** amit has joined #openstack-marconi | 16:24 | |
| kgriffs | alcabrera: I'm pretty sure that change was accidental | 16:24 |
| alcabrera | thanks for the double check. :) | 16:25 |
| *** amit is now known as Guest6544 | 16:25 | |
| kgriffs | I must have thought the code was already like that; otherwise I wouldn't have written the comment starting on L424 | 16:25 |
| *** Guest6544 has quit IRC | 16:25 | |
| kgriffs | alcabrera: anything else I can help with? | 16:26 |
| alcabrera | eom:bastion - hmm... I'll be taking some time in a few hours (~3h) to finalize that patch. Another review on that would be nice. oz_akan_ would love to see that land soon. | 16:27 |
| alcabrera | s/nice/awesome | 16:27 |
| alcabrera | :) | 16:27 |
| oz_akan_ | alcabrera: +1 | 16:27 |
| alcabrera | Oh yeah, one more thing... | 16:27 |
| alcabrera | kgriffs: keep making the most of the summit! :D | 16:28 |
| *** amitgandhi has quit IRC | 16:28 | |
| *** amitgandhi has joined #openstack-marconi | 16:28 | |
| kgriffs | :) | 16:28 |
| kgriffs | alcabrera: btw, I was trying to come up with a "realistic" example for queue metadata to put on a slide | 16:29 |
| kgriffs | I'm having trouble thinking of a good use of metadata, tbh. :p | 16:29 |
| kgriffs | queue options, yet, but metadata? | 16:29 |
| alcabrera | kgriffs: I don't think metadata will really stand out until we have... | 16:29 |
| alcabrera | tag filtering | 16:30 |
| kgriffs | if you have any brainstorms let me know. | 16:30 |
| alcabrera | content filtering | 16:30 |
| kgriffs | so maybe you want to filter the queue list by what is in the metadata? | 16:30 |
| alcabrera | yup | 16:30 |
| alcabrera | for example | 16:30 |
| alcabrera | in a dynamic discovery setting | 16:30 |
| alcabrera | you could have a set of queues to process jpegs, pngs, gifs | 16:30 |
| alcabrera | and the metadata is used as the queue type | 16:31 |
| alcabrera | and workers would "discover" these queues somehow | 16:31 |
| alcabrera | it's cleaner than encoding the "image type" in the queue name | 16:31 |
| alcabrera | and you could also encode other properties, like - max_image_size, minimum_image_size | 16:32 |
| alcabrera | etc. | 16:32 |
| * kgriffs is thinking about that | 16:32 | |
| kgriffs | if metadata didn't exist, how would this application have to be implemented? | 16:32 |
| alcabrera | encode the information in the queue name somehow | 16:33 |
| alcabrera | jpeg_100_200_1, jpeg_100_200_2, gif_200_400_1, ... | 16:33 |
| alcabrera | or solve the problem outside of the queuing framwork | 16:34 |
| alcabrera | *framework | 16:34 |
| alcabrera | by having a filter worker that maintains a static mapping: "these are gif queues", "these are png queues", "that image is too big for any of the queues" | 16:34 |
| alcabrera | Hmm... | 16:36 |
| alcabrera | Given that we have LIST queues?detailed=True that also yields the metadata, the cost of dynamic discovery in this fashion is similar to the cost of queue name introspection. | 16:36 |
| alcabrera | Only - there's possibly better separation of concerns. | 16:37 |
| alcabrera | (a name is metadata, after all). | 16:37 |
| kgriffs | what about this workflow? | 16:40 |
| kgriffs | 1. a user uploads a video file | 16:40 |
| kgriffs | 2. the web head puts it to swift and then posts a message to a feeder queue | 16:41 |
| kgriffs | 3. a pool of routing workers pulls the message off, decides how big the file is, what it's media type is | 16:42 |
| kgriffs | maybe also checks to see if the customer is paying | 16:42 |
| kgriffs | 4. Based on the above, routes to one of X job queues | 16:42 |
| kgriffs | the router would auto-discover the queues using metadata at boot | 16:42 |
| kgriffs | so the router doesn't have to look that up somewhere | 16:43 |
| kgriffs | something like that. :p | 16:43 |
| alcabrera | works for me. :) | 16:43 |
| alcabrera | I feel like the key idea we're arriving at is that metadata simplifies auto-discovery and filtering. | 16:43 |
| kgriffs | ok. Let's keep thinking about use cases. I want to make sure metadata is actually useful, and not just there because "everyone else was doing it" | 16:43 |
| alcabrera | annotations, types == metadata | 16:44 |
| alcabrera | s/,/== | 16:44 |
| kgriffs | good thinking | 16:44 |
| kgriffs | thanks! | 16:44 |
| alcabrera | while not necessary, it can be put to good use | 16:44 |
| alcabrera | np. :) | 16:44 |
| alcabrera | (much like python3 annotations) | 16:44 |
| oz_akan_ | kgriffs: hi | 16:59 |
| oz_akan_ | what was the story about "failed to cache shard id" ? | 17:00 |
| kgriffs | hi | 17:00 |
| kgriffs | hmmm | 17:00 |
| kgriffs | so, there isn't a lot to the story | 17:00 |
| kgriffs | it was working, then suddenly stopped working. Basically, we ask oslo cache to set a value | 17:01 |
| kgriffs | and it returns "False" | 17:01 |
| kgriffs | meaning, it failed. | 17:01 |
| kgriffs | if you look at the memcached driver | 17:01 |
| kgriffs | sorry, I mean backend, for the cache module | 17:02 |
| kgriffs | I think it just calls python-memcached and that library fails silently | 17:02 |
| kgriffs | let me take a look at the code | 17:02 |
| kgriffs | woah, wait a second | 17:04 |
| kgriffs | alcabrera: take a look in common.cache._backends.memcached:58 | 17:05 |
| * alcabrera stops and looks | 17:05 | |
| alcabrera | def set(self, key, value, ttl=0): | 17:06 |
| alcabrera | alright, let's see... | 17:06 |
| kgriffs | now look at the base class | 17:07 |
| kgriffs | the contract says it should return True or False | 17:07 |
| kgriffs | ?! | 17:07 |
| alcabrera | Oh, ohhhhhh | 17:07 |
| alcabrera | I see. | 17:07 |
| alcabrera | Yikes | 17:07 |
| kgriffs | yeah | 17:07 |
| alcabrera | There's no 'return' | 17:08 |
| alcabrera | So it always returns None. | 17:08 |
| alcabrera | And... | 17:08 |
| kgriffs | exactly | 17:08 |
| alcabrera | Let's see the implications... | 17:08 |
| alcabrera | In the case where we expect a True, it'll treat it like a false. :/ | 17:08 |
| kgriffs | yep | 17:08 |
| kgriffs | so we always think it failed, when it didn't | 17:08 |
| kgriffs | oz_akan_: ^^^ | 17:09 |
| alcabrera | lameness | 17:09 |
| alcabrera | good catch, kgriffs! | 17:09 |
| kgriffs | it may have already been fixed upstream | 17:09 |
| kgriffs | nope | 17:10 |
| kgriffs | https://review.openstack.org/#/c/42878/5/openstack/common/cache/_backends/memcached.py | 17:10 |
| alcabrera | :( | 17:10 |
| kgriffs | let me just add a comment there... | 17:10 |
| alcabrera | sweet | 17:10 |
| kgriffs | can you fix it in our version? | 17:10 |
| kgriffs | s/can/will | 17:10 |
| kgriffs | ;) | 17:10 |
| oz_akan_ | kgriffs: at the moment, can we ignore that error message then/ | 17:11 |
| alcabrera | yup, I can patch it locally and see what else it is affecting. | 17:11 |
| alcabrera | np | 17:11 |
| kgriffs | kewl | 17:12 |
| kgriffs | thanks! | 17:12 |
| *** yassine has quit IRC | 17:13 | |
| *** alcabrera is now known as alcabrera|afk | 17:18 | |
| *** amitgandhi has quit IRC | 17:59 | |
| *** amitgandhi has joined #openstack-marconi | 18:00 | |
| *** vkmc has quit IRC | 18:00 | |
| *** amitgandhi has quit IRC | 18:04 | |
| *** amitgandhi has joined #openstack-marconi | 18:16 | |
| *** amitgandhi has quit IRC | 18:17 | |
| *** amitgandhi has joined #openstack-marconi | 18:17 | |
| *** malini is now known as malini_afk | 18:21 | |
| *** cpallares has joined #openstack-marconi | 18:24 | |
| *** gleicon___ has quit IRC | 18:25 | |
| *** Alex_Gaynor has quit IRC | 18:25 | |
| *** Alex_Gaynor has joined #openstack-marconi | 18:34 | |
| kgriffs | ttfn. It's getting *really* late in HKG | 18:36 |
| *** kgriffs is now known as kgriffs_afk | 18:45 | |
| *** gleicon___ has joined #openstack-marconi | 19:01 | |
| *** cpallares has quit IRC | 19:05 | |
| *** jamieh has quit IRC | 19:13 | |
| *** amitgandhi has quit IRC | 19:16 | |
| *** amitgandhi has joined #openstack-marconi | 19:17 | |
| *** jamieh has joined #openstack-marconi | 19:21 | |
| *** amitgandhi has quit IRC | 19:22 | |
| *** alcabrera|afk is now known as alcabrera | 19:27 | |
| *** amitgandhi has joined #openstack-marconi | 19:30 | |
| *** vkmc has joined #openstack-marconi | 19:49 | |
| *** vkmc has quit IRC | 19:49 | |
| *** vkmc has joined #openstack-marconi | 19:49 | |
| *** jburkhar1 has joined #openstack-marconi | 20:00 | |
| *** vkmc has quit IRC | 20:02 | |
| *** vkmc has joined #openstack-marconi | 20:09 | |
| *** vkmc has quit IRC | 20:09 | |
| *** vkmc has joined #openstack-marconi | 20:09 | |
| *** jburkhart has quit IRC | 20:09 | |
| *** briancline has quit IRC | 20:09 | |
| *** briancline has joined #openstack-marconi | 20:09 | |
| *** malini_afk is now known as malini | 20:10 | |
| *** cpallares has joined #openstack-marconi | 20:58 | |
| *** tedross has quit IRC | 22:05 | |
| *** alcabrera has quit IRC | 22:09 | |
| *** ekarlso- has quit IRC | 22:19 | |
| *** ekarlso has joined #openstack-marconi | 22:20 | |
| *** tedross has joined #openstack-marconi | 22:20 | |
| *** malini is now known as malini_afk | 22:36 | |
| *** tedross has quit IRC | 22:46 | |
| *** amitgandhi has quit IRC | 23:24 | |
| *** amitgandhi has joined #openstack-marconi | 23:24 | |
| *** oz_akan_ has quit IRC | 23:25 | |
| *** amitgandhi has quit IRC | 23:29 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!