| *** tnovacik has quit IRC | 00:38 | |
| *** Poornima has joined #openstack-sahara | 02:32 | |
| *** chandan_kumar has joined #openstack-sahara | 02:44 | |
| *** tellesnobrega_ has joined #openstack-sahara | 02:59 | |
| openstackgerrit | Ken Chen proposed openstack/sahara: [CDH] Convert node group config dict https://review.openstack.org/137987 | 03:12 |
|---|---|---|
| *** tellesnobrega_ has quit IRC | 04:05 | |
| *** Poornima has quit IRC | 04:14 | |
| *** chandan_kumar has quit IRC | 04:39 | |
| *** Poornima has joined #openstack-sahara | 04:46 | |
| *** chandankumar has joined #openstack-sahara | 04:59 | |
| *** wangbo has joined #openstack-sahara | 06:07 | |
| *** Longgeek has joined #openstack-sahara | 06:21 | |
| *** Longgeek has quit IRC | 06:25 | |
| *** Longgeek has joined #openstack-sahara | 06:53 | |
| *** Longgeek has quit IRC | 06:54 | |
| *** Longgeek has joined #openstack-sahara | 06:54 | |
| *** Longgeek has quit IRC | 06:55 | |
| *** Longgeek has joined #openstack-sahara | 06:55 | |
| *** Longgeek has quit IRC | 06:57 | |
| *** Longgeek_ has joined #openstack-sahara | 06:57 | |
| *** Longgeek_ has quit IRC | 06:59 | |
| *** Longgeek has joined #openstack-sahara | 07:01 | |
| *** Longgeek has quit IRC | 07:02 | |
| *** tnovacik has joined #openstack-sahara | 07:18 | |
| *** k4n0 has joined #openstack-sahara | 07:24 | |
| *** Longgeek has joined #openstack-sahara | 08:03 | |
| *** Longgeek has quit IRC | 08:08 | |
| *** witlessb has joined #openstack-sahara | 08:39 | |
| *** ekarlso- has quit IRC | 08:52 | |
| *** Longgeek has joined #openstack-sahara | 08:57 | |
| *** skolekonov has joined #openstack-sahara | 09:02 | |
| *** Longgeek_ has joined #openstack-sahara | 09:08 | |
| *** ekarlso- has joined #openstack-sahara | 09:09 | |
| *** Longgeek_ has quit IRC | 09:09 | |
| *** Longgeek_ has joined #openstack-sahara | 09:09 | |
| *** Longgeek has quit IRC | 09:10 | |
| openstackgerrit | Denis Egorenko proposed stackforge/sahara-ci-config: Add extra-index-url for PYPI https://review.openstack.org/138016 | 09:27 |
| *** tellesnobrega_ has joined #openstack-sahara | 09:31 | |
| *** IvanBerezovskiy has left #openstack-sahara | 09:40 | |
| *** IvanBerezovskiy has joined #openstack-sahara | 09:41 | |
| *** tellesnobrega_ has quit IRC | 09:42 | |
| *** openstackgerrit has quit IRC | 09:50 | |
| *** openstackgerrit has joined #openstack-sahara | 09:50 | |
| *** Longgeek_ has quit IRC | 09:57 | |
| *** Longgeek has joined #openstack-sahara | 09:58 | |
| *** tellesnobrega_ has joined #openstack-sahara | 10:05 | |
| *** wangbo has quit IRC | 10:11 | |
| *** Longgeek_ has joined #openstack-sahara | 10:15 | |
| *** Longgeek has quit IRC | 10:18 | |
| *** chandankumar has quit IRC | 10:25 | |
| *** tellesnobrega_ has quit IRC | 10:35 | |
| *** chandankumar has joined #openstack-sahara | 10:39 | |
| *** tellesnobrega_ has joined #openstack-sahara | 10:51 | |
| *** Longgeek_ has quit IRC | 11:00 | |
| *** tellesnobrega_ has quit IRC | 11:02 | |
| *** vrovachev has quit IRC | 11:04 | |
| *** Longgeek has joined #openstack-sahara | 11:09 | |
| *** chandankumar has quit IRC | 11:42 | |
| *** chandankumar has joined #openstack-sahara | 11:43 | |
| *** Longgeek has quit IRC | 11:50 | |
| *** sreshetnyak has joined #openstack-sahara | 11:53 | |
| *** Longgeek has joined #openstack-sahara | 11:53 | |
| *** tnovacik has quit IRC | 11:55 | |
| openstackgerrit | Adrien Vergé proposed openstack/sahara: Inherit Context from oslo https://review.openstack.org/137458 | 12:29 |
| *** chandankumar has quit IRC | 13:06 | |
| openstackgerrit | Ken Chen proposed openstack/sahara: Specify CDH version https://review.openstack.org/137879 | 13:25 |
| *** Poornima has quit IRC | 13:31 | |
| *** egafford has joined #openstack-sahara | 13:56 | |
| *** _crobertsrh is now known as crobertsrh | 14:27 | |
| *** Longgeek_ has joined #openstack-sahara | 14:32 | |
| *** Longgeek has quit IRC | 14:33 | |
| *** tmckay has joined #openstack-sahara | 14:53 | |
| openstackgerrit | Merged openstack/sahara: [CDH] Convert node group config dict https://review.openstack.org/137987 | 14:59 |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Open all ports for private network for auto SG https://review.openstack.org/136083 | 15:15 |
| *** k4n0 has quit IRC | 15:16 | |
| *** jayunit1000 has quit IRC | 15:20 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack/sahara: Add list of open ports for Spark plugin https://review.openstack.org/134927 | 15:22 |
| openstackgerrit | Merged openstack/sahara: Add db/conductor ops to work with new events objects https://review.openstack.org/133458 | 15:25 |
| openstackgerrit | Merged openstack/sahara: Sample JSON files for Sahara EDP APIs https://review.openstack.org/136530 | 15:31 |
| openstackgerrit | Merged stackforge/sahara-ci-config: Credentials source refactoring https://review.openstack.org/135582 | 16:12 |
| openstackgerrit | Chad Roberts proposed openstack/sahara-specs: Updating query filtering support for Sahara https://review.openstack.org/136865 | 16:15 |
| *** skolekonov has quit IRC | 16:21 | |
| crobertsrh | tmckay: bah....found another special case | 16:35 |
| crobertsrh | slightly different special case this time.... | 16:35 |
| crobertsrh | For Job Executions, filtering by Status | 16:35 |
| tmckay | ok | 16:35 |
| crobertsrh | status is actually part of "info" | 16:36 |
| tmckay | okay, I'll take a look | 16:36 |
| crobertsrh | I wouldn't bother, but it seems like filtering by status might be useful | 16:36 |
| *** Networkn3rd has joined #openstack-sahara | 16:41 | |
| *** chandankumar has joined #openstack-sahara | 16:54 | |
| *** Longgeek_ has quit IRC | 17:11 | |
| *** chandankumar has quit IRC | 17:28 | |
| *** stannie has quit IRC | 17:29 | |
| *** tellesnobrega_ has joined #openstack-sahara | 17:31 | |
| *** ylobankov has joined #openstack-sahara | 17:42 | |
| *** Longgeek has joined #openstack-sahara | 17:43 | |
| *** Longgeek has quit IRC | 17:52 | |
| openstackgerrit | Merged openstack/sahara: Adding uuids to exceptions https://review.openstack.org/136826 | 17:55 |
| crobertsrh | SergeyLukjanov: Are there plans to push a new python-saharaclient version before kilo-1? I need it before I can get my UI filtering patch reviews going....and we know how long horizon reviews can take. | 18:05 |
| openstackgerrit | Andrew Lazarev proposed openstack/sahara: Fixed auto security group for nova network https://review.openstack.org/137201 | 18:15 |
| openstackgerrit | Merged openstack/sahara-image-elements: Image Elements for MapR https://review.openstack.org/126148 | 18:25 |
| crobertsrh | Question: I intend to write-up a blueprint for adding edit capabilities to templates. Do I also need to write a spec for this? Do we need a spec for all blueprints? Or is there some point where a bp can exist without a spec? | 18:32 |
| tmckay | crobertsrh, specs were invented because blueprints were too sparse on details | 18:33 |
| crobertsrh | right | 18:33 |
| crobertsrh | I recall that, but if we're just doing a 1:1 spec:blueprint ratio, bps are essentially useless, right? | 18:33 |
| crobertsrh | Since they supposedly lack detail | 18:33 |
| tmckay | crobertsrh, well, bps catch all the commits | 18:34 |
| crobertsrh | So, bps are just commit catchers? | 18:34 |
| tmckay | If you put enough detail in the bp, you could skip the spec. But, as elmiko pointed out, we can set up the spec repo to carry specs from milestone to milestone if they don't get implemented | 18:35 |
| crobertsrh | BPs can already be moved from milestone to milestone | 18:35 |
| tmckay | I look at them as general overviews, managed by launchpad, with milestone fields, keywords, etc | 18:35 |
| tmckay | specs give us something more flexible that we can control from a repo | 18:36 |
| crobertsrh | I'm not against more documented details....I'm just not seeing the huge benefit of having both. | 18:36 |
| crobertsrh | I'm only thinking about this today because I'm doing a horizon blueprint....in horizon, they just have a template for bps that must be followed (essentially similar to our spec template) | 18:37 |
| tmckay | major advantage of bp is searchiing in launchpad and finding associated reviews. | 18:37 |
| tmckay | yes, I could see that working | 18:37 |
| tmckay | bp is not approved unless the template is filled in | 18:37 |
| tmckay | biggest issue I've found is there is not a good way to comment on a BP | 18:37 |
| crobertsrh | So, naturally, I'm jaded when I return "home" to sahara and have to create 2 docs for a single change | 18:37 |
| crobertsrh | fair enough. specs are better for comments, that I agree with. | 18:38 |
| tmckay | my current patter is a very light bp, like 2 lines. Then set the spec URL to the review of the spec | 18:39 |
| tmckay | a lot of people don't set the spec URL, it bugs me | 18:39 |
| crobertsrh | Right. | 18:39 |
| crobertsrh | Ok, I'll take off my grumpy pants for now. Thanks for indulging me. | 18:39 |
| * crobertsrh may or may not replace grumpy pants with other pants. | 18:40 | |
| tmckay | oh, so on this status thing -- info is stored as a string in sqlalchemy. So, I could use filter.like(blahblah), but a big chunk of info is opaque. It's just filled in by the client | 18:40 |
| tmckay | the trouble is that Oozie for example uses 'status: blah' in 'actions' that get listed in 'info' | 18:41 |
| crobertsrh | oooh | 18:41 |
| tmckay | since I don't know what else will be in info, and since I can't be sure of the dictionary order, I can't search for 'status' with a query | 18:41 |
| tmckay | I have to filter it after the string is turned into a dictionary | 18:41 |
| crobertsrh | that's kinda ugly | 18:42 |
| tmckay | not a big deal, but maybe not particularly efficient | 18:42 |
| crobertsrh | No other good way of doing it though | 18:42 |
| tmckay | yeah, info is a catchall for "whatever the client said" plus our own stuff on top | 18:42 |
| crobertsrh | What should the magic syntax be for searching the Status field? | 18:43 |
| tmckay | I would just search on "status" | 18:43 |
| tmckay | It's another special field, but ... unless we add something to job_executions called 'status', it should be clear enough | 18:44 |
| crobertsrh | Ok, so horizon-wise, it will not be a special case. I can certainly handle that :) | 18:45 |
| crobertsrh | Do you just want to ammend your "special case" CR, or do you want to fire up a new one for this? | 18:45 |
| tmckay | I might just add a new one. You know, in hindsight, I wonder why 'status' is in info | 18:46 |
| crobertsrh | I remember when I discovered that it was in info, but I don't recall why | 18:47 |
| tmckay | It started off as something reported by oozie, but we count on it. And for spark, we create it ourselves | 18:47 |
| crobertsrh | I thought we talked about moving it at one point | 18:47 |
| *** tellesnobrega_ has quit IRC | 18:53 | |
| *** tellesnobrega_ has joined #openstack-sahara | 18:55 | |
| tmckay | crobertsrh, yeah, I wonder if we should move it. It's this thing partially set by Sahara, partially set by Oozie. In the case of Spark we control it completely (and we don't know what storm will do) | 18:57 |
| crobertsrh | How big of a job is moving it? | 18:59 |
| tmckay | probably not too bad. The trouble is backward compatibility -- if something is counting on it being there. | 19:01 |
| crobertsrh | There is some stuff on the UI-side that specifically looks in 'info' for status, but it's easy to change. | 19:02 |
| tmckay | I wonder what tooling there might be around the values in the API | 19:03 |
| crobertsrh | looks like there is a place in the cli stuff that also does the same thing the UI does | 19:05 |
| tmckay | well, the json is what I meant | 19:05 |
| tmckay | hmm, it looks like it changes it a little | 19:06 |
| tmckay | well, when it prints | 19:06 |
| crobertsrh | yeah, nothing too horrible | 19:07 |
| tmckay | yeah, bad choice. We could do this -- we could duplicate the status field as its own field in the record, leave the old one | 19:07 |
| tmckay | call it job_status | 19:07 |
| tmckay | leave it there for a release in the spark stuff, etc, then remove it | 19:08 |
| tmckay | oozie would continue to add it | 19:08 |
| tmckay | but that's just oozie specific | 19:08 |
| tmckay | this way we could always guarantee that Sahara job status is consistent (if oozie changes something, it doesn't matter) | 19:08 |
| crobertsrh | That approach doesn't bother me too much. I can see where others might object, but I kinda like it. | 19:09 |
| tmckay | this is one of those oozie-specific things that leaked | 19:10 |
| tmckay | I think we need to move away from it | 19:10 |
| tmckay | we have 'info' and 'extra' in job executions, doh | 19:11 |
| crobertsrh | good point | 19:17 |
| tmckay | I think we probably need a review of job execution and how that stuff is being used, and a migration toward sanity | 19:20 |
| *** Longgeek has joined #openstack-sahara | 19:21 | |
| tmckay | for the moment, I can probably just add your search on status in the db layer | 19:21 |
| *** Longgeek has quit IRC | 19:22 | |
| openstackgerrit | Adrien Vergé proposed openstack/sahara: Inherit Context from oslo https://review.openstack.org/137458 | 19:25 |
| *** tmckay has quit IRC | 19:29 | |
| *** hsurti has joined #openstack-sahara | 19:48 | |
| SergeyLukjanov | crobertsrh, I'll release it today | 20:22 |
| SergeyLukjanov | crobertsrh, re client | 20:22 |
| crobertsrh | wow, instant service! Nice! | 20:22 |
| *** tmckay has joined #openstack-sahara | 20:33 | |
| *** tellesnobrega_ has quit IRC | 20:59 | |
| *** alexiz has joined #openstack-sahara | 21:20 | |
| *** egafford has quit IRC | 21:25 | |
| *** tellesnobrega_ has joined #openstack-sahara | 21:27 | |
| *** crobertsrh1 has joined #openstack-sahara | 21:28 | |
| *** egafford has joined #openstack-sahara | 21:28 | |
| *** shakayumi has quit IRC | 21:35 | |
| *** russellb has quit IRC | 21:35 | |
| *** tellesnobrega has quit IRC | 21:35 | |
| *** crobertsrh has quit IRC | 21:35 | |
| *** crobertsrh1 is now known as _crobertsrh | 21:41 | |
| *** egafford has quit IRC | 22:22 | |
| *** hsurti has quit IRC | 22:24 | |
| *** witlessb has quit IRC | 23:07 | |
| *** tellesnobrega__ has joined #openstack-sahara | 23:20 | |
| openstackgerrit | Andrew Lazarev proposed openstack/sahara: Added get_open_ports description to plugin SPI https://review.openstack.org/138203 | 23:30 |
| *** hdd has joined #openstack-sahara | 23:39 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!