| -openstackstatus- NOTICE: Zuul was restarted due to an error; events (such as approvals or new patchsets) since 23:01 UTC have been lost and affected changes will need to be rechecked | 00:06 | |
| *** saneax has quit IRC | 00:35 | |
| openstackgerrit | li,chen proposed openstack/sahara: Fix wrong compute nodes name in doc https://review.openstack.org/211806 | 01:42 |
|---|---|---|
| *** egafford has quit IRC | 02:07 | |
| openstackgerrit | lu huichun proposed openstack/python-saharaclient: Add job execution update https://review.openstack.org/208001 | 03:03 |
| *** uvirtbot has quit IRC | 03:25 | |
| *** htruta has quit IRC | 03:36 | |
| *** htruta has joined #openstack-sahara | 03:40 | |
| *** uvirtbot has joined #openstack-sahara | 03:46 | |
| *** Poornima has joined #openstack-sahara | 04:18 | |
| *** links has joined #openstack-sahara | 04:42 | |
| *** saneax has joined #openstack-sahara | 05:00 | |
| *** tmckay has quit IRC | 05:43 | |
| *** hdd has quit IRC | 05:53 | |
| *** tmckay has joined #openstack-sahara | 05:56 | |
| *** Poornima has quit IRC | 06:09 | |
| *** hdd has joined #openstack-sahara | 06:18 | |
| *** Nikolay_St has joined #openstack-sahara | 06:18 | |
| *** esikachev has joined #openstack-sahara | 06:22 | |
| *** Poornima has joined #openstack-sahara | 06:29 | |
| *** esikachev has quit IRC | 06:34 | |
| openstackgerrit | lu huichun proposed openstack/python-saharaclient: Add job execution update https://review.openstack.org/208001 | 06:57 |
| *** ig0r_ has joined #openstack-sahara | 07:14 | |
| *** hdd has quit IRC | 07:20 | |
| *** esikachev has joined #openstack-sahara | 07:29 | |
| *** esikachev has quit IRC | 07:37 | |
| *** ig0r__ has joined #openstack-sahara | 07:38 | |
| *** ig0r_ has quit IRC | 07:41 | |
| *** esikachev has joined #openstack-sahara | 07:47 | |
| *** sgotliv has joined #openstack-sahara | 07:50 | |
| *** vgridnev has joined #openstack-sahara | 08:00 | |
| *** ig0r__ has quit IRC | 08:13 | |
| *** ig0r_ has joined #openstack-sahara | 08:17 | |
| *** ig0r_ has quit IRC | 08:19 | |
| degorenko | SergeyLukjanov, looking | 08:36 |
| openstackgerrit | Merged openstack/puppet-sahara: Add an ability to manage use_stderr parameter https://review.openstack.org/210404 | 08:45 |
| *** esikachev has quit IRC | 08:50 | |
| *** esikachev has joined #openstack-sahara | 08:58 | |
| *** tosky has joined #openstack-sahara | 09:12 | |
| openstackgerrit | Denis Egorenko proposed openstack/puppet-sahara: Update parameters for Sahara https://review.openstack.org/205987 | 09:46 |
| *** ig0r_ has joined #openstack-sahara | 10:09 | |
| *** ig0r_ has quit IRC | 10:46 | |
| *** esikachev has quit IRC | 10:49 | |
| *** Poornima has quit IRC | 11:13 | |
| *** ylobankov has quit IRC | 11:17 | |
| *** esikachev has joined #openstack-sahara | 11:17 | |
| *** esikache1 has joined #openstack-sahara | 11:19 | |
| *** esikachev has quit IRC | 11:19 | |
| *** Poornima has joined #openstack-sahara | 11:19 | |
| *** ylobankov has joined #openstack-sahara | 11:22 | |
| openstackgerrit | Andrey Pavlov proposed openstack/sahara: Formatting and mounting methods changed for ironic https://review.openstack.org/200483 | 11:31 |
| *** hdd has joined #openstack-sahara | 12:06 | |
| *** tellesnobrega_ has joined #openstack-sahara | 12:08 | |
| *** tellesnobrega_ has quit IRC | 12:16 | |
| *** tmckay has quit IRC | 12:16 | |
| *** Poornima has quit IRC | 12:24 | |
| *** Nikolay_St has quit IRC | 12:49 | |
| *** _mattf is now known as mattf | 12:51 | |
| *** _crobertsrh is now known as crobertsrh | 12:52 | |
| *** sgotliv has quit IRC | 13:00 | |
| *** sgotliv has joined #openstack-sahara | 13:00 | |
| *** elmiko has joined #openstack-sahara | 13:06 | |
| *** esikache1 has quit IRC | 13:14 | |
| *** tmckay has joined #openstack-sahara | 13:14 | |
| *** sreshetnyak has quit IRC | 13:14 | |
| *** ylobankov has quit IRC | 13:14 | |
| *** vgridnev has quit IRC | 13:15 | |
| *** IBerezovskiy has quit IRC | 13:15 | |
| *** degorenko has quit IRC | 13:15 | |
| *** AndreyPavlov has quit IRC | 13:15 | |
| *** saneax has quit IRC | 13:19 | |
| *** degorenko has joined #openstack-sahara | 13:21 | |
| *** IBerezovskiy has joined #openstack-sahara | 13:23 | |
| *** ylobankov has joined #openstack-sahara | 13:23 | |
| *** AndreyPavlov has joined #openstack-sahara | 13:23 | |
| *** vgridnev has joined #openstack-sahara | 13:23 | |
| *** sreshetnyak has joined #openstack-sahara | 13:26 | |
| *** crobertsrh has left #openstack-sahara | 13:30 | |
| *** esikache1 has joined #openstack-sahara | 13:38 | |
| *** tellesnobrega has quit IRC | 13:41 | |
| *** tellesnobrega has joined #openstack-sahara | 13:42 | |
| *** links has quit IRC | 13:51 | |
| *** hdd has quit IRC | 14:02 | |
| *** hdd has joined #openstack-sahara | 14:02 | |
| *** egafford has joined #openstack-sahara | 14:04 | |
| esikache1 | tellesnobrega: ping | 14:16 |
| tellesnobrega | esikache1: pong | 14:19 |
| esikache1 | tellesnobrega: did you see my comment? #link https://review.openstack.org/#/c/196713/ | 14:19 |
| tellesnobrega | looking | 14:21 |
| openstackgerrit | Nikita Konovalov proposed openstack/sahara-specs: Drop Hadoop v1 support https://review.openstack.org/192563 | 14:23 |
| tellesnobrega | esikache1: can you explain, i didn't fully understand | 14:29 |
| tellesnobrega | SergeyLukjanov: https://review.openstack.org/#/c/203067/ can you take a look? | 14:30 |
| esikache1 | tellesnobrega: in master branch i see last migration id 24 | 14:33 |
| openstackgerrit | Nikita Konovalov proposed openstack/sahara-specs: Drop Hadoop v1 support https://review.openstack.org/192563 | 14:33 |
| esikache1 | tellesnobrega: and all patches must have next version id | 14:33 |
| esikache1 | tellesnobrega: now is 25 | 14:34 |
| *** esikache1 has left #openstack-sahara | 14:37 | |
| *** esikachev has joined #openstack-sahara | 14:38 | |
| tellesnobrega | mine is 25 i think | 14:42 |
| esikachev | yes | 14:43 |
| esikachev | but and my id must be 25 too :) | 14:44 |
| esikachev | tellesnobrega: ^^ | 14:44 |
| tellesnobrega | yes, whatever goes in first we will have a merge conflict, i got a lot of +1 and one +2 | 14:45 |
| tellesnobrega | so its pretty close to getting in | 14:45 |
| vgridnev | tellesnobrega, I thinks that is wrong approach to rebase one patch to another with completely different modifications | 14:46 |
| tellesnobrega | i got it | 14:46 |
| tellesnobrega | removing the -1 | 14:46 |
| esikachev | tellesnobrega: thanks | 14:47 |
| tellesnobrega | no problem | 14:47 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara: Implement ability of creating flavor for scenario tests https://review.openstack.org/203567 | 14:57 |
| *** esikachev has quit IRC | 15:05 | |
| *** DuncanT has quit IRC | 15:05 | |
| *** DuncanT has joined #openstack-sahara | 15:07 | |
| *** esikachev has joined #openstack-sahara | 15:20 | |
| SergeyLukjanov | tellesnobrega approved : | 15:20 |
| vgridnev | tellesnobrega, do you have an implementation of multiple cluster creation in horizon? | 15:21 |
| tellesnobrega | SergeyLukjanov: thanks | 15:23 |
| tellesnobrega | vgridnev: yes | 15:23 |
| vgridnev | tellesnobrega, could you please add link to change request to https://etherpad.openstack.org/p/sahara-reviews-in-horizon ? | 15:24 |
| vgridnev | I don't see that right now | 15:24 |
| tellesnobrega | sure | 15:24 |
| tellesnobrega | done | 15:25 |
| tellesnobrega | not sure about the priority so i put as last place | 15:25 |
| *** esikachev has quit IRC | 15:26 | |
| *** esikachev has joined #openstack-sahara | 15:29 | |
| openstackgerrit | Ken Chen proposed openstack/sahara: Enable HDFS HA in CDH cluster test https://review.openstack.org/212074 | 15:36 |
| openstackgerrit | Ken Chen proposed openstack/sahara: Add CDH test enabling HDFS HA https://review.openstack.org/212074 | 15:40 |
| *** esikachev has quit IRC | 15:51 | |
| *** tosky has quit IRC | 15:57 | |
| *** vgridnev has quit IRC | 16:04 | |
| *** saneax has joined #openstack-sahara | 16:14 | |
| *** saurabhs has joined #openstack-sahara | 17:02 | |
| *** vgridnev has joined #openstack-sahara | 17:05 | |
| *** sgotliv has quit IRC | 17:12 | |
| *** saneax has quit IRC | 17:14 | |
| *** tosky has joined #openstack-sahara | 17:48 | |
| openstackgerrit | Merged openstack/sahara: Increase internal_ip and management_ip column size https://review.openstack.org/203067 | 17:51 |
| *** saurabhs has quit IRC | 18:04 | |
| *** saurabhs has joined #openstack-sahara | 18:05 | |
| *** saurabhs1 has joined #openstack-sahara | 18:09 | |
| *** saurabhs has quit IRC | 18:12 | |
| *** hdd has quit IRC | 18:13 | |
| *** bapalm_ has joined #openstack-sahara | 18:15 | |
| *** hdd has joined #openstack-sahara | 18:18 | |
| *** crobertsrh has joined #openstack-sahara | 18:19 | |
| *** esikachev has joined #openstack-sahara | 18:26 | |
| openstackgerrit | lu huichun proposed openstack/sahara: [EDP] Add scheduling EDP jobs in sahara(oozie engine implementation) https://review.openstack.org/182310 | 18:26 |
| vgridnev | egafford, which space inconsistent there -Puber -P hadoop-2? | 18:30 |
| *** hdd has quit IRC | 18:33 | |
| egafford | vgridnev: I'm assuming that "-Puber -P hadoop-2" is equivalent to "-P uber -P hadoop-2", unless "-Puber" is a standalone key. It's not a big deal (not worth a -1, certainly.) Just wondering if it's significant/intentional. | 18:34 |
| vgridnev | egafford, I think that -Puber -Phadoop-2 is equivalent for the current version, not -P uber .. | 18:37 |
| vgridnev | see for reference: https://oozie.apache.org/docs/4.2.0/ENG_Building.html#Building_an_Oozie_Distribution | 18:37 |
| egafford | vgridnev: I agree; that looks right. | 18:38 |
| openstackgerrit | Michael McCune proposed openstack/sahara: adding sessions module https://review.openstack.org/207208 | 18:39 |
| vgridnev | how to write tests without actual writing tests? | 18:41 |
| *** esikachev has quit IRC | 18:53 | |
| *** hdd has joined #openstack-sahara | 18:54 | |
| *** esikachev has joined #openstack-sahara | 18:58 | |
| openstackgerrit | Trevor McKay proposed openstack/sahara: Add manila nfs data sources https://review.openstack.org/209615 | 19:07 |
| tmckay | egafford, okay, see how that works for you ^^ | 19:08 |
| egafford | tmckay: Will do; thanks. | 19:09 |
| *** hdd has quit IRC | 19:13 | |
| *** hdd has joined #openstack-sahara | 19:16 | |
| *** esikachev has quit IRC | 19:17 | |
| *** Nikolay_St has joined #openstack-sahara | 19:22 | |
| openstackgerrit | lu huichun proposed openstack/sahara-specs: [EDP]enable recurrence edp job in sahara https://review.openstack.org/192097 | 19:23 |
| *** hdd has quit IRC | 19:25 | |
| openstackgerrit | Trevor McKay proposed openstack/sahara: Add manila nfs data sources https://review.openstack.org/209615 | 19:35 |
| egafford | tmckay: (Sorry for bad timing there...) | 19:37 |
| tmckay | egafford, no prob. What was the -1 for? | 19:37 |
| tmckay | typos? | 19:38 |
| egafford | tmckay: No, abstraction and privacy. Better to make default_mount public, if it's accessed outside the class, and to add an @abstractmethod to ShareHandler about it. | 19:38 |
| tmckay | egafford, okay, re-ran on a fresh spark cluster with automount. Worked great | 19:38 |
| egafford | tmckay: \o/ | 19:39 |
| elmiko | nice | 19:39 |
| tmckay | egafford, abstractmethod or just leave it defaulted in Sharehandler? | 19:41 |
| egafford | tmckay: Ah, I see, you just moved it out of the abstraction entirely. | 19:41 |
| egafford | tmckay: That's actually fine until we have more share types and need divergence. | 19:41 |
| egafford | Sorry, I just was stuck in a world in which _get_path was implemented in the concretion. | 19:42 |
| tmckay | okay. It seemed to me this way, we can diverge on _get_path or on the default, plenty of flex tehre | 19:42 |
| tmckay | there | 19:42 |
| egafford | tmckay: Wow, gerritfail. Managed to comment on a different patch set than I reviewed. Wow. | 19:42 |
| tmckay | nice | 19:43 |
| egafford | tmckay: I'm forgetting now: in Python, is privacy always defined at the module level? | 19:44 |
| egafford | Am I allowed to get up in the private grill of my classes if I'm in their module? (Basically, is the convention Java protected or Java private?) | 19:44 |
| tmckay | I would say yes | 19:45 |
| tmckay | I could change the name of _default_mount and clear it up | 19:45 |
| tmckay | just make it public | 19:45 |
| egafford | tmckay: If by convention module members are allowed to access private class members, then all's well. | 19:46 |
| tmckay | It's fine by my convention, not sure about the Python world at large. | 19:47 |
| *** crobertsrh has left #openstack-sahara | 19:47 | |
| *** tosky has quit IRC | 19:47 | |
| egafford | Google is basically just full of people saying "programming languages are tools", which is not helping me. Where are my hobgoblins, Google? | 19:47 |
| egafford | I need my hobgoblins. | 19:48 |
| *** tosky has joined #openstack-sahara | 19:48 | |
| *** sgotliv has joined #openstack-sahara | 19:48 | |
| tmckay | egafford, to me "_blah" means don't call blah unless you know what you're doing. A utility function in a module that calls a _blah in a module class to me believably knows what it is doing | 19:49 |
| egafford | tmckay: Sure, I can see that. I'm just looking to see if there's any actual canonical convention around this. | 19:49 |
| tmckay | if Python really cared, Python would have a formal privacy mechanism :) | 19:50 |
| egafford | Well, that's not true. | 19:50 |
| egafford | I'd ding the heck out of you if you accessed a private var or method out of module, and I'd be right to. | 19:50 |
| egafford | EAFP != YOLO. | 19:50 |
| *** crobertsrh has joined #openstack-sahara | 19:51 | |
| egafford | I do take your point, though, that Python is intentionally less restrictive about this sort of thing, and that's essential to its nature. | 19:52 |
| egafford | tmckay: Yeah, it's impossible to tell whether any convention exists, outside of articles written by obvious Java guys trying to make Python dynamic Java. Therefore, I'm going to assume no prohibitive convention exists. Manila libre! | 19:53 |
| tmckay | ack, works for me | 19:54 |
| tmckay | alright, time to write a spec I think. I'll read through the existing nfs spec first, though | 19:54 |
| egafford | tmckay: I'll wait for Jenkins, but this patch set looks great to me. | 19:58 |
| tmckay | egafford, so, hmm. The https://review.openstack.org/#/c/210839/5 spec seems to be about extending hdfs so that it recognizes nfs:// file schemas | 20:00 |
| egafford | tmckay: True. Do you believe that this can work with your current patch? | 20:01 |
| openstackgerrit | Michael McCune proposed openstack/sahara-specs: adding the API v2 specification to backlog https://review.openstack.org/212172 | 20:01 |
| tmckay | I think so -- I think it's transparent. The question is whether or not the manila hdfs driver actually requires any mount on the node | 20:01 |
| elmiko | \o/ | 20:01 |
| egafford | elmiko: :O | 20:01 |
| elmiko | and my AXE! | 20:01 |
| egafford | elmiko: I actually just LOLed. | 20:02 |
| elmiko | haha | 20:02 |
| egafford | elmiko: That really happened. | 20:02 |
| * egafford can't stop laughing now. It's a problem. | 20:02 | |
| tmckay | if the hdfs driver is just reaching out to the manila server to pull data in, then as long as the hdfs jars can decode the schema and serialize, it wouldn't conflict at all I think | 20:02 |
| elmiko | lol, nice. glad to be of service =) | 20:02 |
| *** sgotliv has quit IRC | 20:02 | |
| tmckay | egafford, in my mind having manila:// for job binaries but not for data soures might be a little strange | 20:03 |
| egafford | tmckay: That was what I thought when I read over it too. Yeah, true. | 20:04 |
| tmckay | it's a natural extension of the binary stuff | 20:04 |
| egafford | tmckay: We could change 'manila://' to 'nfs://', but nfs doesn't imply manila. | 20:04 |
| tmckay | if you have binaries on a share, maybe you want some data there too. Even if it's mostly for dev/test support | 20:04 |
| tmckay | egafford, right. The real question for me is whether or not the nfs:// solution actually does a mount or not | 20:05 |
| tmckay | if it doesn't, then for poking around the manila:// solution for data sources adds a little something extra, because you can muck with the data directly | 20:05 |
| tmckay | egafford, I also think Sahara is more about adding options, more than one way to skin a cat. swift, manila nfs, hdfs, manila-pretending-to-be-nfs, etc | 20:06 |
| tmckay | I can see these options living side by side | 20:06 |
| egafford | tmckay: Right, I don't think the use-NFS-on-local-FS option is bad to have, certainly. | 20:07 |
| tmckay | I should go read about the manila nfs hadoop drive | 20:08 |
| egafford | tmckay: Seems like they might be parallel specs, but I like your data source NFS patch, and I don't see that it conflicts with the NFS Hadoop driver spec, certainly not directly. | 20:08 |
| tmckay | egafford, bonus, the nfs:// might work with Pig ... | 20:09 |
| tmckay | not sure about that | 20:09 |
| tmckay | egafford, another differentiating factor is that the nfs:// solution, working with hadoop, is only applicable to frameworks that use hdfs. So far that includes both spark and hadoop, but ... theoretically .... | 20:10 |
| egafford | tmckay: Valid. | 20:11 |
| egafford | tmckay: Theoretically valid, at any rate. | 20:11 |
| openstackgerrit | Michael McCune proposed openstack/sahara-specs: adding the API v2 specification to backlog https://review.openstack.org/212172 | 20:11 |
| *** tellesnobrega is now known as tellesnobrega_af | 20:13 | |
| tmckay | egafford, another difference. This sets up hadoop to use a particular nfs server (multiple in the future) at cluster launch, and that's what you've got. With the manila:/ solution, you can whip up any old share any time you like and access it | 20:16 |
| tmckay | egafford, btw, do we have a way to do an unmount from Sahara? | 20:17 |
| egafford | tmckay: ...We do not at present. | 20:17 |
| *** tosky has quit IRC | 20:17 | |
| egafford | I think I handwaved toward that as a clear next step, but chose not to tackle it in spec rev 1. | 20:17 |
| tmckay | I think these are complementary | 20:18 |
| *** tosky has joined #openstack-sahara | 20:18 | |
| egafford | tmckay: They're certainly not contradictory. | 20:19 |
| egafford | And I can see clear use cases for both. | 20:19 |
| *** saurabhs has joined #openstack-sahara | 20:21 | |
| *** saurabhs1 has quit IRC | 20:24 | |
| *** Nikolay_St has quit IRC | 20:33 | |
| *** hdd has joined #openstack-sahara | 20:34 | |
| vgridnev | elmiko, I've got nice axe, I believe | 20:39 |
| elmiko | vgridnev, hehe yay \o/ | 20:40 |
| vgridnev | oops, you* | 20:40 |
| *** tmckay has quit IRC | 20:40 | |
| elmiko | works either way ;) | 20:40 |
| *** crobertsrh has left #openstack-sahara | 20:41 | |
| openstackgerrit | Marianne Linhares Monteiro proposed openstack/sahara: Rename get_job_status to get_job_info https://review.openstack.org/212184 | 20:44 |
| *** crobertsrh has joined #openstack-sahara | 20:45 | |
| *** vgridnev has quit IRC | 20:50 | |
| *** hdd has quit IRC | 20:52 | |
| *** hdd has joined #openstack-sahara | 20:52 | |
| *** vgridnev has joined #openstack-sahara | 21:07 | |
| *** crobertsrh is now known as _crobertsrh | 21:10 | |
| *** tosky has quit IRC | 21:47 | |
| *** tosky has joined #openstack-sahara | 21:48 | |
| *** vgridnev has quit IRC | 22:21 | |
| *** mattf is now known as _mattf | 22:26 | |
| *** egafford has quit IRC | 22:40 | |
| *** devlaps has joined #openstack-sahara | 22:44 | |
| *** hdd has quit IRC | 22:47 | |
| *** saneax has joined #openstack-sahara | 22:58 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/sahara: Updated from global requirements https://review.openstack.org/212219 | 23:14 |
| *** tosky has quit IRC | 23:17 | |
| *** tosky has joined #openstack-sahara | 23:18 | |
| *** tosky has quit IRC | 23:19 | |
| *** hdd has joined #openstack-sahara | 23:22 | |
| *** chlong has quit IRC | 23:25 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!