| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara master: Adding Ambari 2.6 to image pack https://review.openstack.org/579024 | 00:01 |
|---|---|---|
| *** masuberu has quit IRC | 01:09 | |
| *** masber has joined #openstack-sahara | 01:29 | |
| *** masuberu has joined #openstack-sahara | 02:12 | |
| *** rodrigod` has joined #openstack-sahara | 02:18 | |
| *** masber has quit IRC | 02:19 | |
| *** rodrigods has quit IRC | 02:19 | |
| *** hoonetorg has quit IRC | 02:31 | |
| *** hoonetorg has joined #openstack-sahara | 02:47 | |
| openstackgerrit | Nguyen Hung Phuong proposed openstack/sahara-image-elements master: fix tox python3 overrides https://review.openstack.org/581176 | 03:33 |
| *** masuberu has quit IRC | 03:43 | |
| *** Bhujay has joined #openstack-sahara | 03:49 | |
| openstackgerrit | Tuan Do Anh proposed openstack/sahara master: Trivial fix typo of description https://review.openstack.org/581181 | 04:01 |
| openstackgerrit | Nguyen Hung Phuong proposed openstack/sahara-specs master: fix tox python3 overrides https://review.openstack.org/581193 | 04:47 |
| *** links has joined #openstack-sahara | 05:22 | |
| *** pgadiya has joined #openstack-sahara | 05:22 | |
| *** pgadiya has quit IRC | 05:22 | |
| *** links has quit IRC | 06:38 | |
| *** links has joined #openstack-sahara | 06:53 | |
| *** tesseract has joined #openstack-sahara | 07:13 | |
| *** tosky has joined #openstack-sahara | 07:39 | |
| *** links has quit IRC | 07:44 | |
| *** rcernin has quit IRC | 07:47 | |
| *** links has joined #openstack-sahara | 07:57 | |
| *** rcernin has joined #openstack-sahara | 09:44 | |
| *** rcernin has quit IRC | 09:47 | |
| *** Bhujay has quit IRC | 10:15 | |
| *** Bhujay has joined #openstack-sahara | 10:15 | |
| *** Bhujay has quit IRC | 10:17 | |
| *** Bhujay has joined #openstack-sahara | 10:18 | |
| *** Bhujay has quit IRC | 10:19 | |
| *** Bhujay has joined #openstack-sahara | 10:19 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Rework saharaclient authentication https://review.openstack.org/580693 | 13:49 |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara-tests master: Allow for the future python-saharaclient https://review.openstack.org/581372 | 14:03 |
| *** jeremyfreudberg has joined #openstack-sahara | 14:04 | |
| jeremyfreudberg | tosky: eyes on https://review.openstack.org/#/c/581372/ ? | 14:04 |
| tosky | jeremyfreudberg: I was checking it right now | 14:10 |
| tosky | jeremyfreudberg: more then "remove me after requrement bump" is "leave only the code after the except after the code bump", but yeah | 14:13 |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Fixup args in data source create https://review.openstack.org/579680 | 14:16 |
| tosky | so, is this ^^ major bump required to support S3 data sources? | 14:17 |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Support of S3 data sources in OSC https://review.openstack.org/580727 | 14:18 |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Remove ResourceManager aliases https://review.openstack.org/581095 | 14:19 |
| jeremyfreudberg | okay, i think i got the patch chain correct there | 14:19 |
| jeremyfreudberg | tosky: yes, "Fixup args in data source create" is necessary for S3 data sources | 14:20 |
| jeremyfreudberg | some of the other patches are just other things which I felt were good to cram into the new saharaclient release | 14:20 |
| tosky | so, if I get it correctly, this new release is going to be backward incompatible with older code which uses saharaclient | 14:21 |
| jeremyfreudberg | tosky: yes, it will backwards incompatible, but also signified as so by semver | 14:22 |
| tosky | I guess I will ask the TC (or the appropriate body) if the the deprecation and compatibility rules apply also to the Python clients | 14:23 |
| jeremyfreudberg | tosky: "Note: this tag can currently only be applied to services (type:service deliverables). The tag definition may evolve in the future to include library feature deprecation policy and be applicable to libraries as a result" | 14:25 |
| tellesnobrega | jeremyfreudberg, we need to rush with apiv2 on saharaclient right | 14:25 |
| tellesnobrega | ? | 14:25 |
| tosky | sure, but better ask now | 14:25 |
| tosky | maybe we are setting a precedent | 14:25 |
| *** Bhujay has quit IRC | 14:26 | |
| jeremyfreudberg | tellesnobrega: yes, we need to rush. library freeze is very soon (coinciding with rocky-3) | 14:26 |
| tosky | I did not check the old review, but the changes for S3 for job binaries did not require a bump - was it much different? | 14:28 |
| jeremyfreudberg | tosky: in the case of job binaries the arguments were already generalized. in the case of data sources they are annoyingly swift specific | 14:29 |
| tosky | oh | 14:30 |
| jeremyfreudberg | it is possible to write more code with lots of optional arguments, but that means more bad UX and bad code that we will have to carry around | 14:30 |
| tellesnobrega | jeremyfreudberg, I will do my best to get a jump on it | 14:30 |
| tellesnobrega | -1 to carry bad code around | 14:31 |
| tosky | I disagree | 14:35 |
| tellesnobrega | jeremyfreudberg, to be sure, we are done with the api side, we need to work on the osc part | 14:36 |
| tosky | asking on -tc, it looks like we should try to keep the compatibility with the old interface | 14:37 |
| tosky | if you want to join #openstack-tc, and discuss about it, it would be better | 14:37 |
| tosky | as it is right now I don't think I can accept a breaking change | 14:37 |
| jeremyfreudberg | tellesnobrega: https://review.openstack.org/#/c/581072/ is one more server-side patch. and i want to do one more patch which will lay the groundwork for microversion, if i have time | 14:38 |
| jeremyfreudberg | tosky: ok, i'll jump over | 14:39 |
| tellesnobrega | jeremyfreudberg, quick question, why don't we have s3 on apiv2 only? | 14:52 |
| jeremyfreudberg | tellesnobrega: good question | 14:53 |
| tellesnobrega | that solves that breakage issue, and "forces" us to work harder on v2 | 14:53 |
| tellesnobrega | but I'm not sure that is what I want | 14:55 |
| tellesnobrega | tosky ^ | 14:55 |
| tosky | you can't ask me what you want O.o | 14:55 |
| jeremyfreudberg | tellesnobrega: that would expose strange bugs with submitting a job through apiv1 and referening an s3 datasource by uuid | 14:55 |
| jeremyfreudberg | i included s3 in apiv1 because the plugins, job binaries, and data sources APIs are sort-of understood to be dynamic | 14:56 |
| tosky | my quick estimate says that changing the client to keep the compatibility is closer than closing the v2 work | 14:56 |
| tellesnobrega | tosky, sorry, it was to the previous info | 14:56 |
| tellesnobrega | if we can work the compatibility issue I'm all for s3 in apiv1 | 14:57 |
| tellesnobrega | and I agree with tosky that fixing it is easier that working apiv2 completely although we really need to do the work | 15:00 |
| jeremyfreudberg | tosky, so, the new arguments would have to be optional, and come *after* is_protected? https://github.com/openstack/python-saharaclient/blob/master/saharaclient/api/data_sources.py#L29 | 15:05 |
| tellesnobrega | jeremyfreudberg, that sounds correct | 15:06 |
| tellesnobrega | following doug's suggestion | 15:06 |
| tosky | yes, that's the way to have it working in a compatible way (unless the last arguments are changed to **kwargs) | 15:07 |
| tosky | and luckily it's python and we don't have ABI | 15:07 |
| jeremyfreudberg | tosky: we can't do kwargs, because there could be someone out there relying on argument order | 15:08 |
| tellesnobrega | you could kwargs for the new stuff | 15:09 |
| tellesnobrega | but I would go with named arguments | 15:10 |
| jeremyfreudberg | yeah, it should be named arguments | 15:11 |
| tellesnobrega | cool | 15:15 |
| tellesnobrega | tosky, I will spawn ambari images today | 15:15 |
| tellesnobrega | can you remind me again which versions I need to test? | 15:15 |
| tosky | Ambari 2.6 images should be able to spawn HDP 2.6, HDP 2.5 and HDP 2.4 | 15:17 |
| tosky | Ambari 2.4 can create HDP 2.4 and HDP 2.3 clusters | 15:17 |
| tellesnobrega | ok | 15:17 |
| tellesnobrega | so ambari 2.4 and try to spawn 2.4 and 2.3 to see if the error comes up? | 15:17 |
| tosky | oh, but that's with sahara-image-elements? | 15:18 |
| tosky | right, that's different | 15:18 |
| tellesnobrega | yes | 15:18 |
| tellesnobrega | I'm going to test the ssl error iirc | 15:18 |
| tosky | ok, sorry, then: Ambari 2.2.1.0 is the one used for HDP 2.4, while Ambari 2.2.0.0 is the one used for HDP 2.3 | 15:19 |
| tellesnobrega | ok | 15:19 |
| tosky | sie can also build Ambari 2.4.2.0 images, but those have been reported to have other issues; we may want to try to fix them (maybe) | 15:19 |
| tosky | but we have sahara-image-pack for them too (apart from pike) | 15:19 |
| tellesnobrega | ok | 15:20 |
| tellesnobrega | I will generate all sorts of ambari images and start testing | 15:20 |
| tellesnobrega | while working on apiv2 on client osc | 15:20 |
| tellesnobrega | and test new spark image | 15:20 |
| tellesnobrega | I might go crazy at some point | 15:20 |
| tellesnobrega | :) | 15:21 |
| tellesnobrega | heading out for lunch | 15:21 |
| tellesnobrega | bbl | 15:21 |
| jeremyfreudberg | tosky: feel free to use your review powers on the "good to merge" section here: https://etherpad.openstack.org/p/jeremy-20180710-patches | 15:25 |
| jeremyfreudberg | and also free to poke into the "to discuss" patches | 15:26 |
| *** links has quit IRC | 15:26 | |
| tosky | oh, oki | 15:26 |
| *** Bhujay has joined #openstack-sahara | 15:30 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Allow S3 credentials in data source create https://review.openstack.org/579680 | 15:51 |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Support of S3 data sources in OSC https://review.openstack.org/580727 | 15:51 |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Support of S3 data sources in OSC https://review.openstack.org/580727 | 15:58 |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara-dashboard master: Support of S3 data sources in dashboard https://review.openstack.org/580748 | 16:25 |
| *** jeremyfreudberg has left #openstack-sahara | 16:35 | |
| *** Bhujay has quit IRC | 16:38 | |
| *** tesseract has quit IRC | 17:07 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: Support of S3 data sources in OSC https://review.openstack.org/580727 | 17:48 |
| tellesnobrega | tosky, have you generated ubuntu images recently? | 20:03 |
| tosky | tellesnobrega: uhm, no | 20:06 |
| tellesnobrega | the image is just huge | 20:06 |
| tellesnobrega | 9GB | 20:06 |
| tosky | more than centos? | 20:07 |
| tellesnobrega | no reference | 20:12 |
| tellesnobrega | testing spark now | 20:12 |
| tellesnobrega | we never did spark on centos | 20:12 |
| *** jeremyfreudberg has joined #openstack-sahara | 20:29 | |
| jeremyfreudberg | tellesnobrega: your spark image was 9gb? | 20:29 |
| tellesnobrega | yes | 20:29 |
| jeremyfreudberg | i've never seen that happen | 20:30 |
| jeremyfreudberg | for s3 work all the spark images came out normal sized | 20:30 |
| tellesnobrega | I tried creating it with normal xenial file size it fails with no disk left | 20:30 |
| tellesnobrega | so I increased | 20:30 |
| jeremyfreudberg | when you get a chance (could be tomorrow), can you paste the entire output from a failed run? | 20:31 |
| jeremyfreudberg | a failed run, meaning, one which failed with no disk left | 20:32 |
| tellesnobrega | jeremyfreudberg, sure | 20:32 |
| jeremyfreudberg | just whenever you get a chance, any time this week | 20:33 |
| jeremyfreudberg | no huge rush | 20:33 |
| *** jeremyfreudberg has quit IRC | 20:33 | |
| tellesnobrega | tosky, on an unrelated matter, the i18n workaround worked | 20:37 |
| tosky | tellesnobrega: so full deployment with python3? | 20:39 |
| tellesnobrega | venv with python3 | 20:40 |
| tosky | nice | 20:42 |
| tellesnobrega | jeremyfreudberg, http://paste.openstack.org/show/725467/ | 20:45 |
| tellesnobrega | tosky, not a good news, cluster fails to access the instances on python3 | 21:16 |
| tellesnobrega | instance: none, cluster: 0ed8da4e-7e33-42ca-9e15-af43b8b0ebce] Can't login to node, IP: 10.0.0.14, reason Ran out of input | 21:17 |
| tosky | that's probably more complicated than the i18n workaround | 21:18 |
| tosky | and who knows which component is failing | 21:18 |
| tellesnobrega | I'm not sure the whole devstack is under python3 | 21:22 |
| tellesnobrega | could also be conflict between py2 and py3 | 21:23 |
| *** rcernin has joined #openstack-sahara | 22:22 | |
| *** tosky has quit IRC | 23:08 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!