Tuesday, 2014-11-18

*** tellesnobrega has joined #openstack-sahara00:07
openstackgerritAndrew Lazarev proposed openstack/sahara: Add indirect VMs access implementation  https://review.openstack.org/13359000:15
*** tellesnobrega has quit IRC00:17
*** tellesnobrega has joined #openstack-sahara00:30
openstackgerritAndrew Lazarev proposed openstack/sahara: Fixed pep8 after oslo.db config update  https://review.openstack.org/13513600:37
*** hogepodge has quit IRC01:21
*** hogepodge has joined #openstack-sahara01:39
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/python-saharaclient: Replacing data_processing with data-processing  https://review.openstack.org/13514801:54
*** tellesnobrega has quit IRC02:21
*** Viswanath has joined #openstack-sahara02:38
*** Viswanath has quit IRC02:41
*** tellesnobrega has joined #openstack-sahara03:17
*** tellesnobrega has quit IRC03:52
*** macjack has joined #openstack-sahara03:52
*** tellesnobrega has joined #openstack-sahara03:53
*** chandankumar has joined #openstack-sahara03:57
*** chandankumar has quit IRC04:02
*** Networkn3rd has quit IRC04:07
*** tellesnobrega has quit IRC04:34
openstackgerritMerged openstack/sahara: Fixed pep8 after oslo.db config update  https://review.openstack.org/13513605:05
*** chandankumar has joined #openstack-sahara05:36
*** miqui has joined #openstack-sahara05:38
*** _crobertsrh has quit IRC05:55
*** crobertsrh has joined #openstack-sahara06:05
openstackgerritThomas Bechtold proposed openstack/sahara-image-elements: Add openSUSE support for diskimage-create.sh  https://review.openstack.org/13517606:14
openstackgerritThomas Bechtold proposed openstack/sahara-image-elements: Add openSUSE support for diskimage-create.sh  https://review.openstack.org/13517606:29
*** Longgeek has joined #openstack-sahara06:34
*** k4n0 has joined #openstack-sahara06:57
*** macjack has quit IRC07:47
*** Longgeek has quit IRC07:53
*** Longgeek has joined #openstack-sahara07:53
*** Longgeek_ has joined #openstack-sahara07:57
*** Longgeek has quit IRC08:00
*** Longgeek_ has quit IRC08:09
*** Longgeek has joined #openstack-sahara08:10
*** miqui has quit IRC08:11
*** Longgeek_ has joined #openstack-sahara08:14
*** Longgeek has quit IRC08:18
*** stannie has joined #openstack-sahara08:42
openstackgerritMerged stackforge/sahara-ci-config: Fix node label for ui tests  https://review.openstack.org/13503308:55
*** venza has quit IRC09:03
*** venza has joined #openstack-sahara09:04
openstackgerritDenis Egorenko proposed openstack/sahara-image-elements: Add CentOS-6.6 cloud image  https://review.openstack.org/13519809:08
*** witlessb has joined #openstack-sahara09:24
*** Poornima has joined #openstack-sahara09:46
*** tellesnobrega has joined #openstack-sahara10:05
*** nikunj2512 has joined #openstack-sahara10:08
nikunj2512in launch cluster, what does base image means???10:08
*** tellesnobrega has quit IRC10:16
nikunj2512jpich: Will having a "upload keypair" button will be benifical??10:24
*** tosky has joined #openstack-sahara10:34
*** tellesnobrega has joined #openstack-sahara10:47
*** Longgeek_ has quit IRC11:04
*** Longgeek has joined #openstack-sahara11:05
*** tellesnobrega has quit IRC11:10
*** chandan_kumar has joined #openstack-sahara11:37
*** chandankumar has quit IRC11:40
*** Poornima has quit IRC11:43
openstackgerritLuigi Toscano proposed stackforge/sahara-ci-config: Explicitly enable/disable all set of tests (every plugin)  https://review.openstack.org/13450912:02
tellesnobrega_alazarev, ping12:05
openstackgerritVitaly Gridnev proposed openstack/sahara: And new endpoint and periodic task with events ops  https://review.openstack.org/13450412:08
openstackgerritAbbass Marouni proposed openstack/sahara: Enable HDFS NameNode High Availability with HDP 2.0.6 plugin  https://review.openstack.org/13205112:11
*** nikunj2512 has quit IRC12:19
openstackgerritMerged openstack/sahara-dashboard: Refactoring UI tests  https://review.openstack.org/10962212:26
openstackgerritLuigi Toscano proposed stackforge/sahara-ci-config: Explicitly enable/disable all set of tests (every plugin)  https://review.openstack.org/13450912:29
*** chandan_kumar has quit IRC12:32
openstackgerritAbbass Marouni proposed openstack/sahara: Enable HDFS NameNode High Availability with HDP 2.0.6 plugin  https://review.openstack.org/13205112:36
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add tempest tests for python-saharaclient  https://review.openstack.org/13527212:43
*** chandan_kumar has joined #openstack-sahara12:50
*** chandan_kumar has quit IRC12:50
*** chandankumar has joined #openstack-sahara12:50
*** nikunj2512 has joined #openstack-sahara12:54
*** nikunj2512 has left #openstack-sahara12:54
*** DenisEgorenko has joined #openstack-sahara12:58
*** k4n0 has quit IRC13:01
tellesnobrega_have you guys set up a multinode devstack with sahara?13:06
openstackgerritAndrey Pavlov proposed openstack/sahara: Fixed bug with Hive jobs fail  https://review.openstack.org/13395213:07
*** stannie1 has joined #openstack-sahara13:08
elmikotellesnobrega_: i have not, but i think crobertsrh has13:13
crobertsrhI have indeed13:13
openstackgerritKen Chen proposed openstack/sahara: Use first_run to Start Services  https://review.openstack.org/13447113:13
openstackgerritSergey Reshetnyak proposed openstack/sahara: Fix for getting auth url for hadoop-swift  https://review.openstack.org/13527813:13
crobertsrhEssentially, I run one machine with "compute only" and the other with "everything"13:13
tellesnobrega_do i need to enable sahara only at the controller node or on all of them?13:14
crobertsrhI believe just the controller node.13:14
tellesnobrega_i see13:14
tellesnobrega_how about swift?13:14
crobertsrhcontroller node only, I think.13:14
crobertsrhI wish I had my old multinode local.conf handy, but I'm pretty sure I only had the compute services running on my non-controller host.13:15
crobertsrhIt was probably 6 months ago though.13:15
crobertsrhI will brb13:15
tellesnobrega_hum, i'm following the configs from the devstack page, im getting a cinder error13:15
tellesnobrega_ok13:15
crobertsrhI don't recall cinder problems, but I suspect a lot might have changed since I tried it.13:17
tellesnobrega_i just found a bug report related to that13:18
tellesnobrega_https://bugs.launchpad.net/devstack/+bug/139372113:18
uvirtbotLaunchpad bug 1393721 in devstack "devstack multinode fails during cinder volume type creation" [Undecided,Confirmed]13:18
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Saharaclient tests for tempest  https://review.openstack.org/13076713:22
tellesnobrega_just removing cinder from compute node13:24
elmikois it just a problem with cinder on compute?13:25
tellesnobrega_apparently yes13:27
tellesnobrega_removed and it seems to be working13:28
elmikocool!13:34
openstackgerritSergey Reshetnyak proposed openstack/sahara: Refactoring integration tests for Vanilla 1 plugin  https://review.openstack.org/13115513:38
openstackgerritSergey Reshetnyak proposed openstack/sahara: Enable auto security group for vanilla integration tests  https://review.openstack.org/13528213:38
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Saharaclient tests for tempest  https://review.openstack.org/13076713:41
*** egafford has joined #openstack-sahara13:43
*** miqui has joined #openstack-sahara14:01
openstackgerritVitaly Gridnev proposed openstack/sahara: And new endpoint and periodic task with events ops  https://review.openstack.org/13450414:10
*** tmckay has joined #openstack-sahara14:19
tellesnobrega_elmiko, crobertsrh the multinode worked14:29
crobertsrhgreat!14:30
elmikonice14:30
*** macjack has joined #openstack-sahara14:37
openstackgerritVitaly Gridnev proposed openstack/sahara: Add provision step in creating cluster with direct engine  https://review.openstack.org/13449014:38
openstackgerritChad Roberts proposed openstack/sahara: Adding CDH to the list of default plugins  https://review.openstack.org/12483514:42
*** tosky has quit IRC14:45
*** chandankumar has quit IRC14:46
*** tosky has joined #openstack-sahara14:47
*** Poornima has joined #openstack-sahara14:50
*** tmckay1 has joined #openstack-sahara14:50
*** tmckay has quit IRC14:50
*** Poornima has quit IRC14:51
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add tempest tests for python-saharaclient  https://review.openstack.org/13527214:52
*** Poornima has joined #openstack-sahara14:52
openstackgerritMichael McCune proposed openstack/sahara-specs: Specification Repository Backlog Refactor  https://review.openstack.org/13531314:56
openstackgerritDenis Egorenko proposed stackforge/sahara-ci-config: Add tempest tests for python-saharaclient  https://review.openstack.org/13527215:03
*** openstackgerrit has quit IRC15:04
*** openstackgerrit has joined #openstack-sahara15:04
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Saharaclient tests for tempest  https://review.openstack.org/13076715:14
openstackgerritChad Roberts proposed openstack/sahara: Adding CDH to the list of default plugins  https://review.openstack.org/12483515:24
openstackgerritAndrey Pavlov proposed openstack/python-saharaclient: Saharaclient tests for tempest  https://review.openstack.org/13076715:33
*** chandankumar has joined #openstack-sahara15:54
*** mattf is now known as _mattf16:03
*** tmckay1 has quit IRC16:21
*** Poornima has quit IRC16:29
*** Poornima has joined #openstack-sahara16:30
*** tmckay has joined #openstack-sahara16:36
*** Longgeek has quit IRC16:55
tellesnobrega_i'm playing here with cluster creation, and i noticed that I cannot edit a node group or cluster template16:58
tellesnobrega_is there a reason for that?16:58
tellesnobrega_i can copy it and create a new one based on the other, but i cant edit it16:58
elmikotellesnobrega_: yes, there is no update for those currently.17:00
elmikowe talked about it at summit and i think we are going to fix it this cycle17:01
tellesnobrega_awesome, let me know when that is happening17:01
tellesnobrega_i want to help out17:01
elmikocool17:01
elmikoi think there are some problems to work out, like dependent objects referencing updated templates and the like17:02
tellesnobrega_that happens as well when trying to delete17:02
tellesnobrega_i'm not sure, but maybe we could allow recursive delete17:02
crobertsrhYeah.  At one point, someone took a crack at a patch set to allow editing (at least in the UI), but I think it eventually got abandoned.17:02
elmikomaybe, i think it's something we should bring up at the meeting17:03
tellesnobrega_sure17:03
tellesnobrega_what you think about recursive delete? for example, i want to delete a node group and it deletes all templates and cluster that are using this ng17:04
tellesnobrega_of course that would have to require a confirmation from the user17:04
elmikoyea, i dunno, that could get really weird if there are multiple clusters using a node group or something17:04
tellesnobrega_but it may be a good thing, i find kinda "hard" having to remove everything by hand17:04
*** IvanBerezovskiy has left #openstack-sahara17:04
elmikoagreed, the current workflow needs improvement17:05
*** egafford has quit IRC17:06
*** crobertsrh has quit IRC17:14
*** elmiko is now known as elmiko_lunch17:17
*** elmiko_lunch is now known as elmiko17:17
*** Poornima has quit IRC17:37
tellesnobrega_hi, i'm getting a weird error when launching a cluster. It fails saying that the publicURL for the network can't be found. I can create instances by hand and access them, but the heat stack fails17:39
tellesnobrega_i listed the services now, and there is no network endpoint17:39
tellesnobrega_elmiko, do you use sahara with nova-network or neutron?18:00
elmikotellesnobrega_: neutron18:03
*** jodah has quit IRC18:03
tellesnobrega_i think its better that i start using it as well18:03
tellesnobrega_facing problems with n-net18:03
*** jodah has joined #openstack-sahara18:05
*** crobertsrh has joined #openstack-sahara19:12
tmckayHi folks, preview item for Thursday meeting.  https://review.openstack.org/#/c/130153/19:12
tmckayalazarev, aignatov, SergeyLukjanov, elmiko, _mattf, ^^19:13
tmckayinteresting question19:13
tmckayI wonder if there is an option to carry a library under the CDH plugin somehow.  Rather than make it an openstack requirement19:14
elmikoyea, good spot19:14
tmckayimho it's up to vendors in general to push libs into Fedora/Ubuntu/Centos if they are going to be listed as requirements19:15
tmckaybut that's just my opinion19:15
elmikoit's a tough question19:16
tmckaycarrying under the plugin, though, is different.  Although I'm not sure how that would work logistically.19:16
tmckayIt would have to be source in that case, not an rpm19:16
tmckaywhich would be tough to maintain19:16
elmikoplus, in this case, the package is in pypi19:16
tmckaybut, it might bridge the gap until packaging could happen19:16
tmckayelmiko, hmm, are global requirements pulled from rpms or pypi?  Quote from 3rd comment:19:18
tmckay"By adding something to OpenStack global-requirements.txt we are  basically demanding that Linux Distros package this for the next release  of OpenStack. If they already have, great. If not, we should be  cautious of adding it."19:19
elmikotmckay: good question, i imagine it depends on how the requirements are consumed. e.g. how are you installing locally, i think pip is probably the default19:19
*** egafford has joined #openstack-sahara19:23
tmckayelmiko, that quote is from https://github.com/openstack/requirements/blob/master/README.rst#id119:28
tmckayI'm unclear on the relationship of pip, rpms, and global requirements19:30
* elmiko looks19:30
elmikotmckay: here's my take on the requirements.txt and test-requirements.txt19:31
elmikoif you install with `python setup.py install` it will try to install those requirements, likewise you can use pip like `pip install -r requirements.txt`19:32
elmikothose are using pip to do the installs, nothing to do with rpm, or deb if using debian/ubuntu19:33
elmikoin most cases that i have seen, the version numbers coming out of the requirements file don't always sync up with what the distro provides19:33
tmckayright, but this statement that adding a package to global requirements essentially requires the distros to package that requirement for the next OS release suggests non-pip19:34
tmckayso, maybe RDO is yum based19:35
tmckayfor instance19:35
tmckaynot sure what stack.sh is doing19:35
elmikowell yea, i think that's more what Sean is talking about. supporting distros that release full versions of openstack19:35
elmikostack.sh is using pip19:35
tmckaythat would make sense19:36
elmikoif you just install these source repos(Nova, Keystone, Sahara, etc) then you would be doing so outside the normal packagers anyway19:36
tmckayhaven't dabbled in this stuff before19:36
elmikobut if i'm packaging Sahara for Fedora or RDO, then i need to be able to use the packaging system to install because no one wants stuff installed by pip without their knowledge19:37
tmckayright19:37
elmikoi think that's his main point, with regards to packaging19:37
tmckayyeah, that makes perfect sense now that we've chatted about it19:40
tmckaytaking a look to see how cdh uses cm-api19:40
tmckaywe could just leave it up to a documentation issue.  If you want to run the cdh plugin, go get cm-api on your system.19:41
*** tosky has quit IRC19:41
tmckayif you're a cloudera fan, you've probably got it already19:41
elmikothat sounds like a lot of work for our users. not saying i have a better idea, just opining19:42
tmckayagreed.  But I'm sensitive to the issue of adding a requirement that is not packaged yet.  In that case, someone from the Sahara/CDH community needs to take on the work of packaging for supported OSs19:44
tmckayotherwise it breaks anyway19:45
tmckayand, *all* of Sahara breaks19:45
openstackgerritOpenStack Proposal Bot proposed openstack/sahara-dashboard: Updated from global requirements  https://review.openstack.org/13279919:45
tmckayunless for instance RDO specifically doesn't include the requirement.  That would be the only choice19:45
elmikoagreed, we most certainly can't include a requirement that is not packaged19:45
tmckay# cm_api client is not present in OS requirements19:46
tmckaytry:19:46
tmckay    from cm_api import api_client19:46
tmckay    from cm_api.endpoints import services19:46
tmckayexcept ImportError:19:46
tmckay    api_client = None19:46
tmckay    services = None19:46
tmckaythat's the current solution in Sahara.19:46
elmikounfortunately, that's probably the best we can do for now19:46
tellesnobrega_elmiko, i started devstack with neutron, set not to use floating ips (i need this) and started a cluster. its getting an ip address, but it is never reachable19:54
tellesnobrega_anyone had this problem?19:54
elmikotellesnobrega_: i think without floating ips you might need to enable namespace access. i'm not 100% sure about that though.19:55
tellesnobrega_hum20:00
openstackgerritChad Roberts proposed openstack/sahara-specs: Adding query filtering support to Sahara  https://review.openstack.org/13431920:05
crobertsrhcan you ping the server manually?20:06
crobertsrhor ssh to it?20:06
tellesnobrega_no20:10
tellesnobrega_trying again after setting use_neutron=True and namespace=True20:11
crobertsrhAlso, might want to try running a "regular" non-sahara instance to see if it's a neutron problem in general or if it's somehow related to Sahara.20:12
tellesnobrega_will do20:13
tellesnobrega_cant ping to any instance :(20:17
tellesnobrega_dont know what i did wrong20:18
elmikotellesnobrega_: if they don't have floating ips, you would probably need to use the namespace in your ping to hit them20:18
tellesnobrega_so i ping with the instance name?20:19
elmikonot quite, you would need to use the namespace created for the instances to route your calls. let me see if i can dig something up20:19
tellesnobrega_thanks20:21
elmikotellesnobrega_: here's a little something, http://www.opencloudblog.com/?p=42 , look at the section "Executing a command in a network namespace"20:21
elmikoyou will need to determine the namespace created, i think the last time i did this i used the openvswitch commands to dump the namespaces20:22
*** chandankumar has quit IRC20:27
openstackgerritMerged openstack/sahara: Fixed auto security group cleanup in case of creation error  https://review.openstack.org/13434520:29
tellesnobrega_elmiko, i have zero experience with this kind of stuff, but i will find my way around20:30
elmikotellesnobrega_: ok, sorry for the brief explanation. it gets pretty deep.20:30
tellesnobrega_i'm reading a presentation from mirantis, explaining how namespaces work20:31
elmikoin a nutshell, when using namespaces, the kernel creates a named grouping for the virtual network. you can use this name to access the virtual group.20:31
elmikocool, that's a good start20:31
*** witlessb has quit IRC20:40
*** witlessb has joined #openstack-sahara20:40
tmckaycrobertsrh, you know your comment yesterday about 30 lines of code for filtering?20:44
tmckayI think you're right20:44
crobertsrhheh :)  I had faith in you :)20:44
tmckayquery.filter_by(**search_opts) looks like it will do it20:44
crobertsrhawesome.  That's pretty close to the db.magicstuff() call that I was thinking of.20:45
tmckaywe just have to pass the dict down, maybe handle exceptions on crappy search args, and I think that's it20:45
tmckaygonna hack a POC for templates or something20:46
crobertsrhWay more lines in the spec than code to do the whole thing!20:46
tmckaylol, yes20:46
tmckayI feel guilty about this20:46
crobertsrhWe can estimate it as a "medium" and then take a few days off to go and get donuts!20:46
tmckaysssh20:47
crobertsrhI guess I should get going on the UI side so I don't finish last!20:47
crobertsrhNot sure I can beat 30 lines though.  Probably about 10 lines per table I want to filter.20:47
tmckaymy kids race to see who gets their teeth brushed first.  I'm over it :)20:47
crobertsrhI'll think overtime though to bring that down20:47
crobertsrhwow...good problem to have!20:48
tmckaythey'll compete on anything20:48
crobertsrhfastest room cleanup...go!20:48
tellesnobrega_elmiko, i was able to ssh20:59
elmikoyay!20:59
tellesnobrega_it was a security groups problem20:59
elmikoahh, well that was easy then =)20:59
tellesnobrega_missing the rule that allowed me ssh20:59
tellesnobrega_or ping20:59
elmikocool, glad you got it worked out21:00
tellesnobrega_better than find out who namespace works from scratch21:00
tellesnobrega_thanks21:00
tellesnobrega_*how21:00
elmikolol, yes. much better21:00
tellesnobrega_i gotta go21:02
tellesnobrega_i will be back later21:02
crobertsrhtellesnobrega...if you're using devstack, try putting those rules in your local.sh script.  That way you'll never forget them :)21:04
tellesnobrega_will do21:05
tellesnobrega_thanks21:05
*** stannie has quit IRC21:11
*** miqui has quit IRC21:44
*** _mattf is now known as mattf21:52
openstackgerritAndrew Lazarev proposed openstack/sahara: Add indirect VMs access implementation  https://review.openstack.org/13359021:55
openstackgerritAndrew Lazarev proposed openstack/sahara: Add indirect VMs access implementation  https://review.openstack.org/13359021:58
tmckaycrobertsrh, so filtering looks pretty simple.  I wonder if we should do any validation of the filter values passed in?22:03
crobertsrhHmm.  What do the other servies do?22:03
tmckayso, if like me you search for clusters with {'goat': 'cheese'} you get some sqlalchemy-ese saying there is no such entity22:04
crobertsrhI assume something is at least preventing sql injection, right?22:04
tmckayI don't know22:04
crobertsrhempty result sets are fine22:04
tmckayI would guess so.  It's still wrapped in query.filter_by()22:04
tmckayso squash the exception, and return {}?22:04
crobertsrhYeah22:05
tmckaythat sounds good to me.22:05
crobertsrhgarbage in -> nothing out22:05
tmckayI can key off the particular exception22:05
tmckayfrom the UI, it shouldn't be possible to ask for the wrong thing anyway22:05
tmckaycrobertsrh, okay, I should be able to post this tomorrow.  I might do a patch set for each object type22:06
tmckayIt will make the review simple22:07
elmikocrobertsrh: it shouldn't be possible sql exploit from the url with what we are using. _shouldn't_ being the operative there22:07
tmckayotherwise I think people will get lost in all the **kwargs22:07
crobertsrhSounds good.  I will probably get the UI done tomorrow too.22:07
tmckayelmiko, yeah, I am trusting the query.filter_by() stuff in sqlalchemy is checking22:07
elmikotmckay: i think flask also does filtering on the url requested to properly escape the input22:09
*** tmckay has left #openstack-sahara22:10
*** crobertsrh has quit IRC22:12
openstackgerritMerged openstack/sahara-specs: Adding query filtering support to Sahara  https://review.openstack.org/13431922:16
*** egafford has quit IRC22:20
openstackgerritMerged openstack/sahara-image-elements: Add openSUSE support for diskimage-create.sh  https://review.openstack.org/13517622:24
*** tellesnobrega has joined #openstack-sahara22:33
tellesnobregaalazarev, i'm working on the data_processing to data-processing bug. In horizon, as you pointed out, we need to change the definition of the service right?23:07
alazarevtellesnobrega, yeap23:08
tellesnobregawhat i'm working on is, changing the definition in saharaclient23:08
tellesnobregahorizon and devstack23:08
alazarevtellesnobrega, in Sahara it is docs only23:08
tellesnobregai already have a patch for the saharaclient23:08
tellesnobregacool23:09
tellesnobregabut it failed, i think the problem is that the devstack that runs the tempest is still using data_processing23:09
tellesnobregaso i will submit a patch for devstack first23:10
alazarevtellesnobrega, it is good question what should we do with pythonclient23:11
alazarevpythonclient is supposed to work with old versions of openstack23:11
tellesnobregathats true23:11
tellesnobregathats what i submitted https://review.openstack.org/#/c/135148/23:12
tellesnobregabut now that you pointed this out, maybe we need to rethink this patch23:12
tellesnobregahttps://review.openstack.org/#/c/135454/1 the devstack portion of the code23:14
alazarevtellesnobrega, voted on both patches23:16
tellesnobregathanks23:16
tellesnobregastill horizon to submit23:16
tellesnobregahorizon: https://review.openstack.org/13545823:35
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/python-saharaclient: Replacing data_processing with data-processing  https://review.openstack.org/13514823:48
openstackgerritAndrew Lazarev proposed openstack/sahara: Pylint check was broken after pylint update  https://review.openstack.org/13546423:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!