Tuesday, 2018-08-28

*** Bhujay has joined #openstack-sahara04:18
*** Bhujay has quit IRC04:19
*** tmckay has quit IRC06:02
*** links has joined #openstack-sahara06:16
*** pcaruana has joined #openstack-sahara06:39
*** rcernin has quit IRC07:02
*** tosky has joined #openstack-sahara07:58
*** rcernin has joined #openstack-sahara08:59
*** hoonetorg has quit IRC09:20
*** rcernin has quit IRC09:47
tellesnobregatosky, have you seen this error come up?11:54
tellesnobregaFile "/usr/lib64/python3.6/selectors.py", line 348, in __init__11:54
tellesnobrega    self._poll = select.poll()11:54
tellesnobregaAttributeError: module 'select' has no attribute 'poll'11:54
toskytellesnobrega: no, but a quick search may show a connection with eventlet11:58
tellesnobregathis is showing up when I run the tests from plugin split12:00
toskyI see that it's using python3; is the non-split code working with python3 too?12:01
tellesnobregaweird enough, when I run on sahara master code using tox -e debug it appears, but not when I run tox -e py27 or tox -e py3612:02
toskyuhm, if it's really about eventlet, maybe some issue with monkeypatching?12:03
tellesnobregathat is what I read12:04
tellesnobregasomeone saying that gevent removed poll from subprocess12:04
tellesnobregaand its failing now12:04
tellesnobregathat is on the gevent project12:05
tellesnobregamaybe eventlet does something similar12:05
tellesnobregahttps://github.com/gevent/gevent/issues/44612:05
tellesnobregahttp://git.openstack.org/cgit/openstack/glance/commit/?id=630402204e83b9e36c6822fc251fa06cf28963ac12:06
toskyfor the record, and it matches the expectation, I see the same error on the current (unsplit) master12:09
toskywe probably need to call the code from sahara/utils/patches.py also somewhere else, maybe12:10
toskysahara/tests/__init__.py call patches.patch_all(), which should take care of it12:10
tellesnobregayes12:11
toskyand I even fixed it12:11
toskyO.o12:11
tellesnobregajust looking at it12:11
openstackgerritTobias Urdin proposed openstack/puppet-sahara stable/rocky: Update Gemfile for stable/rocky  https://review.openstack.org/59709312:11
toskyI totally forgot about https://review.openstack.org/#/c/449261/12:12
tellesnobregai guess I forgot to pull that into the plugins part12:13
tellesnobregaI'm down to 65 tests failing on the plugins side now12:14
tellesnobregasahara side is still a bunch12:14
tellesnobregabut I don't think it will be too much trouble12:14
tellesnobregaonce I'm done with the plugins side12:14
tellesnobregaI will ask you to take a look on how I did things so we can discuss if there is anything I can do to improve the architecture12:15
toskyare you keeping the fake plugin inside sahara.git, so that it can be tested standalone?12:15
tellesnobregaI will do that next12:15
tellesnobregait is not there right now12:15
tellesnobregabut I will move it back12:16
toskyand here is my description of the eventlet issue (thanks, logs): http://eavesdrop.openstack.org/irclogs/%23openstack-sahara/%23openstack-sahara.2017-03-22.log.html#t2017-03-22T17:53:2312:16
tellesnobregajust have to figure out how to use stevedore to load from different sources12:16
toskyI thought it did the magic somehow12:17
tellesnobregatosky, I'm not sure, will look12:52
*** cmart has joined #openstack-sahara13:07
openstackgerritxuan.mingyi proposed openstack/puppet-sahara master: Add the missing 'Table of Contents' in README  https://review.openstack.org/59681313:43
toskytellesnobrega: can you please +1 again the release of sahara 8.0.3? https://review.openstack.org/#/c/596409/13:54
toskyI added the last fix13:54
tellesnobregatosky, yes13:54
*** links has quit IRC14:27
*** pcaruana has quit IRC14:29
*** pcaruana has joined #openstack-sahara14:30
*** remix_tj has joined #openstack-sahara14:31
remix_tjhi tosky are you there?14:31
toskyremix_tj: hi14:31
remix_tji've an ancient newton setup where a cluster is in DELETING status since days14:32
remix_tji've seen this https://bugs.launchpad.net/sahara/+bug/1647411 but i'm sure is not implemented, because i've not updated recently14:32
openstackLaunchpad bug 1647411 in Sahara "Clusters can stay in Deleting state forever" [Medium,Triaged]14:32
remix_tjdo you know any workaround to unlock the situation? i can't update node template because is in use14:33
*** Bhujay has joined #openstack-sahara14:33
*** Bhujay has quit IRC14:34
*** Bhujay has joined #openstack-sahara14:35
toskyremix_tj: is the underlying heat object removed?14:35
remix_tjtosky: seems no, openstack stack list --hidden shows a DELETE_FAILED stack with the matching name14:35
toskyremix_tj: kill that14:36
remix_tjstack delete is enough?14:36
toskyremix_tj: also, --hidden --nested14:36
toskybecause it may be stuck on deletion of some subelement14:36
toskyyes, start with openstack stack delete on that top-level element14:36
remix_tjtosky: i'm sure it was suck on creation because nova had problems because if network troubles and redis was not serving nova-* correctly14:37
remix_tjhas moved to DELETE_IN_PROGRESS i keep the situation monitored and i case i ask you again14:40
remix_tjthanks14:40
remix_tjResource DELETE failed: Conflict: resources.clouderarocketv2-clouderanodegrouptemplate-8652b800: Security Group e2bae36c-31f5-4c97-a3a5-d1b260b9441d in use14:41
remix_tjwow but i don't have any instance!14:41
toskyjust force it again14:42
toskythe deletion14:42
remix_tjyup14:42
remix_tjmaybe there is still something using that group14:43
*** cmart has quit IRC14:46
*** Bhujay has quit IRC14:58
*** pcaruana has quit IRC15:00
*** cmart has joined #openstack-sahara15:33
*** cmart has quit IRC15:35
toskyremix_tj: was it finally removed?16:06
remix_tjno because is stuck on deleting a security group that cannot be removed16:06
remix_tjtosky: there were a pending creation of a server, which has failed due to troubles on nova. but the network port were allocated and locking the secgroup. I had to make a query on the db to find which neutron port was holding16:10
remix_tjthe secgroup16:10
remix_tjhttps://access.redhat.com/solutions/300981116:10
remix_tjbut now the stack disappeared16:10
*** dave-mccowan has joined #openstack-sahara16:13
remix_tjdeletion completed, thanks tosky16:13
toskyremix_tj: so hopefully the cluster should disappear too16:14
remix_tji executed again openstack dataprocessing cluster delete and disappeared16:14
remix_tjlast simple question16:15
remix_tji have a cluster which is reporting Creating cluster failed for the following reason(s): CM API error: HTTP Error 400: Bad Request Error ID: 6e28c85d-6a99-485d-aea6-bd0245d9d5d216:15
remix_tjhow do i see which request was and what happened? that error id is about what?16:15
remix_tji see in cloudera manager some errors, but i want to understand what went wrong from outside16:16
*** Bhujay has joined #openstack-sahara16:17
toskyif cloudera manager did not report a better error, there is no much to do16:17
toskyif you set the logging to debug level and no other details are in sahara-engine.log, then only cloudera manager can say what happened16:17
toskysahara wraps the requests to cloudera manager16:18
remix_tjok, debug is required. I think is because there has been missing parameters on the node template side, but i'm not sure, i don't know cloudera16:18
remix_tjand the cluster is being deployed by a data analyst, so don't know also what has set16:19
remix_tjtosky: but the error id is an openstack id or CM id?16:21
*** Bhujay has quit IRC16:22
remix_tjgoing home, see you tomorrow16:23
toskyI don't remember16:26
*** Bhujay has joined #openstack-sahara16:30
*** Bhujay has quit IRC16:43
*** Bhujay has joined #openstack-sahara16:44
*** Bhujay has quit IRC17:24
*** tosky has quit IRC18:23
openstackgerritMerged openstack/puppet-sahara stable/rocky: Update Gemfile for stable/rocky  https://review.openstack.org/59709318:27
*** dave-mccowan has quit IRC18:36
*** dave-mccowan has joined #openstack-sahara18:46
*** hoonetorg has joined #openstack-sahara19:22
*** hoonetorg has quit IRC19:24
*** dave-mccowan has quit IRC19:26
*** hoonetorg has joined #openstack-sahara21:36
*** rcernin has joined #openstack-sahara21:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!