Sep 4, 2012 3:26 PM
zepConnectionError timeout service
-
Like (0)
Hello,
I recently installed Zenoss 4.2 in the network I manage, everything is working fine, except that I'm getting errors while trying to aknowledge or clear events. After adding all devices and organizing them I started to clear all the events that happen normally when first adding devices, but when I select one, two, some or all the events in the different devices and I click acknowledge or clear, it doesn't do anything for about one minute and popup a message saying "There was an error handling your request" and also a yellow bar appears at the top for a second saying "ZepConnectionError: Timed out connecting to service".
I have looked at all the logs and can't find something conclusive. Please guide me what could this mean and where to look for logs or hits about this error.
Thanks in advance!
printscreen of the messages is attached.
I am seeing a similar issue on 4.2.0 Core. Sometimes when I try to drill into an event the details do not appear and "ZepConnectionError: Timed out connecting to service" is logged in event.log.
I'm seeing the same issue as the poster on one of my instances. I've tried a ton of different stuff (including completely removing, reinstalling, and reconfiguring rabbitmq, and I still have the same issue. I sure would like to know a fix for this in case I run in to it again in the future but because of deadlines I may have to just give up and re-build this box. I'm working on troubleshooting this for the next little while. I'll post back here if I find a solution.
I'll add that this happened right out of the blue. All I did was add 250 network devices without disabling monitoring of the interfaces and left it sitting there. Since my instance polls for ifOperStatus and generates alerts it was sitting with over 5000 events in the console for a few days until I got back to working on it. It was then that I noticed the problem.
Ryan:
I want to resume trying to get this fixed later today, ping me in IRC.
--Shane
Just ran into this problem today. Any resolution?
I tracked it down to a component that appeared to be in the system but not linked to anything. Because of deadlines I had to rebuild the whole box but I have the screwed up image saved. I'll take another crack at fixing it when I have some time. Apparently the bug that causes this sort of thing to occur is fixed in the next version, but that's no good for people that it has happened to.
Here is the Jira ticket that I opened for the issue: http://jira.zenoss.com/jira/browse/ZEN-4565
Looks like ticket might show resolution for 4.2.3, but am now experiencing this exact same issue after upgrading from 4.2.0 to 4.2.3 about a month ago (issue just started).
Any ideas how to resolve?
--Rob
Rob,
Have you tried clearing your zep indexs?
--Shane
Shane,
First off thanks for the reply, but no, only a reindex(). Not sure how that task would be completed? Can you point me to a reference?
--Rob
Rob,
Ssh to the master then su zenoss; zopectl stop; zeneventserver stop; rm -Rf $ZENHOME/var/zeneventserver/index; zeneventserver start; zopectl start
--Shane Scott (Hackman238)
Shane,
commands ran fine and system came up but still same issue, any ideas?
--Rob
From event.log:
2013-06-24T07:37:17 INFO extdirect Direct request failed: Timed out connecting to service.: EventsRouter.acknowledge [{'evids': [], 'params': {'severity': [5, 4, 3], 'eventClass': '/status/ping*', 'eventState': [0, 1, 2], 'tags': []}, 'limit': 100, 'excludeIds': {}}]
------
2013-06-24T07:37:17 INFO extdirect DirectRouter suppressed the following exception (Response b29b2948-a82a-4809-8593-a6c7d286e370):
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenUtils/extdirect/router.py", line 179, in _processDirectRequest
response.result = _targetfn(**data)
File "<string>", line 2, in acknowledge
File "/opt/zenoss/Products/Zuul/decorators.py", line 69, in wrapped_fn
return f(self, *args, **kwargs)
File "/opt/zenoss/Products/Zuul/routers/zep.py", line 537, in acknowledge
limit=limit,
File "/opt/zenoss/Products/Zuul/facades/zepfacade.py", line 410, in acknowledgeEventSummaries
limit)
File "/opt/zenoss/lib/python/zenoss/protocols/services/zep.py", line 162, in acknowledgeEventSummaries
limit=limit)
File "/opt/zenoss/lib/python/zenoss/protocols/services/zep.py", line 102, in updateEventSummaries
search_uuid = self.createSavedSearch(event_filter=event_filter, exclusion_filter=exclusion_filter, timeout=timeout)
File "/opt/zenoss/lib/python/zenoss/protocols/services/zep.py", line 191, in createSavedSearch
status, response = self.client.post(url, query)
File "/opt/zenoss/lib/python/zenoss/protocols/services/__init__.py", line 175, in post
return self._request_with_body(path, method='POST', body=body, params=params, headers=headers)
File "/opt/zenoss/lib/python/zenoss/protocols/services/__init__.py", line 170, in _request_with_body
response, content = self._executeRequest(request)
File "/opt/zenoss/lib/python/zenoss/protocols/services/__init__.py", line 142, in _executeRequest
raise self._connection_error_class('Timed out connecting to service.', e)
ZepConnectionError: Timed out connecting to service.
Rob,
Can you post your zeneventserver.log and mysql log?
--Shane
Shane,
Can't seem to find how to attach the logs to the post - Going to try replying to the email?
--Rob
Follow Us On Twitter »
|
Latest from the Zenoss Blog » | Community | Products | Services Resources | Customers Partners | About Us | ||
Copyright © 2005-2011 Zenoss, Inc.
|
||||||||