I'm seeing the same issue as the poster on one of my instances.  I've tried a ton of different stuff (including completely removing, reinstalling, and reconfiguring rabbitmq, and I still have the same issue.  I sure would like to know a fix for this in case I run in to it again in the future but because of deadlines I may have to just give up and re-build this box.  I'm working on troubleshooting this for the next little while.  I'll post back here if I find a solution.

Ryan:

I want to resume trying to get this fixed later today, ping me in IRC.

--Shane

I tracked it down to a component that appeared to be in the system but not linked to anything.  Because of deadlines I had to rebuild the whole box but I have the screwed up image saved.  I'll take another crack at fixing it when I have some time.  Apparently the bug that causes this sort of thing to occur is fixed in the next version, but that's no good for people that it has happened to.

Looks like ticket might show resolution for 4.2.3, but am now experiencing this exact same issue after upgrading from 4.2.0 to 4.2.3 about a month ago (issue just started).

Any ideas how to resolve?

--Rob

Shane,

First off thanks for the reply, but no, only a reindex().  Not sure how that task would be completed?  Can you point me to a reference?

--Rob

Shane,

commands ran fine and system came up but still same issue, any ideas?

--Rob

From event.log:

2013-06-24T07:37:17 INFO extdirect Direct request failed: Timed out connecting to service.: EventsRouter.acknowledge [{'evids': [], 'params': {'severity': [5, 4, 3], 'eventClass': '/status/ping*', 'eventState': [0, 1, 2], 'tags': []}, 'limit': 100, 'excludeIds': {}}]

------

2013-06-24T07:37:17 INFO extdirect DirectRouter suppressed the following exception (Response b29b2948-a82a-4809-8593-a6c7d286e370):

Traceback (most recent call last):

  File "/opt/zenoss/Products/ZenUtils/extdirect/router.py", line 179, in _processDirectRequest

    response.result = _targetfn(**data)

  File "<string>", line 2, in acknowledge

  File "/opt/zenoss/Products/Zuul/decorators.py", line 69, in wrapped_fn

    return f(self, *args, **kwargs)

  File "/opt/zenoss/Products/Zuul/routers/zep.py", line 537, in acknowledge

    limit=limit,

  File "/opt/zenoss/Products/Zuul/facades/zepfacade.py", line 410, in acknowledgeEventSummaries

    limit)

  File "/opt/zenoss/lib/python/zenoss/protocols/services/zep.py", line 162, in acknowledgeEventSummaries

    limit=limit)

  File "/opt/zenoss/lib/python/zenoss/protocols/services/zep.py", line 102, in updateEventSummaries

    search_uuid = self.createSavedSearch(event_filter=event_filter, exclusion_filter=exclusion_filter, timeout=timeout)

  File "/opt/zenoss/lib/python/zenoss/protocols/services/zep.py", line 191, in createSavedSearch

    status, response = self.client.post(url, query)

  File "/opt/zenoss/lib/python/zenoss/protocols/services/__init__.py", line 175, in post

    return self._request_with_body(path, method='POST', body=body, params=params, headers=headers)

  File "/opt/zenoss/lib/python/zenoss/protocols/services/__init__.py", line 170, in _request_with_body

    response, content = self._executeRequest(request)

  File "/opt/zenoss/lib/python/zenoss/protocols/services/__init__.py", line 142, in _executeRequest

    raise self._connection_error_class('Timed out connecting to service.', e)

ZepConnectionError: Timed out connecting to service.

Shane,

Can't seem to find how to attach the logs to the post - Going to try replying to the email?

--Rob