Oct 16, 2012 5:23 AM
zenhub dies seconds after starting
-
Like (0)
Hello all.
I have an old 3.2.1 instance that I have been toying around with for a while, testing Zenoss and monitoring a few devices (35-40 servers). Recently I was ready to move into production state, and wanted to upgrade to the latest Zenoss version. I decided to do a fresh start to get away from the old stack installer that I was using, and wasn't planning on migrating any settings - the old system was ready to be ditched completely. Hence, yesterday I installed a CentOS 6.3 basic server and went through the autodeploy-process. Within minutes, my shiny, new Zenoss Core 4.2.0 instance was up and running.
For a couple of hours, I hardened CentOS a bit (enabling firewall, tightening ssh) and started adding devices to Zenoss. Everything was working perfectly, but then I noticed that the system clock was two hours off, and I startet NTP to get an accurate clock. I also renamed the server (as it was moved from preprod to production). I am not sure if this is the reason for the problems that followed, but thought I should mention it.
Now, after a system reboot and/or Zenoss restart, zenhub appears to be starting, then dies. All other Zenoss processes start as expected. In the zenhub.log, I get the following entries:
2012-10-16 10:43:45,330 INFO zen.HubService.RenderConfig: Starting graph retrieval listener on port 8090
2012-10-16 10:43:48,342 INFO zen.zenoss.protocols.amqp: amqp connection was closed [Errno 104] Connection reset by peer
2012-10-16 10:43:51,346 INFO zen.zenoss.protocols.amqp: amqp connection was closed [Errno 104] Connection reset by peer
2012-10-16 10:43:51,346 ERROR zen.ZenHub: Unable to send an event
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenHub/zenhub.py", line 576, in sendEvent
self.zem.sendEvent(Event(**kw))
File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 60, in sendEvent
event = self._publishEvent(event)
File "/opt/zenoss/Products/ZenEvents/MySqlSendEvent.py", line 82, in _publishEvent
publisher.publish(event)
File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 283, in publish
self._publish("$RawZenEvents", routing_key, event, mandatory=mandatory, immediate=immediate)
File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 302, in _publish
mandatory, immediate)
File "/opt/zenoss/Products/ZenMessaging/queuemessaging/publisher.py", line 376, in publish
headers=headers, declareExchange=declareExchange)
File "/opt/zenoss/lib/python/zenoss/protocols/amqp.py", line 138, in publish
raise Exception("Could not publish message. Connection may be down")
Exception: Could not publish message. Connection may be down
2012-10-16 10:43:57,255 INFO zen.zenoss.protocols.amqp: amqp connection was closed [Errno 104] Connection reset by peer
2012-10-16 10:44:00,259 INFO zen.zenoss.protocols.amqp: amqp connection was closed [Errno 104] Connection reset by peer
I can log in to the web GUI, and I can see all my devices, but no errors are logged, and no graphs are graphed. I can no longer add anything to the system (no users or devices), and no changes to the devices/groups/systems/locations hierarchy can be made. Basically, I have a shiny new system that tells me that nothing is wrong as no events are received or handled, but clearly something is very wrong.
Does Zenoss store the hostname and/or IP address in any of it's config files or the database, causing these problems when changing name/IP?
Does any of you know where I should start looking, to avoid spending hours looking in the wrong place? Instead of spending a workday or two trying blindly to fix the problem, I might as well reinstall CentOS and Zenoss, but that wouldn't teach me anything and wouldn't be any fun.
Thanks in advance!
Follow up:
I have reverted to the original hostname and DNS name, and everything seem to be working OK now. I guess some config file or database entry had the old hostname stored inside it, but I was unable to find it.
have a read through message/67159#67159
Follow Us On Twitter »
|
Latest from the Zenoss Blog » | Community | Products | Services Resources | Customers Partners | About Us | ||
Copyright © 2005-2011 Zenoss, Inc.
|
||||||||