Hi All,

We are running the latest version of Zenoss open source on a Quad Core 3GB ram server.I have it all setup with the zenpacks/templates I want and it was working fine, once I added 35 devices, I can't seem to model any device anymore nor can I add a new device.

Adding/Modeling devices with snmp off, and only ssh on works fine, but snmp modeling has stopped working. The monitoring system is working fine though.

Solutions I have tried:

Removing all snmp collector plugins and try to model:

Adding Devices in different classes: /Devices/Linux, /Devices/SSH/Linux etc

Currently I am storing the devices in the class SSH/Linux/ with snmp on because Openmanage uses SNMP and we use SSH collector plugins as well. Once again everything was working fine and this has popped out of nowhere, I still have another 45 servers to add.

Here is the errors I get when I attempt to model a existing device.

monitor:~ # /usr/local/zenoss/zenoss/bin/zenmodeler run --now --monitor localhost -F -d doozer167
2010-04-27 11:34:53,995 INFO zen.ZenModeler: Connecting to localhost:8789
2010-04-27 11:34:54,009 INFO zen.ZenModeler: Connected to ZenHub
2010-04-27 11:34:54,491 INFO zen.ZenModeler: Collecting for device doozer167
2010-04-27 11:34:54,650 INFO zen.ZenModeler: No WMI plugins found for doozer167
2010-04-27 11:34:54,653 INFO zen.ZenModeler: No Python plugins found for doozer167
2010-04-27 11:34:54,656 INFO zen.ZenModeler: Using SSH collection method for device doozer167
2010-04-27 11:34:54,656 INFO zen.ZenModeler: plugins: zenoss.cmd.uname, zenoss.cmd.uname_a, zenoss.cmd.df, zenoss.cmd.linux.cpuinfo, zenoss.cmd.linux.memory, zenoss.cmd.linux.ifconfig, zenoss.cmd.linux.netstat_an, zenoss.cmd.linux.netstat_rn, zenoss.cmd.linux.process, zenoss.cmd.linux.suse_uname_a, zenoss.cmd.linux.suse_rpm
2010-04-27 11:34:54,661 INFO zen.ZenModeler: SNMP collection device doozer167
2010-04-27 11:34:54,661 INFO zen.ZenModeler: plugins: DellCPUMap, DellPCIMap, DellDeviceMap, zenoss.snmp.HRSWInstalledMap, zenoss.snmp.InformantHardDiskMap
2010-04-27 11:34:54,737 INFO zen.ZenModeler: No portscan plugins found for doozer167
2010-04-27 11:34:54,737 INFO zen.ZenModeler: Running 2 clients
2010-04-27 11:34:55,062 INFO zen.SshClient: Connected to device doozer167
2010-04-27 11:34:56,407 INFO zen.CmdClient: command client finished collection for doozer167
2010-04-27 11:34:56,407 INFO zen.ZenModeler: Collecting routes for device doozer167
2010-04-27 11:34:56,408 INFO zen.ZenModeler: Processing the uname -a info for device doozer167
2010-04-27 11:34:56,409 INFO zen.ZenModeler: Collecting filesystems for device doozer167
2010-04-27 11:34:56,411 INFO zen.ZenModeler: Processing the uname -a info for device doozer167
2010-04-27 11:34:56,411 INFO zen.ZenModeler: Modeler zenoss.cmd.linux.ifconfig processing data for device doozer167
2010-04-27 11:34:56,448 INFO zen.ZenModeler: Collecting process information for device doozer167
2010-04-27 11:34:56,454 INFO zen.ZenModeler: Processing the OS uname info for device doozer167
2010-04-27 11:34:56,455 INFO zen.ZenModeler: Collecting CPU information for device doozer167
2010-04-27 11:34:56,457 INFO zen.ZenModeler: Collecting memory and swap for device doozer167
2010-04-27 11:34:56,457 INFO zen.ZenModeler: Modeler zenoss.cmd.linux.netstat_an processing data for device doozer167
2010-04-27 11:34:56,458 INFO zen.ZenModeler: Collecting Software Installed information for device doozer167
2010-04-27 11:35:03,309 INFO zen.SnmpClient: snmp client finished collection for doozer167
2010-04-27 11:35:03,312 INFO zen.ZenModeler: processing DellCPUMap for device doozer167
2010-04-27 11:35:03,314 INFO zen.ZenModeler: processing DellPCIMap for device doozer167
2010-04-27 11:35:03,316 INFO zen.ZenModeler: processing DellDeviceMap for device doozer167
2010-04-27 11:35:03,329 INFO zen.ZenModeler: processing zenoss.snmp.HRSWInstalledMap for device doozer167
2010-04-27 11:35:03,381 INFO zen.ZenModeler: processing zenoss.snmp.InformantHardDiskMap for device doozer167

2010-04-27 11:36:59,453 WARNING zen.ZenModeler: Client doozer167 timeout
Traceback (most recent call last):
  File "/usr/local/zenoss/zenoss/Products/DataCollector/zenmodeler.py", line 999, in ?
    dc.run()
  File "/usr/local/zenoss/zenoss/Products/ZenHub/PBDaemon.py", line 272, in run
    reactor.run()
  File "/usr/local/zenoss/zenoss/Products/DataCollector/zenmodeler.py", line 908, in reactorLoop
    reactor.runUntilCurrent()
--- <exception caught here> ---
  File "/usr/local/zenoss/python/lib/python2.4/site-packages/Twisted-8.1.0-py2.4-linux-i686.egg/twisted/internet/base.py", line 705, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/usr/local/zenoss/zenoss/Products/DataCollector/zenmodeler.py", line 893, in timeoutClients
    self._timeoutClients()
  File "/usr/local/zenoss/zenoss/Products/DataCollector/zenmodeler.py", line 877, in _timeoutClients
    client.stop()
  File "/usr/local/zenoss/zenoss/Products/DataCollector/SnmpClient.py", line 170, in stop
    self.proxy.close()
  File "/usr/local/zenoss/python/lib/python2.4/site-packages/pynetsnmp/twistedsnmp.py", line 189, in close
    assert self.session
exceptions.AssertionError:

It will loop the above errors over and over and eventually die out. At first thought it was running out of memory but I added another 1GB and it is not even using all the available memory and still throws these errors.

Any help is greatly appreciated otherwise I will have to start from scratch which is almost 2 weeks of work.

Thanks

Darvesh