Archived community.zenoss.org | full text search
Skip navigation
Currently Being Moderated

Zenoss Users Forum FAQ

VERSION 17  Click to view document history
Created on: Sep 14, 2009 11:21 AM by Matt Ray - Last Modified:  May 17, 2011 11:49 AM by Nick Yeates

Please Note: A second extended version of this FAQ is updated here#: Zenoss Users Forum FAQ part 2 Be sure to check both.

 

#ZenossUsersForumFAQ

These FAQs are literally questions that come up enough times on the Zenoss User forum that it's quicker to link here then to copy in or re-type the answer, questions that forum members think will come up a bunch in the future and questions pulled from "Getting Started With Zenoss" Webinars.

 

 

New Users

What should I know before using Zenoss?

There are some things that are helpful in installing, configuring and using Zenoss. You don't have to know any of them, but you probably will hit up problems as you go if you don't have some knowledge of the upper items in the list. I will list them in roughly decending order.

  1. (VMWare Administration) - if you're running the VMware Appliance 
    1. (VMWare Converter) - if you want to run on ESX
  2. Linux Administration
    1. Bash use
    2. Bash (Shell) Scripting
  3. OS Administration and configuration of whatever you are monitoring.
    1. Especially how to configure:
      1. SNMP
      2. WMI
      3. SSH
  4. SNMP
  5. Python
  6. Zope

These are just a general overview from my own experiance and from reading on the forums, using the generally expected method for new users. (If you've decided to install on Solaris, of course replace Linux Administration with Solaris Administration. If you're compiling from source, add in knowledge of the build environment you're using. etc...)

Some comments - You probably can use the VMWare Appliance with just knowledge of 1, but you probably won't be able to monitor much that isn't out of the box supported. I'd suggest getting conversant with 2 and 3 at a minimum if you want things to work smoothly; 4 and 5 will really let you customize your monitoring of just about anything.

I'm also not really going into what level of knowledge is needed, but knowing that you can use

su - zenoss

to get to the Zenoss User's CLI in Linux is going to be pretty useful if you want to extend Zenoss.

What Browsers are Supported?

  • Zenoss officially supports   
    • IE7 and 8 (Zenoss 2.5.x requires compatibility mode for IE8 to fully function)
    • Firefox 3 and 3.5
    • You can report bugs against these, and they will probably get fixed.
  • Community members successfully use:  
    • Opera 10 for most functions
    • Safari 3/4 mostly works
    • Chrome mostly works
    • Zenoss isn't going to actively work on bugs with these. You can submit patches for any browser though.
  • Known to have issues:  
    • IE6 (not going to support directly)

What OS should I install Zenoss on?

  • Most Enterprise users are using Red Hat Enterprise Linux v4 or v5, or it's derivatives.
  • Community members have been successful using (In roughly ease of install order. The latter options are quite involved):  
    • CentOS5, Scientific Linux 5 (RHEL derivatives) (RPM or BIN stack)
    • Ubuntu/Debian (DEB or BIN stack installer)
    • SLES and OpenSUSE (BIN stack installer)
    • Mac OSX (stack installer)
    • FreeBSD (source only)
    • Solaris (source only)

Can I run Zenoss on Windows and monitor Linux devices?

  • Not really. Zenoss is POSIX based, and usually only runs on the various *nix's with varying degrees of ease. So, you cannot install Zenoss onto Win32 for instance directly.
  • You CAN use VMWare to run the appliance (an rPath Linux) on anything VMWare runs on, including Windows. But you're not really installing on Windows, and you still will need to have Linux knowledge to extend, customize and troubleshoot your rPath Linux install.
  • Any install of Zenoss, native or VMWare can monitor the same things.

What Webserver does Zenoss use?

Zenoss uses the ZOPE environment, which does its own webserving on port 8080. Some users are using Apache as a "front end" with local redirection to enable SSL.

What is the maximum number of devices Zenoss can monitor?

  • This is a hard question to answer. Really there is no limit, it depends on your hardware and the number of things you are polling. This is not the number of devices, but the number of OIDs, Pings, WMI Queries, Command Datasources etc...
  • You could have a very large number of Ping only monitored devices but a smaller number of fully SNMP monitored devices on the same hardware.
  • The Forums have discussions on sizing, tuning, and more:  

Should I use the VMWare Appliance?

  • You can. It is somewhat different than using a standard OS install, and you will want to familarize yourself with rpath Linux if you choose to use it.
  • You will want to increase the resources (RAM, Disk) over what it comes configured as for anything more than a demo.
  • You can run the Appliance on VMWare Player or VMWare Server without conversion.
  • You will need to convert the Appliance to use with ESX.

How do I set a static IP in the VMWare Appliance?

  1. Open the console of the VMWare appliance.
  2. Edit the config file using vi or nano
    nano /etc/sysconfig/network-scripts/ifcfg-eth0
  3. Change the following settings
    BOOTPROTO=static 
    IPADDR=x.x.x.x
    NETMASK=x.x.x.x
    NETWORK=x.x.x.x
    ONBOOT=yes
    TYPE=Ethernet

How can I monitor my devices and servers?

  • SSH
  • WMI
  • SNMP
  • Command Based
  • JMX

See the section on each below.

Do I need to install Agents on my monitored Servers?

  • No. Zenoss does not use Agents, though you can use SSH and some plugins which are sort of like an agent if you want.
  • Zenoss can use all sorts of methods to monitor devices, SNMP is probably the most common.

Is Zenoss used often for workstation monitoring?

There are a lot of users using it for monitoring workstations in conjunction with their other monitoring.  Since it supports WMI, SSH and SNMP it is easy to monitor workstations.

How do I enable my Zenoss Download?

  • The Community version - Zenoss Core, is GPLv2 and free to use. You just install it and go.
  • The Enterprise version needs a fee paid, and has a License agreement beyond the GPL. Contact Sales for more information.

What version do I get if I download from the Zenoss.com site?

  • The Core / Community version.

How do I install Zenoss?

  • See the Zenoss installation guide: http://www.zenoss.com/community/docs/
  • Note there are several guides, depending on what and how you're installing Zenoss. It can be worth while to read over the guides, but also to search the forums for topics on source vs RPM vs Stack install.

What is the difference between the 3 installation methods (Source, Stack, Package)?

  • Zenoss provides a Stack or Bin installer based on a Bitrock installer program. This installer works somewhat like a Windows installer. You run the program, it installs Zenoss, the needed dependancies, and extra packages like MySQL that you need to use it. Most of these are all installed specially for use by Zenoss. This is a good install method if you have a clean Linux install, and want to have Zenoss manage all parts of the Zenoss Stack.  
    • Note: This can be problematic if you want this OS instance or server to run other applications in addition to Zenoss, especially MySQL or JBoss.
    • The .DEB file is actually a stack installer that is managed by apt, so it contains all of its dependencies including MySQL.
  • Linux normally uses packages (usually via repositories, but you can download them as well) to install and manage programs. On Red Hat this is RPM and on Debian and derivatives (Ubuntu etc) it is DEB. You use the appropriate package manager for your platform (which is outside the scope of this document, see the OS documents) to install the package you've downloaded. The package manager will handle most dependencies, though you may need to install some components separately, like MySQL. This is a good way to manage multiple applications on top of MySQL...  
    • Note: This setup is more manual work than using the Stack installer.
    • Both the RPM and DEB packages include dependencies for the Zenoss platform, rather than proper distribution dependencies.
  • Source installs are usually a last resort to get a program working on a platform / OS. If you can use the Stack installer OR the Package, use those.  
    • Note: If you're thinking of a source install, you don't need any install instructions here, you should already know what you're doing.

Does Zenoss provide CMDB services?

Zenoss records all configuration data in an object database that can be accessed through reports, custom reports, the command line, programatically and remotely.

Are there any discovery protocols besides SNMP?

Discovery in 2.4 can use WMI, SNMP or SSH.

How do I reset my admin password?

As the zenoss user:

$ZENHOME/bin/zenpass
zopectl restart

How do I become the Zenoss User?

There are many instructions for configuring Zenoss that start "As the Zenoss User". This is a generic Linux use question, generally at the command line:

su - zenoss

Note: It's probably not obvious to many new users, but any development of custom datasources, event commands and the like should be done or at least tested from the zenoss user account.

I've posted in the forums, why is no one is helping me?

  1. Make sure you've followed the guidelines here:  
  2. Try IRC as an alternative.  
    • use irc.freenode.net
    • #zenoss
  3. It's always possible no one knows the answer who has time to reply. Sadly, only about 50% of forum posts get a solution, though it is trending upwards as more users join the community and gain expertise.

SNMP

What SNMP versions are supported?

  • SNMP v1 is supported for polling OIDs and for TRAPS
  • SNMP v2c is supported for polling OIDs and for TRAPS
  • SNMP v3 is supported only for polling OIDs.   
    • Note, you can use SNMP v3 for monitoring and still handle v2 or v1 traps. SNMP v3 Traps have not been added to 2.3.3 or to the upcoming 2.4 release.

What version of SNMP should I use?

That is pretty much up to you. Zenoss supports SNMP v1, SNMP v2 completely and SNMP v3 OID monitoring but not SNMP v3 traps.  SNMP v2 has better performance than v1 and v3 adds additional security features.  Most documentation uses SNMP v1 for simplicity.

How do I set up SNMP with my devices?

  • There are entire books on SNMP, and pretty much every device or OS is slightly different.
  • The basics are covered in our documentation.  The Getting Started with Zenoss guide and the Administration Guide both have basic information on configuring SNMP. http://www.zenoss.com/community/docs/  Depending on the device, you may need to get further documentation from your vendor or distribution.
  • For Windows Server 2003, you need to go into the SNMP service configuration, and allow SNMP from the Zenoss Server IP. You'll also want to make sure that the clients and Zenoss are using the same community string. The defaults should work however.
  • For most Linux clients (and net-snmp in general), you can get away with the following in your /etc/snmp/snmpd.conf:   
    • rocommunity public
    • This isn't particularily secure, but it is easy to get going.
  • Specifically, for retrieving software info see: http://blog.zenoss.com/2009/02/18/tip-of-the-month-snmp-software-inventory-for-debian-and-ubuntu-machines/
  • Most network devices have a place in their config for IPs to allow SNMP from, and where to send SNMP Traps. Point both to the Zenoss Server IP.

How do I have my Device's send SNMP Traps to the Zenoss Server?

Everything above on SNMP setup applies. A simple setup would be to: Put a line in the /etc/snmp/snmpd.conf file like the following.

trapsink 10.10.10.1 public 162

Replace the 10.10.10.1 IP with the IP address of your Zenoss server.

How do I troubleshoot SNMP connectivity?

See this community HOWTO.

Why doesn't Zenoss pick up my NFS mountpoints in monitoring?

There is Net-SNMP option that can be put in the /etc/snmp/snmpd.conf   called "storageUseNFS" that could be used to expose the NFS information.

storageUseNFS 2

Windows

How do I use WMI with my Windows Servers?

You'll probably want to be more specific, but in general:

  1. Add the Server to /Devices/Server/Windows
  2. Set the zProperties zWinUser and zWinPassword to an Administrator account on the Windows server.  
    • Note, formatting is very important. For a Domain Admin account, use DOMAINusername. For a local Admin account, use .username
  3. Wait for the service information to be pulled.

Do I have to use an Administrator user to access WMI?

No, but it is somewhat complex to use a LUA account in this instance. Be aware you'll need to be pretty comfortable with reading Microsoft KBs...

  1. Create a limited user account or group.
  2. Use the WMI Control (found in the Computer -> Manage console) to add permissions to the rootcimv2 namespace for your user and/or group. See: http://technet.microsoft.com/en-us/library/cc787533.aspx
  3. Use dcomcnfg to give Remote Launch & Remote Activation permissions for your user and/or group. See: http://msdn.microsoft.com/en-us/library/aa393266.aspx
  4. For event log monitoring, you have to grant permissions to the user and/or group for the specific event logs you want access to (this is normally a very restrictive task). See: http://support.microsoft.com/kb/323076

For Enterprise Customers, there is also:

  1. If you are using our ZenWinPerf zenpack then you'll also want to allow remote registry access for the performance data to be read. See: http://support.microsoft.com/default.aspx?scid=kb;en-us;164018

You can also set all of this up across multiple machines using a domain group policy (although the DCOM remote privilege is not easily done this way).

Thanks to cgibbions on the forums for this.

Why is my Windows 2003 server not filling all perf tab graphs? I have WMI setup.

  • You also need SNMP Informant(free) installed on the server for full graphs.
  • Alternatively, look at the WMI Zenpack to use native polling rather than the Informant MIBs.

Can I use WMI instead of SNMP Informant to get Memory on Windows Servers?

Yes! There are several optional ZenPacks that get all sorts of information via WMI or built in Windows SNMP instead of using SNMP Informant.

  • The best method is to use the WMI Windows Performance ZenPack as this uses a WMI based collector rather than command datasources. This then protects your service account credentials.  
    • Note that this does have the WMI Datasource Zenpack dependency. This is a good generic way to make your own WMI calls, as it also will protect credentials whereas wmic command based datasources do not.

The following Zenpacks are older and use command based datasources. These may not scale as well and may expose service account credentials in Events. However, the Dastrup one also obtains additional information, and both have been in use for over a year by community members.

How do I troubleshoot WMI?

See this community HOWTO.

Is there any way to specify what event log entries get brought over as events in Zenoss?

There is a zProperty called zWinEventlogMinSeverity that sets the minimum severity evenlog that will be picked up by Zenoss. It is a bit confusing since windows event logs actually use a reverse severity scheme. Here's the table:

1 = Error 
2 = Warning
4 = Information
8 = Security audit success
16 = Security audit failure

I believe the default setting of zWinEventlogMinSeverity is 2. This would get you warning and error events.

How do I create my own WMI Queries?

There are two methods:

  1. You could use wmic and command data sources.
  2. You can install the new WMI datasource and use that.

 

Zenoss GUI Interface

Can I set granular permissions for users?

  • The answer is, it depends.   
    • In Enterprise, you can get as granular as you want, and this is fully supported by Zenoss Inc.
    • In Core, no. You have 2 permission levels, User and Manager for most things. You cannot limit Users to specific devices or groups. Everyone can see everything, though Users cannot restart daemons for instance.
    • Furthermore - in Core, if you feel like coding, you can put in or change permissions to some extent. See:http://www.skills-1st.co.uk/papers/jane/users_events_zproperties_paper.pdf. Note, this is a very limited and specific example, not a step by step guide.

What do the existing roles (ZenUser, ZenManager, Manager) mean?

  • ZenUser: A login has read only access. Cannot write/edit and the 'management' menu (on the bottom left side of the Zenoss dashboard) is not visible for this user.
  • ZenManager: Can do everything in the Zenoss Application, even switch roles. Can access the 'management' menu (on the bottom left side of the Zenoss dashboard).
  • Manager: Same as ZenManager, plus can manage the Zope (web application) server.

What does availability in the Status tab mean?

This is the percentage of time that this machine is responding to monitoring, specifically ping.

How do I use the Network Map / The Network Map doesn't work?

  1. Make sure you have flash installed in your browser.
  2. Read the Admin Guide(as of 2.4.5).
  3. Make sure you're starting at the root of one of your networks - like 192.168.0.1 ...  
    • If you've got multiple networks, you'll either need to create routes in Zenoss between them, or just start manually at each "root" for the networks.
  4. Change # of hops to 1 or higher, hit refresh.
  5. Is Zenoss discovering the routes to your devices? Zenoss needs the route topology to generate the map.

Is there a way to compare multiple performance graphs across several machines?

Yes. See MultiGraph Reports.

Zenoss Configuration

How does one change the ping/polling interval for a specific device class?

  • You cannot for ping, SNMP, WMI, Process or Modeler. Intervals are set for all devices under the Collectors->localhost->Edit.  
    • You could have multiple collectors with different cycle times.
    • You may, via distributed collectors have different collectors for different devices, and set different ping times there.
  • For Command datasources change zCommandCycleTime

How does one change the log severity level for a Zenoss daemon such as Zenactions?

You can edit the configuration files for the daemons in $ZENHOME/etc or run the daemons from the commandline with the '-v10' flag to get additional debugging.

  • An easy way to do this from the GUI would be to go to Settings in the left hand menu, Daemons in the top tabs, and select "edit config" next to the daemon you want to configure. Then enter
    logseverity  10

How do I only run selected Daemons? (i.e. I don't want to monitor Windows devices, so want to disable ZenWin etc)

See this blog post.

How do I configure Zenoss to use LDAP role mapping?

A Zenoss Master has provided a post with information on how to do this.

 

How do I change the URL in alert e-mails?

Go to Settings -> Daemons and click edit config next to zenactions. Enter

 

zopeurl http(s)://my.zenoss.server

 

Remember to make the url correct for your environment.

Tuning Zenoss

I'm getting lots of false alerts!

This is often a tuning issue, and there are several methods to make Zenoss less sensitive. Depending on the particulars, here are some suggestions:

First, always consider the count filter in your Alerting rules.

Count > 1

is a good idea, as you probably don't want to be alerted to transient events. The event should still show up in the GUI, but you don't get an e-mail until it's present for 2 polling cycles (usually that's longer than a minute).

I'm getting lots of PING alerts!

It's not a bad idea to consider how sensitive you want the ping collector to be. The default is:

  • Ping Timeout 1.5 seconds
  • Ping Retries 2

This means that if your network or device fails to respond to a ping in 1.5 seconds three times, an event gets created. Depending on all sorts of factors, you might want to increase this.

NOTE: Zenoss uses python libraries to generate pings, so these are not the same packets as you'll get with the ping utility. Use Wireshark to take a look if you like. Some firewalls end up blocking Zenoss pings, so if you get every ping blocked, look at your firewall configuration.

I'm getting SNMP alerts

Generally SNMP down alerts, when the device isn't down is a misconfiguration somewhere.

  1. First check your devices SNMP configuration, make sure the Zenoss server can access it.
  2. Then consider if the timeout is too low, some devices can take a while to reply. See zSnmpTries and zSnmpTimeout for that device or class.
  3. Also, consider some devices don't like too many queries in one request. Try lowering zMaxOIDPerRequest for that device or class.

I'm getting Timeout Connecting to Zenhub, what should I do?

You may be overloading your Zenhub daemon. This may be because your server isn't powerful enough, or may be because you've not increased the number of Zenhub Workers. By default, Zenhub is a single process/daemon. If your server is single core, you need a more powerful server.

How many ZenHub Workers should I have?

Thanks to Chet Luther, we have this information: The default configuration is to not use workers at all. What you   should be on the lookout for in this configuration for that would tip   you off to enable workers would be regular "Timeout connecting to   zenhub: is it running?" messages when you attempt to remodel devices   or in other areas.

So now.. now many workers? There's no right answer for every possible   scenario, but there are some good rules of thumb. The first would be   that you never want to exceed the total number of CPU cores minus one.   So on a 4 core box, you wouldn't want more than 3 workers. This is to   keep the web interface responsive.

Maybe more importantly, you could start by enabling 2 workers. Then   you can keep an eye on the size of your zenhub queue. This isn't one   of the standard collector metrics right now, but you can enabled it by   following the following steps.

  1. Go to http://yourzenoss:8080/zport/dmd/Monitors/perfConfig
  2. Click into the PerformanceConf template
  3. Add a new data source. The name is zenhub and the type is Built-In
  4. Add a data point to this zenhub data source called workListLength
  5. Add a new graph to the PerformanceConf template
  6. Add the zenhub_workListLength to this graph.

Now when you click on the Performance tab of your collector you will   be able to see what the size of the zenhub queue has been over time.   You should never see it go over 50, and it should stay below 10 the   vast majority of the time. If this is not the case, you can increase   the number of zenhub workers up to the total cores minus one ceiling   mentioned above.

If you get to this point and your zenhub queue is still too high, your   server likely needs more horsepower.

 

Linux

What is the setting to enable collection of software info from Linux Servers?

This varies from distro to distro.  Here is a recent blog post for Ubuntu and Debian: http://blog.zenoss.com/2009/02/18/tip-of-the-month-snmp-software-inventory-for-debian-and-ubuntu-machines/

Can I monitor a specific process running on a Linux server from Zenoss?

  • Yes.  Processes can be monitored via SNMP.
  • Zenoss will use the same methods for monitoring Windows processes.

Why does Zenoss and df locally show different available disk space?

This is because SNMP doesn't take into consideration the reserved disk space and df does, and Zenoss by default uses SNMP to get this information. In 2.4.x, use zFileSystemSizeOffset to set what percentage change you need. See this wiki page for exhaustive discussion.

Event Management

Does Zenoss support dependencies?

Yes and no.

  • Zenoss has Layer 3 dependencies, if you've fully discovered the network route / map to the devices. Zenoss keeps an internal topology map of the network, so when a router is no longer reachable the devices that connect to it are marked as unavailable as well, suppressing their events based on the more severe event of the router being down.
  • Zenoss doesn't currently have built in support for layer 2 or arbitrary dependencies, though you can use Event Transforms to cludge in arbitrary dependencies.

Why is my Zenoss generating events for devices that show Status Down from a Ping check?

Some Zenoss daemons are not yet topology aware. There are tickets open about this.

How do I stop Zenoss from generating these events when the device is down?

See the tip of the month:

Why am I getting 25 alerts on a router being down if Zenoss has dependencies?

  • Some daemons are not currently checking if a device is down. ZenCommand? in the likely culprit. There are current tickets opened against this issue.
  • You may have a problem in building the topology in Zenoss. See this forum thread for help: http://forums.zenoss.com/viewtopic.php?t=6761

What about switch failures? I'm getting 25 alerts on one switch going down!

  • Layer 2 is not currently supported for dependencies.
  • You could create arbitrary dependencies.

How do I create arbitrary dependencies in Zenoss?

  • You're going to use Event Transforms to create these arbitrary dependencies.
  • netdata has a nice wiki page on this here.

Can events from a specific device class be auto-downgraded, e.g. Critical Events --> Warnings?

  • Event Transforms will be your friend. You could add an Event Transform that checks if the device is in a System or Group and downgrade the event accordingly. For instance:
    if evt.device == "mycomp":
        severity = evt.severity - 1
        evt.severity = severity

What re-mapping options are there for events in Event Transforms?

There are all sorts of things you can do with Event Transforms. Some of those are to move an event, or otherwise change it's severity, put it into the history or just drop it.

#        evt.severity = 2          # set the event severity to Info
                                   # 1 = Debug; 2 = Info; 3 = Warning; 4 = Error, 5 = Critical
#        evt.eventState = 2      # suppress the event
                                   # 0 = New, 1 = Acknologed 2 = Suppressed
#        evt._action = 'history'  # send the event to history
#        evt._action = 'drop'     # drop the event

Note these are python code, so the # is a comment... remove it from the beginning of the line to use. Usually this will be the end of a more complex expression where you test for something before changing the event severity etc...

Can you give me some examples of Event Transforms?

Yes. There is a wiki page of useful Event Transforms here. It is occasionally updated with interesting ones from the forums.

How do I suppress or prevent CLEAR messages for certain events?

Zenoss has an auto-clear function. To turn this off, you'll use the following event transform in the event class:

if evt.severity == 0: evt._action = "drop"

How do I stop automatic Event Deduplication?

Zenoss will automatically try and deduplicate events and instead just increment a count variable. If some of your events are getting deduplicated and you would prefer that they create new events for each instance, you can create an event mapping for those events, and then in that Transform use this code:

mydedupfix1 = getattr(evt,'triggerLastOccur')
mydedupfix2 = getattr(evt,'triggerEventName')
evt.eventKey = '%s - %s' % (mydedupfix1, mydedupfix2)

This will use the time-eventname to ensure uniqueness, and because the Event Key is part of the dedupid, will stop the automatic deduplication for these events.

MIBs

How do I load MIBs?

  • This is a somewhat tricky process that is best handled on the command line. See this step by step guide.

How does Zenoss use MIBs?

  • Zenoss uses MIBs for decoding SNMP Traps.

Why is Zenoss not monitoring my device "stuff"? I've loaded the MIBs.

Zenoss does not use MIBs for monitoring/polling. Instead it uses internal Templates that you design to poll specific OIDs. For "Plug-In" monitoring of devices, see ZenPacks

.

Templates

Where are the thresholds for server parameters like Windows, Linux (free memory, CPU utilization,etc) modified?

  • You can modify these on the device templates under 'Thresholds' or make local copies and modify those.
  • Best practice would be to make a subclass to make the local copy, and modify those in the new subclass so you can
    1. Revert to the defaults if you mess it up.
    2. Easily apply your changes to multiple devices.
    3. Test your changes while leaving some functionality still running under the old templates.

Is there a way to change tresholds, OIDs, templates, for just one device?

  • You can make a local template (or copy) for just one device, however best practices would be to create a new device class under the current one you want to change, in case you want to apply the change to more than one device later on. Devices in the class will use those settings.

Can I have per filesystem thresholds?

  • No. Not as thresholds.
  • But you can work around this problem with event transforms. See this.

How do I change the Graph intervals from 5 minutes to something else?

  • First, realise that Zenoss uses RRDTool. So basically that is the underlying software that creates the graphs. Any complex "stuff" will be better described and documented at the RRDTool site.
  • Also, realise that you can't change the graphs for data that is already collected. It's like uncompressing a jpeg - you can't get back data that wasn't saved. All the perf data is saved in rrds.
  • RRDTool sets up a database of fixed size, so over time it will prune out data to less resolution. By default, it uses 5 minute steps, and over time averages them out over longer steps. Quoting from this thread:
The default Round-Robin Archive settings in Zenoss are based around 600 
pixel graphs showing 5 minute averages over approximately 30 hours
(hourly graph), 20 minute averages over approximately 10 days (daily),
etc.
The default size of rra[0].rows is 2016. (See example below)That 
means only 2016 values are recorded. At 10 seconds per value that means
that you have only 20160 seconds (slightly over 5.5 hours) recorded. If
your graph is displaying a time-frame greater than 5.5 hours, regardless
of the graph step, it will fall back to whatever step value that
provides enough values for the supplied time-frame. You either need to
increase the number of values recorded to fit the default Zenoss
time-frames, or modify the default graph display to show a smaller
time-frame. For 48 hours worth of values, you would need 17280 ( (60 sec
* 60 min *48 hours)/10 secs) values recorded in rra[0]. Which method you
should use will depend on how you intend to use Zenoss overall for support.

OK, great, so we understand a bit of the theory. What do we do now? First, we need to change the graph definition in Zenoss. Go into your Template, and look at the graph definitions. Select custom definition and create the appropriate definition per the RRDTool template. An example would look like:

rrdtool create printerX-pages-printed.rrd --step=300  
DS:pages:ABSOLUTE:600:0:U 
RRA:LAST:0.5:1:288 
RRA:MAX:0.5:1:288 
RRA:AVERAGE:0.5:288:600

In this example, we're using the every 5 minutes graph resolution. You can change the --step= to 60 for instance to get 1 minute resolution. Then you need to go under Graph Commands and create an appropriate command. Again, a matching example would be:

rrdtool graph pages.png --start=end-1day --end=now --step=300  
--height=100 
--width=500 
--vertical-label=pages/5 minutes 
DEF:mypages=printerX-pages-printed.rrd:pages:LAST 
AREA:mypages#00FF00:Pagesl 
GPRINT:mypages:LAST:cur:%0 
GPRINT:mypages:MAX:max:%0j

Again, you'd want to change the default step from 300 to 60 lets say.

Now, to effect the change you need to:

  1. zenperfsnmp stop (or Settings->Daemons->stop zenperfsnmp)
  2. Delete the existing rrd files in $ZENHOME/perf/<devicename>
  3. zenperfsnmp start
  4. Check your work. Wait until the rrd files are recreated (maybe 10 minutes) and go to one of the rrd files and run 'rrdtool info <filename>'. The step size should be toward the top.

Now, this is all great, but also remember that you have to be collecting the information you're graphing at least as often as your step interval, so check your collector poll interval as well.

This may be reasonable to get you started, but more help is available from the rrdtool site linked above, and failing that, perhaps from the Zenoss forums.

General Monitoring

How do I monitor application log files?

This depends. Generally if the logs are limited/reasonable write frequency, Syslog is a good option. Zenoss also monitors Windows Event logs automatically. Note that Zenoss cannot process high volume syslog activity as fast as syslog-ng.

  • Tip: On windows, try using epilog to convert text logs into Syslog files.

Can Zenoss monitor Web Transactions?

  • Yes, in Zenoss Enterprise this is built in.
  • In Zenoss Core, you could use twill in a custom script as a Command Datasource. You might also try webinject.

Can Zenoss monitor a device by ping only?

Yes.

  • When you use Add Device, set Discovery Protocol from auto to none. It will only do ping monitoring.
  • There is a "ping only" class that just does ping availability monitoring.
  • For an existing device, you can set the zProperty zSnmpMonitorIgnore to True.

Does Zenoss monitor the status of services running on a server?

Yes.  Windows services and IP services may be monitored in addition to process monitoring.

How do I add multiple devices with the same IP?

You can't really. There are several workarounds, depending on what you're doing.

  • If you're trying to monitor devices behind a firewall/NAT box:  
    • You could get Zenoss Enterprise and use remote collectors
    • You could configure multiple collectors in Zenoss Core.
    • You could tunnel through the firewall/NAT via VPN.

Command Based Data Sources

What are Command Based Data Sources (CBDS)?

One common way to extend Zenoss that is referenced in the forums often are CBDS. They are how Nagios Plugins are integrated for instance. They are basically a command that you run on your Linux OS that returns some performance data or success/failure. Anything you can run on a command line can be a CBDS and this allows very easy extension of Zenoss, though at a bit of a performance penalty and some security risk (If you pass a credential on a command line, the information will show up in fail events as the entire used command line is put in there.).

Nagios Plugins

Nagios Plugins are an often used source of CBDS.

How do I use Nagios plugins with Zenoss?

You use command datasources. These are an easy way to extend Zenoss, either by writing scripts that output in Nagios format, or by using actual Nagios Plugins.

Can you give me an example of using Nagios Plugins with Zenoss?

Yes.

How do I fix Check_Ping not working correctly and giving errors about Missing RRD files?

  1. If you are on CentOS, make sure the zCommandPath for each device is "/opt/zenoss/libexec/".
  2. If you are trying to Check_ping something like "Crappy_NONSNMP_Router" make sure that is the actual name of the device. In my case the device that I called "Crappy_NONSNMP_Router" was in fact called "Router_1". I found this out after actually pinging the IP address of the device. Change the name of the device in Zenoss to what the system thinks the device is called. In this case "Router_1"
  3. Go to /Devices/Ping/Router_1 ( or whatever your path is in zenoss) and make sure the NagiosCheckPing template is one of the templates bound to the device.
  4. Click on the NagiosCheckping templateunder the device, then under "Data Sources" click on "ipcheck"
  5. Click on the test button next to "test against device". You should get good data at this point. If you do get good data, then problem solved.

Why do I get "Code: 2 - Msg: Misuse of shell builtins" with some Nagios Plugins?

 

The most common cause of the "Code: 2 - Msg: Misuse of shell builtins" is that Zenoss

would have tried to use the Cacti parser for the command output.

This is triggered when the output of the command contains a colon eg.    +[zenoss@server ~]$ZENHOME/libexec/check_something -H server1

OK: Sever1 is ok checked 5:45pm+ 

There are two solutions. One would be to change the output style to replace ":" with something like "-"  You can specify this replacement within code like this:

+Output = ${Output//:/-}+

So the output will look like:

+OK- Sever1 is ok checked 5-45pm+

 

The second option would be to manually select the parser Zenoss uses in your Command Data Source.

Go into your Template and select the Datasource. Select Nagios from the dropdown rather than Auto.

 

Why doesn't my script work? I'm outputting Nagios Format to stdout?

  1. First, make sure you have the output in the format of:
    My Info Is Here|datapoint=0 datapoint2=2
  2. Even if you don't have datapoints for graphing, you do want to end with the |. Example:
    My Check is OK|

Why doesn't my script throw an event when I return "bad"?

Zenoss uses the return code of the script to determine what sort of event, if any, to raise. If you're returning data like

Failure NOT OK|

Zenoss isn't going to know what to do with that.

You want to instead have the last line of your script be

return 1

0 = "good" return. 1 = warning 2 = critical 3 = unknown

More information is available here in section 2.4.

Does Zenoss use severity set by the Nagios Plugin return?

Yes. Per Chet Luther:

You can use the 1/2 exit codes from Nagios plugins to drive the severity of the resulting Zenoss event.

The way it works it that if the plugin exits with a code of 1 (WARNING) Zenoss will set the severity of the event to whatever you selected for severity when you created the data source. If the plugin exits with a code of 2 (CRITICAL) Zenoss will use the next highest severity. An exit code of 3 (unknown) will give the same severity as the data source.

So if you choose Warning as the severity in the data source, and the plugin exits with a code of 2, the event will actually have a severity of Error.

Why doesn't Zenoss seem to poll the status of network interfaces?

Zenoss expects SNMP traps to be sent from most routers, switches, etc on an interface up/down.

My device doesn't send traps, can Zenoss poll for the status of the interfaces?

Yes. Chet Luther has written up a guite here.

How do I use "Heartbeat" style monitoring?

See here.

How can I monitor Postfix with Zenoss?

One way is to extend net-snmp to get the information you need into Zenoss. A HOWTO from Josh Baird is here.

I can run my custom command via the command line, why doesn't it work in Zenoss?

If you've created a Command Datasource or Event Command that tests fine manually from the command line, but Zenoss just won't run it:

  1. Remember to test as the Zenoss User.
  2. Check permissions for the Zenoss User.

Remember, it's best to do any development as the zenoss user.

How can I alert if a process is running?

This is an interesting reversal of normal alerting. In this case, instead of making sure a process is running, you want to ensure a process is *not* running. This can be done, but reqires some work.

  1. Create a global process with the appropriate regex to match this process you don't want to be running.
  2. Set its fail severity to clear so that it won't generate an event when it is found to not be running.
  3. Go to the device(s) that you want this functionality on and add the process on the OS tab.
  4. Lock this process from update and deletion so the modeler doesn't remove it.
  5. Click into the process and its template tab.
  6. Create a local copy of the OSProcess template.
  7. Set a maximum threshold of 0 on the count data point.

Thanks to Chet Luther!

Discovery

Does Zenoss support Autodiscovery?

Yes. Discovery works by entering the network or networks you wish to monitor and having Zenoss 'discover' your devices.  It will ping the networks and attempt to monitor the machines with SNMP, WMI, or SSH. There is a setup wizard to make the process even easier.

Does ZenOSS have an auto-discovery of devices based on IP ranges?

You can specify a subnet range.

Extending Zenoss

Zenpacks

Why do I get ERROR:zen.ZenPackCmd:zenpack stopped: <zenpack file> does not appear to be a valid file or directory?

  • Try unzipping the zenpack so that it ends in .egg and uploading that.
  • It looks like your browser changed the file from a .egg to a .zip. Rename it to <zenpack file>.egg and try to install it.

I am getting [Errno 20] trying to install ZenPacks

  • Try unzipping the zenpack so that it ends in .egg and uploading that.
  • It looks like your browser changed the file from a .egg to a .zip. Rename it to <zenpack file>.egg and try to install it.

Why can't I install the Core Zenpacks RPM?

First, the best practice for Zenpacks is:

  • Only install the ones you need. Zenpacks can conflict.
  • Use the Zenpack management system in Zenoss rather than the system package management system.

Per Matt Ray: The Core ZenPacks RPM has the RHEL RPM as a requirement, so you can't   install it against the .bin installer.

Distributed Collectors

Can I have multiple Zenoss collectors behind firewalls that report to a central Zenoss server?

Yes.

  • The easy way is to purchase Zenoss Enterprise and use their distributed setup.
  • The harder way is to manually set up core to do so. There are some community how-tos:
    1. docs/DOC-2496
    2. docs/DOC-2497

How do I distribute the load across multiple servers for monitoring many devices?

See Distributed Collectors

Updates / Migration

Can I migrate from a VMWare Appliance to a Zenoss Core native install?

Yes. You want to go from the same version of Zenoss VMWare to the same version native install.

  1. Install the native install however you like, source, stack, RPM...
  2. Install the same Zenpacks you had on the VMWare Appliance to the native install.
  3. Do a ZenBackup on the VMWare Appliance.
  4. Copy the resulting backup to your native install.
  5. Use ZenRestore to put the data into the native install.

At this point, you should be where you started, but on the native install.

NOTE: If you want to migrate from a 32 bit to 64 bit platform, you will need to migrate your RRD files.

Can I migrate without copying over everything?

Sort of. If you don't want to copy over your events or historical performance data, you can:

  1. Install the same version of Zenoss on the new server.
  2. Copy over $ZENHOME/var/data.fs
  3. Wait 15 - 30 minutes for the new rrd files to be created and off you go.

Technical Troubleshooting

I'm getting ERROR Zope Couldn't install Five in zopectl log. What do I do?

This is an error that crops up every so often when there's a problem with an upgrade to Zenoss, or working with Zenpacks.

  1. Try running dmd.Devices.reIndex() in zendmd.
  2. See if multiple versions of zope.interface.adapter are in the python search path. Look for /usr/local/zenoss/python/lib/python2.4/site-packages/zope.interface-3.3.0-py2.4-linux-i686.egg/  
    • Move the directory out of the python search path and see if that fixes it.
    • Don't forget to clear browser cache.

Note there is a defect open: http://dev.zenoss.org/trac/ticket/4548

How do I stop Heartbeat errors?

  1. First try clearing all your heartbeats.
    1. Click on Event Manager
    2. Use the dropdown to select Clear all Heartbeats
  2. If you've changed your Zenoss server hostname you may need to do the following: (Replace localhost with the old hostname, use appropriate credentials)
    1. Log into mysql: zenoss$ mysql -uzenoss -pzenoss
    2. Select the events database: mysql> use events
    3. Have a look at the heartbeat table: mysql> select * from heartbeat;
    4. Manually remove any heartbeats seen above that are giving you trouble: mysql> delete from heartbeat where device='localhost';
    5. Check the Event Console and see if they are gone.

My Cisco devices seem to work, but they show SNMP as being down. Why?

Cisco ASA, FWSM and Pix devices are very sensitive to the size of SNMP   requests they're sent. Usually the Zenoss default of packing up to 40   requests into a single packet overflows their buffer.

You can probably fix this by setting the zMaxOIDPerRequest property to   10 for the device class the ASA devices are in.

Thanks to Chet Luther and the forums...

How do I fix WARNING zen.zenperfsnmp: Error loading config for devices in my zenperfsnmp.log?

There are several possible fixes.

  1. Try rebuilding your indexes:
    $ZENHOME/bin/zendmd
       >>> reindex()
       >>> commit()
  2. Try removing the problem devices, and re-adding them.

How do I get the Dashboard back for a particular user?

  1. First, try clearing your browser cache and re-loading. This fixes many display problems, especially right after a Zenoss upgrade.

If that doesn't work then:

  1. Connect to your zenoss server using the URL http://yourserver:8080/zport/dmd/manage
  2. Open ZenUsers on the left-side menu and select the account.
  3. Select the properties tab at the top of the page.
  4. Delete the 'dashboardstate' string.
  5. Save the changes and then load up your Zenoss web page as normal.

My Zenpack / something doesn't work and shows path errors…

$ZENHOME has changed during the development of Zenoss. Depending on when a Zenpack was made, or other custom script, it may reference a hardcoded path rather than $ZENHOME... See if it's pointing towards /opt/zenoss rather than /usr/local/zenoss somewhere in the scripts. I'd replace that part of paths with $ZENHOME, but also check the full path in case something else changed as well.

Devices are Missing / I've moved my devices between classes, and no graphs are working!

This is often caused by the index getting confused. Try reindexing:

$ZENHOME/bin/zendmd
   >>> reindex()
   >>> commit()

I'm unable to delete a device or change its device class!

This is often caused by a broken Layer 3 catalog. Try this.

Upgrading to 2.4.x caused my command based datasources (ZenCommand) to not work, what do I do?

  • One thing to check is to see if zSshConcurrentSessions has a null value. If so, ZenCommand won't work, try setting it to some value. 10 is reported to work fine.

I'm getting DEBUG:zen.SnmpClient:sending queries for plugin zenoss.snmp.RouteMap; WARNING:zen.ZenModeler:client XXXX.XXXXXX.XXXX.XXX timeout

When this happens ZenModeler is timing out after 180 seconds and not populating any info for the device or the interfaces on it. This is caused by too many routes on the device and the plugin  times out; you could try and manually walk the routing table (ipRouteTable / somewhere at 1.3.6.1.2.1.4.21) and see how many it's returning.

 

See this thread for more information.

I've updated to 2.4.2, and now I get BadRequest: The property {zSnmpTries,zLinks,...} does not exist!

See this ticket.

Potential fix:

zendmd 
>>> a=find("rubis.ptx.fr.sopra")
>>> a.hasProperty("zSnmpTries")
0  [b]<- this mean the property doesn't exist[/b]
>>> a.zSnmpTries
??? [b]<- if you get something here, it means it's an attribute and not a propery[/b]
>>> a.__delattr__("zSnmpTries")
>>> commit()

I can't get non-latin characters to display in Events

This is because Zenoss defaults to iso-8859-1 character set. You can hack around this a bit to UTF-8 as shown in this thread.

Why do I have duplicate subnetworks? Why is Zenoss treating my network mask as /24 when it's not?

This seems to be a bit of a bug in how Zenoss is treating network topology. A bunch of useful information to clear this up is in three forum threads. See this post and the threads it links to for some workarounds and or methods to fix this.

How can I debug error loading config for ZenPerfSnmp?

This can be caused by a broken transform for a device. To debug run the following code within zendmd. This will execute the same code that zenperfsnmp does to build its configuration for a device, and throw out any errors it encounters during the process.

from Products.ZenHub.services.SnmpPerfConfig import getDeviceConfig
cfg = getDeviceConfig(find("PROBLEM_DEVICE_NAME"))

If the configuration is successfully built without errors you should just get another >>> prompt. If errors are encountered, the associated traceback will be printed. This traceback should contain useful   information for determining where the problem is.

How can I log in to Zenoss using a URL string?

There are two URL formats that should work, though it might depend on what sort of code environment you're working in.

  1. http://zenuser:zenpassword@zenossserverip:8080/zport/dmd/Devices/deviceList
  2. http://zenossserverip:8080/zport/dmd/Device/deviceList?__ac_name=zenuser&__ac_password=zenpassword

I've upgraded to 2.4.x and am getting RealMemory events for my Linux devices!

This seems to be because in 2.4, Zenoss started monitoring more block device types in Linux for use, in the same way it has been monitoring filesystem use. Of course, this doesn't really make sense for RAM, because of the way Linux will manage the memory, filling empty RAM with filesystem cache... To fix this:

  1. Go to /Devices/Server/Linux
  2. Select zProperties
  3. Add
    ram
    to zFileSystemMapIgnoreTypes
  4. At the CLI, as the zenoss user, run
    zenmodeler run --force --path=/Devices/Server/Linux --collect=HRFileSystemMap 

  5. Move existing RealMemory events to History.

This of course leaves you without warnings on memory use, but if you're using one of the Linux Zenpacks, you should be able to use Thresholds on the real memory used data point to achieve what you actually want to be alerted on. There are other potential entries for zFileSystemMapIgnoreTypes, if they start to generate events for you, you can add

other 
virtualMemory
removableDisk
floppyDisk
compactDisk
ramDisk
flashMemory
networkDisk

I've installed Zenoss 2.4.x, and all my graph legends are boxes!

This is a known issue. It occurs on Ubuntu, FreeBSD, and RedHat. There is a new dependency.

  • Install the liberation fonts.
  • For RedHat, CENTOS:
    # yum -y install liberation-fonts
  • For FreeBSD:  
  • For Ubuntu:  
    • Make sure you have installed the fonts
      ttf-liberation 
      ttf-linux-libertine
      libgcj-common
    • If that doesn't work, try this:
    •  $ rm -rf /usr/local/zenoss/var/cache/fontconfig 
      $ /usr/local/zenoss/bin/fc-cache -f -v /usr/share/fonts/truetype
      $ zenoss restart
    • In your browser refresh the page (Press F5)
    •  $ /usr/local/zenoss/bin/pango-querymodules > /usr/local/zenoss/pango/pango.modules
      $ zenoss restart
    • In your browser refresh the page (Press F5)

I've installed Zenoss 2.4.5 and my Command Based Datasources do not increment event counts!

Run

zenpatch 15003

Comments (3)