Archived community.zenoss.org | full text search
Skip navigation
Up to Discussions in zenoss-users
1 2 Previous Next 23347 Views 18 Replies Latest reply: Apr 25, 2011 8:50 AM by ben RSS
cdillardhsp Rank: Green Belt
cdillardhsp
127 posts since
Mar 17, 2007
Currently Being Moderated

Aug 9, 2009 10:01 PM

Graphs stopped working!

:shock:

All of a sudden we have two devices for which Zenoss has stopped creating graphs. Running zencommand -v 10 doesn't produce any errors at all related to these two devices and I can't find anything in any of the logs that points to why no graphs will render. Doing snmpwalk on both devices returns lots of OIDs as well.

We've restarted Zenoss, upgraded from 2.4.1 to 2.4.2, restarted again and we've even restarted one of the two devices but to no avail.

Can anyone offer some suggestions of where I might look and what for? We have over a year's worth of graphs on one of these hosts that I don't want to lose by deleting and recreating the device.

Thanks,
Clay
  • jbobjensen Rank: White Belt
    jbobjensen
    22 posts since
    Jan 3, 2008
    Currently Being Moderated
    1. Aug 10, 2009 12:44 AM (in response to cdillardhsp)
    Graphs stopped working!
    Did you look at zenperfsnmp.log for errors on those devices?

    On Sun, Aug 9, 2009 at 10:01 PM, cdillardhsp <claytondillard@gmail.com ([email]claytondillard@gmail.com[/email])> wrote:

     

     

    [Shocked]

    All of a sudden we have two devices for which Zenoss has stopped creating graphs.  Running zencommand -v 10 doesn't produce any errors at all related to these two devices and I can't find anything in any of the logs that points to why no graphs will render.  Doing snmpwalk on both devices returns lots of OIDs as well.

    Can anyone offer some suggestions of where I might look and what for?  We have over a year's worth of graphs on one of these hosts that I don't want to lose by deleting and recreating the device.

    Thanks,
    Clay







    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org ([email]zenoss-users@zenoss.org[/email])
    http://lists.zenoss.org/mailman/listinfo/zenoss-users

  • jmp242 ZenossMaster
    jmp242
    4,060 posts since
    Mar 7, 2007
    Currently Being Moderated
    2. Aug 10, 2009 8:18 AM (in response to jbobjensen)
    Graphs stopped working!
    You're a bit unclear - are no graphs displaying, or are you getting an
    increasingly large blank part of the graph, but if you went back 5 days
    you'd have the old data show up?
    --
    James Pulver
    Information Technology Area Supervisor
    LEPP Computer Group
    Cornell University



    cdillardhsp wrote, On 8/9/2009 10:01 PM:

     

     

    [Shocked]

    All of a sudden we have two devices for which Zenoss has stopped creating graphs. Running zencommand -v 10 doesn't produce any errors at all related to these two devices and I can't find anything in any of the logs that points to why no graphs will render. Doing snmpwalk on both devices returns lots of OIDs as well.

    Can anyone offer some suggestions of where I might look and what for? We have over a year's worth of graphs on one of these hosts that I don't want to lose by deleting and recreating the device.

    Thanks,
    Clay







    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users

    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    3. Aug 10, 2009 8:51 AM (in response to jmp242)
    RE: Graphs stopped working!
    Yes James, we have graphs but there is an increasingly large space of blank graph. We can go back in time and see graph data.
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    4. Aug 10, 2009 8:52 AM (in response to cdillardhsp)
    RE: Graphs stopped working!
    In zenperfsnmp.log we have the following error that includes multiple hosts and the device we're discussing in this thread is one of them. However, the other devices listed in this error string do have current graphs (no problems).

    2009-08-10 04:24:50 WARNING zen.zenperfsnmp: Error loading config for devices
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    5. Aug 10, 2009 4:30 PM (in response to cdillardhsp)
    RE: Graphs stopped working!
    Maybe Chet or someone from Zenoss could comment on this? I really hate to have to delete and recreate this device in Zenoss and lose over a years worth of graph data.

    Thanks,
    Clay
  • mwcotton Rank: Brown Belt
    mwcotton
    563 posts since
    Apr 23, 2008
    Currently Being Moderated
    6. Aug 10, 2009 4:54 PM (in response to cdillardhsp)
    RE: Graphs stopped working!
    did you add a new threshold lately? oddly i have read on this forum, a wacked threshold can break the graphing
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    7. Aug 11, 2009 8:59 AM (in response to mwcotton)
    RE: Graphs stopped working!
    The timing of the missing graphs data coincides with the installation of the Fping zenpack which does have thresholds. I've since removed the Fping zenpack and have restarted Zenoss but still have no new graph data. I'm not sure what the problem might be.
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    8. Aug 12, 2009 10:38 AM (in response to cdillardhsp)
    RE: Graphs stopped working!
    Odd thing is that no network interface graphs work, none of the graphs on the Perf tab/page work but there is a graph that is working for the /bin/logger process on the OS tab/page.

    If I delete this device and recreate it, can I first copy all of the RRD files to a safe location and then copy them back to the newly created device and still keep my historical graphs that way?
    [/img]
  • ccole Rank: White Belt
    ccole
    19 posts since
    Aug 18, 2008
    Currently Being Moderated
    9. Aug 12, 2009 12:48 PM (in response to cdillardhsp)
    RE: Graphs stopped working!
    I had a similar problem on the Windows side. If possible, try shutting down Zenoss and deleting the .zec files under $ZENHOME/var and then restarting.

    My (amateur's) guess is that I did something that broke Zenoss, and whatever it was was being cached and therefore survived restarts.
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    10. Aug 13, 2009 1:50 AM (in response to ccole)
    RE: Graphs stopped working!
    I ended up deleting the device that suffered from the loss of performance graphs and recreating it and after nearly an hour, and a Zenoss restart, there are still no graphs. I'm at a loss for what to do.
  • chickenandbeans Rank: White Belt
    chickenandbeans
    11 posts since
    Aug 11, 2009
    Currently Being Moderated
    11. Aug 13, 2009 5:21 AM (in response to cdillardhsp)
    RE: Graphs stopped working!

    "cdillardhsp" wrote:

     

    I ended up deleting the device that suffered from the loss of performance graphs and recreating it and after nearly an hour, and a Zenoss restart, there are still no graphs. I'm at a loss for what to do.




    Hi cdillardhsp,

    I'm having a similar problem to you. Sudden graph loss and no events on a couple of devices.
    When you deleted and recreated your device did you have any multi-graph reports on that device, and if so so did the deletion effect them in any way?
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    12. Aug 13, 2009 3:01 PM (in response to chickenandbeans)
    RE: Graphs stopped working!
    Zenoss just seems determined not to graph for this host. We've tried moving it between classes and still no graphs. We have another Zenoss server at our secondary data center and we added this device and the graphs generate fine.

    Anybody at Zenoss willing to help out with this one? I cannot find any errors in the logs that indicate what the problem might be.

    Thanks,
    Clay
  • cdillardhsp Rank: Green Belt
    cdillardhsp
    127 posts since
    Mar 17, 2007
    Currently Being Moderated
    13. Aug 13, 2009 4:14 PM (in response to cdillardhsp)
    RE: Graphs stopped working!
    Found this in the event.log:

    2009-08-13T16:10:24 ERROR RenderServer Failed to generate a graph
    Traceback (most recent call last):
    File "/opt/zenoss/Products/ZenRRD/RenderServer.py", line 190, in render
    rrdtool.graph(*gopts)
    error

    We are running:
    Zenoss Zenoss 2.4.2
    OS Linux (i686) 2.6.18 (Linux skynet 2.6.18-128.1.10.el5.xs5.5.0.51xen #1 SMP Fri May 29 06:29:19 EDT 2009 i686)
    Zope Zope 2.8.8
    Python Python 2.4.4
    Database MySQL 5.0.45 (Ver 5.0.45)
    RRD RRDtool 1.3.5
    Twisted Twisted 8.1.0
    NetSnmp NetSnmp 5.3.2
    PyNetSnmp PyNetSnmp 0.28.13
    WMI Wmi 1.2.6
  • peak6 Rank: White Belt
    peak6
    51 posts since
    Feb 25, 2010
    Currently Being Moderated
    14. Apr 15, 2010 2:37 PM (in response to cdillardhsp)
    Re: RE: Graphs stopped working!

    Bump

    same problems,

    Servers were graphing fine, One day they just stopped

    in zenperfsnmp

    WARNING zen.zenperfsnmp: Error loading config for devices

1 2 Previous Next

More Like This

  • Retrieving data ...

Incoming Links