zenperfsnmp is deleting current RRD files - Open Source Network Monitoring and Systems Management

PaulH

7 posts since
Feb 15, 2010

Currently Being Moderated

15. Feb 26, 2010 12:55 PM (in response to Ryan Matte)

Re: zenperfsnmp is deleting current RRD files

Hmmmm... This is good to know. Thank you.

My file settings are as follow and my system has ntpdate installed.

/dev/sda1 / ext3 defaults 1 1

Report Abuse
Like (0)

Ryan Matte

653 posts since
Mar 26, 2009

Currently Being Moderated

16. Feb 26, 2010 1:16 PM (in response to PaulH)

Re: zenperfsnmp is deleting current RRD files

Ok, I understand that you have ntpdate installed, but is it actively syncing the time, or only when you manually call on it?

Report Abuse
Like (0)

nikal

45 posts since
Jul 8, 2008

Currently Being Moderated

17. Feb 26, 2010 2:23 PM (in response to Ryan Matte)

Re: zenperfsnmp is deleting current RRD files

Just to confirm that I'm having the described issue on Debian Lenny, everything lvm and xfs, ntpd automatically adjusting the local clock on regular basis.
I agree with your remark about the RRD tool patching, but regardless of the reason, the problem is in place.

Report Abuse
Like (0)

Ryan Matte

653 posts since
Mar 26, 2009

Currently Being Moderated

18. Feb 26, 2010 2:23 PM (in response to nikal)

Re: zenperfsnmp is deleting current RRD files

Well, the code that causes the issue is in $ZENHOME/Products/ZenRRD/zenperfsnmp.py

maxRrdFileAge = 30 * (24*60*60)     # seconds

# remove files older than maxRrdFileAge
        self.fileCleanup = FileCleanup(perfRoot, '.*\\.rrd$',
                                       self.maxRrdFileAge,
                                       frequency=90*60)
        self.fileCleanup.process = self.cleanup
        self.fileCleanup.start()

It's been present for many versions back. I'll be opening a Trac ticket to have it removed by the time Stone Crab comes out for a few reasons:

1. RRD files don't really take up that much space, even if there are a few stale ones laying around.

2. RRD files automatically get deleted when a device is removed from Zenoss.

3. Zenoss users are perfectly capable of cleaning up old RRD files by hand if necessary.

4. Automated data loss is not cool

Before I open the Trac ticket, I'm just trying to piece together what could be causing the timestamps not to get updated.

Report Abuse
Like (0)

PaulH

7 posts since
Feb 15, 2010

Currently Being Moderated

19. Feb 26, 2010 2:42 PM (in response to Ryan Matte)

Re: zenperfsnmp is deleting current RRD files

My understanding is that ntpdate peridically polls. I really never gave much thought to ntp and am not very familiar with it. All one has to do is touch the files and they are set to the current date/time. I can clearly see that the rrd files are getting updated internally with new data, but the timestamps of these files do not reflect that. If I "touch" one of them, the timestamp changes to the current date/time. If this does not anwer the question, please provide me with more context and guidance.

Thanks for opening a ticket. Automated data loss is definitely not

Report Abuse
Like (0)

Ryan Matte

653 posts since
Mar 26, 2009

Currently Being Moderated

20. Feb 26, 2010 5:28 PM (in response to PaulH)

Re: zenperfsnmp is deleting current RRD files

Well, ntpdate is a commandline utility for manually updating the time via ntp but there's also ntpd (a daemon) which needs to be installed and configured to automatically sync the time at intervals.

Report Abuse
Like (0)

PaulH

7 posts since
Feb 15, 2010

Currently Being Moderated

21. Feb 26, 2010 5:37 PM (in response to Ryan Matte)

Re: zenperfsnmp is deleting current RRD files

ntpd is running.

And I did move the perf folder to an XFS partition, and the problem remains. The timestamps on the RRD files do not get updated.

Report Abuse
Like (0)

RyanK

5 posts since
Dec 16, 2009

Currently Being Moderated

22. Feb 26, 2010 5:49 PM (in response to PaulH)

Re: zenperfsnmp is deleting current RRD files

I'm not sure what makes you think that ntpd might be related to.

It does adjust the system time as well as adjust the drift to attempt to keep the clock running at the proper rate, but I'm not sure what that would do to prevent an RRD file from updating its modification time stamp.

I think that the issue lies within kernel mmap'ed files and perhaps filesystems.

Regardless, I think removal of the RRD cleanup code is a great solution. I looked at filing a trac issue back several months ago but wasn't sure how to file it and went for a forum post thinking it was perhaps more of a local issue.

Thanks all for the help!

Report Abuse
Like (0)

PaulH

7 posts since
Feb 15, 2010

Currently Being Moderated

23. Feb 26, 2010 7:22 PM (in response to RyanK)

Re: zenperfsnmp is deleting current RRD files

I don't think ntp has anything to do with it. That is why I was trying to emphasize that "touching" a file brings it to its current date, but I guess I was not clear enough. I know very little about ntp, but I figured if when you "touch" the file and it is timestamped with the current date, then something like ntp is not likely to be the issue unless there is some very strange dependency somewhere. Nevertheless I was still trying to respond to Ryan Matte's question about whether ntp was running on my system.

In a perfect world, the clean up script was a good idea, but unfortunately because other lower level mechanisms aren't doing their job, it will be great if the clean up code is disabled.

Report Abuse
Like (0)

Ryan Matte

653 posts since
Mar 26, 2009

Currently Being Moderated

24. Feb 27, 2010 1:27 PM (in response to PaulH)

Re: zenperfsnmp is deleting current RRD files

In all probability it doesn't, I'm just trying to rule out obvious things related to system time.

Report Abuse
Like (0)

phonegi

446 posts since
Apr 15, 2009

Currently Being Moderated

25. Mar 1, 2010 2:46 PM (in response to Ryan Matte)

Re: zenperfsnmp is deleting current RRD files

Just FYI, CentOS 5.4 does record the datestamp on RRD files. Wonder what the difference is between OSs?

Report Abuse
Like (0)

WebGuyInternet

19 posts since
Jan 16, 2009

Currently Being Moderated

26. Apr 6, 2010 1:38 PM (in response to phonegi)

Re: zenperfsnmp is deleting current RRD files

I too am having a simillar issue, I implemented the "touch" crontab mentioned earlier and am no longer losing graph data (lost two years worth of data). I am running RedHat 5.4 ES and Zenoss Core 2.5. It doesn't seem to be an isolated problem.

Report Abuse
Like (0)

Actions

More Like This

Legend