Jun 11, 2010 3:05 AM
question regarding: Products/ZenRRD/RRDUtil.py
-
Like (0)
Hello,
I'm trying to get rrdcached working with Zenoss (2.5.2). Until now I focused on editing the files in Products/{ZenRRD,ZenModel,ZenUtils} which contain anything related to rrd commands. Somehow my changes were working but rddcached seems not to buffer the updates at all. Then I noticed that in RDDUtil.py at the end of the save() function rrdtool.fetch() is called which triggers a FLUSH (in rrdcached) and writes the updates to the file immediately.
Now my question: what' the purpose of this last rrdtool.fetch and is it necessary because I commented the lines around this rrdfetch and its seems to be working anyhow?
PS: I'd like to try this with the latest beta too but there are no tarballs - will there be tarballs for the beta releases?
Now I extended the latest Zenoss beta to make use of rrdcached. It seems to work just fine. When do you guys plan to tackle this? I ask this because, except for an IRC log, I'm not able to find any ticket/thread/blog entry whatsoever even mentioning rrdcached.
Because of all the other changes going into the 3.0 release and that the rrdcached being brand-new when we started our release cycle, we decided to hold off on this upgrade for the 3.0 release. We will most definitely investigate it for the next release, since it should greatly improve RRD performance.
What have you found with using it? Do you have patches for this you'd like to submit? Like I said, it's definitely something we'll investigate for the next release.
Thanks,
Matt Ray
Zenoss Community Manager
As I said it seems to work just fine. Although I still have to test this with a few more devices/rrds. For this I have to wait for some hardware with which I can test this in our network environment. I'm looking forward to do this within the next two weeks. I think then I'll be able to post some results.
I'll post some patches too. Until now they only extend the rrdtool commands (update, fetch, graph and info) with the --daemon option. I'm still wondering about the code I was referring to in the initial post of this thread.
I share an office with the Product Manager, so I've been bugging him about it. I could have sworn there was an enhancement around it somewhere, but it's definitely on our radar.
-Matt
Dear Massimo,
Could you please instruct us what to do in order to modify Zenoss to correctly use the rrdcached daemon?
I'm using Zenoss 3.0 and I'd like to upgrade the rrdtool to the 1.4.4 version and use rrdcached.
1 - Basically, I need to install rrdtool-1.4.4 and delete zenoss/common/bin/rrdtool and rrdupdate;
2 - Afterwards, it's needed to point rrdcached, but what files/ lines should I modify in Products/{ZenRRD,ZenModel,ZenUtils}?
Thank you very much,
gfranco
Hello *, unfortunately I was still not able to test this with a bigger setup. Nevertheless I want to share this with you: To make this whole thing work you need to install a rrdtool version which provides rrdcached (I picked the latest one 1.4.4). You could simply copy the newer sources to the Zenoss' source directory containing the installation files and let Zenoss' installer compile and install the newer version (worked for me) or just manually install it overwriting the already installed version in Zenoss' installation directory. Furthermore you should be clear about the options of rrdcached - I use this command to start the daemon: rrdcached -l unix:/var/run/rrdcached/rrdcached.sock -w 1800 -j /tmp/ (using the same user who starts zenoss) Using the following files you can see what modifications I did (e.g., using diff): Products/ZenRRD/RenderServer.py http://paste.pocoo.org/show/244621/ Products/ZenRRD/RRDUtil.py http://paste.pocoo.org/show/244622/ Products/ZenRRD/zenstep.py http://paste.pocoo.org/show/244624/ You could simply overwrite the files of your Zenoss installation using the files above. Nevertheless a few things are hardcoded like /var/run/rrdcached/rrdcached.sock . Of course you could adapt this to your needs. You should stop Zenoss, copy or adapt the files, start rrdcached and finally start Zenoss again. Usually rrdcached writes logs to /var/log/messages and keep an eye on Zenoss' logs as well. Note: don't stop and and restart rrdcached without restarting Zenoss - this does not work (yet). HTH, massimo
Dear Massimo,
Thank you very much for sharing this info and your time on this!
I'll try it on a test machine.
Regards,
gfranco
Unfortunately that call to fetch isn't entirely spurious. For COUNTER and DERIVE data points it is required to properly evaluate any thresholds that may be applied to them. We're relying on RRDtool to calculate the rate for us so that we're thresholding on the rate instead of the raw value.
To properly use rrdcached we'll need to have Zenoss itself remember the last value retrieved for each COUNTER and DERIVE data point so that can can calculate the rate internally without fetching it from the RRD file.
Work (almost all of it) has been done on upgrading to rrdtool 1.4.4 and using rrdcached. Barring something very odd, it'll be in 3.1 (heck, it'll probably be in trunk by next week).
Next week was an overestimate; it's in trunk now.
http://dev.zenoss.com/trac/changeset/21852
Pretty similar to Massimo's patches, except:
You can pass more options to zenrrdcached and they'll be sent along to rrdcached, so if you want to change the flushing or the timeout or whatever you can.
What about the issue Chet Luther mentioned too? Like this updates regarding DERIVE and COUNT data points would be written immediately to the corresponding rrd file.
Follow Us On Twitter »
|
Latest from the Zenoss Blog » | Community | Products | Services Resources | Customers Partners | About Us | ||
Copyright © 2005-2011 Zenoss, Inc.
|
||||||||