Q for devs - Broken functionality or incorrect forum post?? - Open Source Network Monitoring and Systems Management

Up to Discussions in zenoss-dev

61378 Views 12 Replies Latest reply: Oct 7, 2011 3:42 PM by Brian Seklecki

HaydnH

71 posts since
May 30, 2008

Currently Being Moderated

Jun 17, 2008 10:09 AM

Q for devs - Broken functionality or incorrect forum post??

Hi Dev's,

I've been looking in to creating performance graphs and alert thresholds for various objects through snmp and I've come across this:

Taking the diskIOReads OID (1.3.6.1.4.1.2021.13.15.1.1.5) as an example, we have various values under that OID for the various disks (e.g: .1.3.6.1.4.1.2021.13.15.1.1.5.17 = IORead count for /dev/sda, 1.3.6.1.4.1.2021.13.15.1.1.5.19 = IORead count for /dev/sdb).

Lets say we want a graph & threshold for the TOTAL diskIOReads for all disks, is it meant to be possible to create the datasource for the graph using the OID (1.3.6.1.4.1.2021.13.15.1.1.5) for this purpose?

I have found a forum post from cluther who certainly appears to think this is possible, quoting post 2 from http://community.zenoss.com/forums/viewtopic.php?t=3647 :

"Create a HardDisk template under /Devices/Server/Linux. You can use the diskIOReads, diskIOWrites, diskIONRead, diskIONWritten OIDs in the template. Leave off the diskIOIndex from the end of the OID as Zenoss will automatically append the modeled index as it does with other SNMP table data."

This doesn't appear to work anymore - has this been broken, changed, or was the forum post incorrect?

Thinking about it, I think that method would be wrong anyway as /dev/sda is listed in the snmpwalk as well as /dev/sda1,2 etc so we'd get the same IO shown multiple times for each parition. Perhaps there is a better way of doing this now anyway? e.g: creating a datasource for /dev/sda & /dev/sdb on the same graph, and a third data point for /dev/sda + /dev/sdb?

Like (0)

beanfield
161 posts since
Apr 16, 2008

Currently Being Moderated

1. Jun 17, 2008 10:47 AM (in response to HaydnH)
Re: Q for devs - Broken functionality or incorrect forum pos

"HaydnH" wrote:

Thinking about it, I think that method would be wrong anyway as /dev/sda is listed in the snmpwalk as well as /dev/sda1,2 etc so we'd get the same IO shown multiple times for each parition. Perhaps there is a better way of doing this now anyway? e.g: creating a datasource for /dev/sda & /dev/sdb on the same graph, and a third data point for /dev/sda + /dev/sdb?

If it is truly using the template to model the device, then I would think what cluther says is correct. The modeling process queries snmp for an index list of all the partitions. Once it gets that index, it does a series of snmpget's instead of an snmpwalk. So for the first index .1, it would do a snmpget on 1.3.6.1.4.1.2021.13.15.1.1.5.1 to retrieve the IORead count for that volume...and then another snmpget on the next oid you specified (i.e. the oid for IOWrite...so it would next query 1.3.6.1.4.1.2021.13.15.1.1.6.1). Once it has gotten each oid for that one volume (index number), it would incrememnt the index number on the end of the OID's you specified.

So you would setup your template, and either re-add the device or remodel it (go to teh device, click on the down arrow at the upper left, click on Manage, click on Model Device). I would think it would populate the each volume appropriately....that's IF I understand it correctly. happy

Report Abuse

Like (0)
HaydnH
71 posts since
May 30, 2008

Currently Being Moderated

2. Jun 18, 2008 2:24 AM (in response to beanfield)
Re: Q for devs - Broken functionality or incorrect forum pos

Hi Beanfield,

Yes, you have understood me correctly - from what you have said I believe this functionality is broken. I can only put in a data source for a specific partition, any OID that has children fails, Using `zenperfsnmp run` to debug:

1.3.6.1.4.1.2021.13.15.1.1.5.18:
Works fine and gives diskIOReads values for /dev/sda1 in this case

1.3.6.1.4.1.2021.13.15.1.1.5:
Prints a "oid 1.3.6.1.4.1.2021.13.15.1.1.5 is bad" message from `zenperfsnmp run`

If we can confirm that Zenoss should (used to?) accept SNMP tables as data sources (instead of single value OIDs) and someone else can confirm that this is broken, let me know and I'll raise a ticket.

Report Abuse

Like (0)
beanfield
161 posts since
Apr 16, 2008

Currently Being Moderated

3. Jun 18, 2008 2:06 PM (in response to HaydnH)
Re: Q for devs - Broken functionality or incorrect forum pos

"HaydnH" wrote:

Hi Beanfield,

Yes, you have understood me correctly - from what you have said I believe this functionality is broken. I can only put in a data source for a specific partition, any OID that has children fails, Using `zenperfsnmp run` to debug:

1.3.6.1.4.1.2021.13.15.1.1.5.18:
Works fine and gives diskIOReads values for /dev/sda1 in this case

1.3.6.1.4.1.2021.13.15.1.1.5:
Prints a "oid 1.3.6.1.4.1.2021.13.15.1.1.5 is bad" message from `zenperfsnmp run`

If we can confirm that Zenoss should (used to?) accept SNMP tables as data sources (instead of single value OIDs) and someone else can confirm that this is broken, let me know and I'll raise a ticket.

Ok, I wasn't sure how you were doing the " UCDHardDiskMap snmp data collector"...but I did some digging and got it working. At first I thought it was a custom modeler you wrote in python. But I eventually found it.

I went to "Devices", "Servers", "Linux", clicked on the top left drop down, clicked on "More", and then clicked on "Collector Plugins". I clicked on "Add Fields" and drug "zenoss.snmp.UCDHardDiskMap" to left/bottom so it was last in the left column. Then I clicked "Save".

Next I create the HardDisk template. I browse to "Devices", "Servers", "Linux" and click on the "Templates" tab. I click the second from the top drop down on the left and choose "Add Template". I name it "HardDisk" (I assume the exact name is important). I create 4 data sources:

diskIONRead 1.3.6.1.4.1.2021.13.15.1.1.3
diskIONWritten 1.3.6.1.4.1.2021.13.15.1.1.4
diskIOReads 1.3.6.1.4.1.2021.13.15.1.1.5
diskIOWrites 1.3.6.1.4.1.2021.13.15.1.1.6

All of them have a DataPoint with the same name as the Data Source. Each of them are type "Counter".

I create two Graph Points:

IO
Access

Under IO change units to "Bytes" (not necessary). For graph points add Data Sources diskIONRead and diskIONWritten. You can change the legend under each of these to "Reads" and "Writes" (respectively) if you like.

Under Access change units to "Accesses" (not necessary). For graph points add Data Sources diskIOReads and diskIOWrites. You can change the legend under each of these to "Reads" and "Writes" (respectively) if you like.

Save everything. Then I restarted zenperfsnmp "zenperfsnmp restart". Then go to each device and remodel....or use the "zenmodel" command on each device.

You should start seeing graphs under each volume that actually has any I/O....this will be displayed in the "Hardware" tab of each server.

It does appear to use the diskIOIndex 1.3.6.1.4.1.2021.13.15.1.1.1 oid for each volume.

Long story short...it's working for me...mapping out each volume automatically and querying the correct OID.[INDEX_NUM]. I'd double check that the HardDisk template is named correctly...I'm pretty sure that the collector is looking for that EXACT template name.

Report Abuse

Like (0)
HaydnH
71 posts since
May 30, 2008

Currently Being Moderated

4. Jun 19, 2008 3:18 AM (in response to beanfield)
Re: Q for devs - Broken functionality or incorrect forum pos

OK, that does work... please would you send that method to those responsible for the documentation as it's not exactly straight forward for those not so familiar with Zenoss?

I think the fact that the template name must be "HardDisk" is why I was failing - it's things like this that really get on my nerves with Zenoss, it seems like it's going to be a great product - it's just a little unrefined at present!

Thank you beanfield for all your help!

Report Abuse

Like (0)
beanfield
161 posts since
Apr 16, 2008

Currently Being Moderated

5. Jun 19, 2008 12:12 PM (in response to HaydnH)
Re: Q for devs - Broken functionality or incorrect forum pos

"HaydnH" wrote:

OK, that does work... please would you send that method to those responsible for the documentation as it's not exactly straight forward for those not so familiar with Zenoss?

I think the fact that the template name must be "HardDisk" is why I was failing - it's things like this that really get on my nerves with Zenoss, it seems like it's going to be a great product - it's just a little unrefined at present!

Thank you beanfield for all your help!

Cool, glad it's working! The ironic part is that my boss had recently asked if we could monitor disk i/o. I didn't even know that the UCDHardDiskMap snmp data collector was available...or that snmp could monitor disk i/o. I thought I'd be doing some kind of custom command plugin, so doing this saved me a lot of time. :)

Also, I didn't really see this mentioned in the docs...just a loose mention of the Collector Plugins Tab http://www.zenoss.com/community/docs/zenoss-guide/2.2.0/ch09s08.html#d0e2913 I'd be happy to forward to anyone at zenoss if it helps anyone else...I just sent an email to community@zenoss.com.

Report Abuse

Like (0)

zmg

2 posts since
Jun 23, 2008

Currently Being Moderated

6. Jun 23, 2008 5:29 PM (in response to beanfield)

Re: Q for devs - Broken functionality or incorrect forum pos

MIB Module: 
UCD-DISKIO-MIB     .1.3.6.1.4.1.2021.13.15
diskIOTable     .1.3.6.1.4.1.2021.13.15.1
diskIOEntry     .1.3.6.1.4.1.2021.13.15.1.1

diskIOIndex     .1.3.6.1.4.1.2021.13.15.1.1.1     Integer32     Range: 0-65535
diskIODevice     .1.3.6.1.4.1.2021.13.15.1.1.2     Text          Size: 0-255
diskIONRead     .1.3.6.1.4.1.2021.13.15.1.1.3     Counter
diskIONWritten     .1.3.6.1.4.1.2021.13.15.1.1.4     Counter
diskIOReads     .1.3.6.1.4.1.2021.13.15.1.1.5     Counter
diskIOWrites     .1.3.6.1.4.1.2021.13.15.1.1.6     Counter
diskIOLA1     .1.3.6.1.4.1.2021.13.15.1.1.9     Integer32     Range: 0-100
diskIOLA5     .1.3.6.1.4.1.2021.13.15.1.1.10     Integer32     Range: 0-100
diskIOLA15     .1.3.6.1.4.1.2021.13.15.1.1.11     Integer32     Range: 0-100
diskIONReadX     .1.3.6.1.4.1.2021.13.15.1.1.12     Counter64
diskIONWrittenX     .1.3.6.1.4.1.2021.13.15.1.1.13     Counter64

Data Points: Types

COUNTER          Saves the rate of change of the value over a step period.
DERIVE          Same as COUNTER, but it allows negative values as well.
ABSOLUTE     Saves the rate of change, but assumes that the previous value is set to 0.
GAUGE          Saves only the actual value itself. There are no divisions or calculations.

A few notes I found helpful when trying to set this up.

The 'zenperfsnmp restart' was a pretty helpful tip, thanks.

This should probably be converted into a community zenpack.

Report Abuse
Like (0)

HaydnH
71 posts since
May 30, 2008

Currently Being Moderated

7. Jun 24, 2008 6:37 AM (in response to zmg)
Re: Q for devs - Broken functionality or incorrect forum pos

"zmg" wrote:

This should probably be converted into a community zenpack.

Actually, there is no need to add this as a community Zenpack, if the single template was included with Zenoss Core, all you would have to do to get diskIO monitoring is add the data collector to /device/server/linux. I've been meaning to create a ticket with this post and a sample template attached but haven't got around to it yet.

Why there isn't a template already is beyond me - the DiskIO data collector does nothing without a matching template!

Report Abuse

Like (0)
zmg
2 posts since
Jun 23, 2008

Currently Being Moderated

8. Jun 24, 2008 4:06 PM (in response to HaydnH)
Re: Q for devs - Broken functionality or incorrect forum pos

I'd second that. Last time I'd used Zabbix I seem to remember going through some hoops to get this too,.

Report Abuse

Like (0)

laguest

39 posts since
Oct 1, 2007

Currently Being Moderated

9. Apr 15, 2009 5:13 PM (in response to zmg)

Re: Q for devs - Broken functionality or incorrect forum pos

I have this working but it only shows sda and sdb, what I am looking for is sda1, sda2, sda3, etc. IO for each partition, can this do it?

What I have is this.


UCD-DISKIO-MIB::diskIONRead.17 = Counter32: 3603906560
UCD-DISKIO-MIB::diskIONRead.18 = Counter32: 18904064
UCD-DISKIO-MIB::diskIONRead.19 = Counter32: 273843712
UCD-DISKIO-MIB::diskIONRead.20 = Counter32: 699032576
UCD-DISKIO-MIB::diskIONRead.21 = Counter32: 4096
UCD-DISKIO-MIB::diskIONRead.22 = Counter32: 184418304
UCD-DISKIO-MIB::diskIONRead.23 = Counter32: 869376
UCD-DISKIO-MIB::diskIONRead.24 = Counter32: 1497600
UCD-DISKIO-MIB::diskIONRead.25 = Counter32: 2425037824

My OID is .1.3.6.1.4.1.2021.13.15.1.1.3

But it only maps .1.3.6.1.4.1.2021.13.15.1.1.3.17 which is sda, the .18, .19 are each slice

Report Abuse
Like (0)

Chet Luther
1,302 posts since
May 22, 2007

Currently Being Moderated

10. Apr 16, 2009 1:48 PM (in response to laguest)
Re: Q for devs - Broken functionality or incor

On Apr 15, 2009, at 5:13 PM, laguest wrote:

I have this working but it only shows sda and sdb, what I am looking
for is sda1, sda2, sda3, etc. IO for each partition, can this do it?

What I have is this.

UCD-DISKIO-MIB::diskIONRead.17 = Counter32: 3603906560
UCD-DISKIO-MIB::diskIONRead.18 = Counter32: 18904064
UCD-DISKIO-MIB::diskIONRead.19 = Counter32: 273843712
UCD-DISKIO-MIB::diskIONRead.20 = Counter32: 699032576
UCD-DISKIO-MIB::diskIONRead.21 = Counter32: 4096
UCD-DISKIO-MIB::diskIONRead.22 = Counter32: 184418304
UCD-DISKIO-MIB::diskIONRead.23 = Counter32: 869376
UCD-DISKIO-MIB::diskIONRead.24 = Counter32: 1497600
UCD-DISKIO-MIB::diskIONRead.25 = Counter32: 2425037824

My OID is .1.3.6.1.4.1.2021.13.15.1.1.3

But it only maps .1.3.6.1.4.1.2021.13.15.1.1.3.17 which is sda, the .
18, .19 are each slice

Prior to version 2.3.0 of Zenoss you need to update the UCDHardDiskMap
modeler plugin directly to change which disks it will model. Open
$ZENHOME/Products/DataCollector/plugins/zenoss/snmp/UCDHardDiskMap.py.
You're looking for the following line.

if not re.search('^[hs]d[a-z]$', om.id): continue

If you wanted to include all of the partitions, you would change it to:

if not re.search('^[hs]d[a-z]\d*$', om.id): continue

Starting with Zenoss 2.3.0, this is controlled by the
zHardDiskMapMatch zProperty so that code modifications are not
necessary.
_______________________________________________
zenoss-dev mailing list
zenoss-dev@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-dev

Report Abuse

Like (0)
laguest
39 posts since
Oct 1, 2007

Currently Being Moderated

11. Apr 21, 2009 7:55 PM (in response to Chet Luther)
Re: Q for devs - Broken functionality or incor

Thanks Chet,

worked great!!

Report Abuse

Like (0)
Brian Seklecki
5 posts since
Sep 23, 2011

Currently Being Moderated

12. Oct 7, 2011 3:42 PM (in response to beanfield)
Re: Q for devs - Broken functionality or incorrect forum pos

beanfield :

Is this still the current best-practice for getting per-device diskIO graphs from Net-SNMP's diskIOTable ?

I added zenoss.snmp.UCDHardDiskMap to my ModelerPlugins but my graphs under hard disk are showing 'no data'

Not a problem, we can just use a custom template instead?

But I'm not sure how your instructions are supposed to work with Table Index magic.

AFAIK, if I add a single numeric OID to a template, it will graph it.

But in your example, you're listing an OID of a tree branch.

How is the graph supposed to know how to handle this?

Is it supposed to Draw a graph for each disk in an SNMP walk operation (minus those in the regex?)

-------------

Here's a picture of my SNMP data points being configured per your instruction

And here's the resulting graphs, showing NAN/Zero values, as I suspect:

Message was edited by: Brian Seklecki

Report Abuse

Like (0)

Go to original post

Jun 17, 2008 10:09 AM

Q for devs - Broken functionality or incorrect forum post??

Actions

More Like This