So I really hammered down and figured out what happened here.
Since the event summary had the component name I KNEW that data was saved somewhere but getting clobered by the empty component value from the COMMAND DataSource when applied to the event.
By inserting log.debug statements into zencommand this behavior is observed. Right around line: 565 you will see code that looks like this
for ev in self.thresholds.check(path, time.time(), value):
ev['eventKey'] = cmd.key(label)
ev['component'] = cmd.component
self.sendThresholdEvent(**ev)
If you add debug statements and make it look like this
for ev in self.thresholds.check(path, time.time(), value):
log.debug("Returned Threshold Data")
log.debug("eventKey: %s component: %s" % (cmd.key(label), cmd.component))
log.debug(ev)
ev['eventKey'] = cmd.key(label)
self.sendThresholdEvent(**ev)
You will find the debug statements mixed in the normal output when running zencommand manually with debug logging enabled.
pilot1 is my Pillar SAN device to be modeled.
zencommand run -v10 -d pilot1
DEBUG:zen.zencommand:Process check_pillar_filesystem.py -H 10.5.3.26 -c r3ad-th1s -i 2 stopped (0), 0.445986 elapsed
DEBUG:zen.zencommand:The result of "$ZENHOME/Products/PillarMonitor/libexec/check_pillar_filesystem.py -H 10.5.3.26 -c r3ad-th1s -i 2" was "STATUS OK| online=1 performance=1 ha=0 redundancy=1 readonly=1 pinneddata=1 management=1"
DEBUG:zen.zencommand:storing online = 1.0 in: Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_online
DEBUG:zen.RRDUtil:/usr/local/zenoss/perf/Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_online.rrd: 1.0
DEBUG:zen.zencommand:rrd save result: 1.0
DEBUG:zen.thresholds:Checking value 1.0 on Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_online
DEBUG:zen.MinMaxCheck:Checking FilesystemHealth_online 1.0 against min 1 and max 1
DEBUG:zen.zencommand:storing performance = 1.0 in: Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_performance
DEBUG:zen.RRDUtil:/usr/local/zenoss/perf/Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_performance.rrd: 1.0
DEBUG:zen.zencommand:rrd save result: 1.0
DEBUG:zen.thresholds:Checking value 1.0 on Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_performance
DEBUG:zen.MinMaxCheck:Checking FilesystemHealth_performance 1.0 against min 1 and max 1
DEBUG:zen.zencommand:storing ha = 0.0 in: Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_ha
DEBUG:zen.RRDUtil:/usr/local/zenoss/perf/Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_ha.rrd: 0.0
DEBUG:zen.zencommand:rrd save result: 0.0
DEBUG:zen.thresholds:Checking value 0.0 on Devices/pilot1/PillarFile/ProdFS/FilesystemHealth_ha
DEBUG:zen.MinMaxCheck:Checking FilesystemHealth_ha 0.0 against min 1 and max 1
INFO:zen.MinMaxCheck:label: ProdFS self.context().componentName: ProdFS
DEBUG:zen.zencommand:Returned Threshold Data
DEBUG:zen.zencommand:eventKey: PillarFilesystem|FilesystemHealth_ha component:
DEBUG:zen.zencommand:{'device': 'pilot1', 'eventClass': '/Pillar', 'component': 'ProdFS', 'severity': 4, 'summary': 'pilot1 ProdFS threshold of FilesystemHealth not met: current value 0.00'}
DEBUG:zen.zencommand:Sending event {'severity': 4, 'component': 'ProdFS', 'agent': 'zencommand', 'summary': 'pilot1 ProdFS threshold of FilesystemHealth not met: current value 0.00', 'manager': 'localhost', 'eventKey': 'PillarFilesystem|FilesystemHealth_ha', 'device': 'pilot1', 'eventClass': '/Pillar'}
If you havent found them yet the important lines are these
DEBUG:zen.zencommand:Returned Threshold Data
DEBUG:zen.zencommand:eventKey: PillarFilesystem|FilesystemHealth_ha component:
DEBUG:zen.zencommand:{'device': 'pilot1', 'eventClass': '/Pillar', 'component': 'ProdFS', 'severity': 4, 'summary': 'pilot1 ProdFS threshold of FilesystemHealth not met: current value 0.00'}
Basically the first list is printing component from cmd.component which is used in the event after processing.
You'll notice that cmd.component just returns empty data. Yet when dumping the "ev" list its contained with the RIGHT value.
So the problem then is that code above simply saves whatever cmd.component is for the final value which we dont want.
So I made a quick patch to solve this issue for me. I left my debug statements in there but you can remove them.
for ev in self.thresholds.check(path, time.time(), value):
log.debug("Returned Threshold Data")
log.debug("eventKey: %s component: %s" % (cmd.key(label), cmd.component))
log.debug(ev)
ev['eventKey'] = cmd.key(label)
#Check to see if the component exists
if cmd.component == '':
#use the component returned from MinMaxThreshold instead.
cmd.component = ev['component']
else:
#fall back on default behavior if component is not empty.
ev['component'] = cmd.component
self.sendThresholdEvent(**ev)
