Did you try the Zenpack?

docs/DOC-5861

--

James Pulver

Information Technology Area Supervisor

LEPP Computer Group

Cornell University

I'm not sure about your specific error, but I did wade through setting up distributed collectors recently. I don't have a formal how to, but here are the high level steps from the notes I took. There maybe *better* ways to do it, but these were the steps I followed, and so far it appears to be working.

1. Install Zenoss on your 'Main' Server' as if it were a standalone host. Ensure that box is working
2. Install ZenPacks.community.DistributedCollectors on your main server
   
3. Generate SSH Keys on your main server
   

   1. sudo su - zenoss
      mkdir ~/.ssh
      ssh-keygen -t rsa -f ~/.ssh/id_rsa -P ""
      chmod 700 ~/.ssh
      chmod go-rwx ~/.ssh/*
      cp ~/.ssh/id_rsa.pub /tmp/
      chmod a+r /tmp/id_rsa.pub

      
      
4. Install Zenoss on your 'Remote Collector'. For now, pretend its another stand alone server, so install it just as you did in step 1 on the main server
5. Open Firewall Rules between your main server and remote collector. docs/DOC-10246
6. Add the Public Key from the keys you generated on the master server to the authorized_keys for the zenoss user on the remote collector.
7. Test that SSH Key authentication is working correctly by logging into the main server as the zenoss user and attempt to ssh into the remote collector, you should be able to connect without needing to specify a password
8. Start Zenoss (If you have not already). Let it run through its standard init process.
9. Add the remote collector to Zenoss
   1. Log into the master server UI
   2. Navigate to Advanced --> Collectors
   3. From the gear menu, select "Add Remote Monitor"
   4. Enter the FQDN of the Remote Collector, and click OK (There might be a pause of several seconds)
   5. Place a check box next to the collector you just added
   6. From the gear menu, select "Update Remote Monitors
   7. You should get a message in the yellow bar indicating it was successfully updated.
10. Restart Zenoss on the remote collector
11. Disable MySQL on the remote collector. Because of the RPM dependencies, I couldn't remove MySQL.  So just turning it off

In a nutshell, those were the steps I took.

That is correct, after the restart a few daemons are disabled, and your 'master' server becomes the only UI presence. It will communicate with the remote collectors to fetch the data it needs

The sterps were written against 3.0, but quick review seems they are still revelant.

run a zenoss status on the master. At first read it sounds like zenjobs might not be running?

Alright, we're back in action. The above error happens when I'm trying to create a device to be monitored by the collector. The job is always active (the green spinny thing is next to it in the "jobs" section until I force kill it), but nothing shows up in the output. I was only able to get the above output because I noticed that navigating away from the page flashes a quick (i.e., a few tenths of a second) messages on the screen before it loads the new page. When you go back to it, the phantom output is gone.

As for zenoss status, here's the output from the master:

[zenoss@localhost ~]$ zenoss status

Daemon: zeoctl program running; pid=8287

Daemon: zopectl program running; pid=8292

Daemon: zenhub program running; pid=8332

Daemon: zenjobs program running; pid=8369

Daemon: zenping program running; pid=8431

Daemon: zensyslog program running; pid=8470

Daemon: zenstatus program running; pid=8476

Daemon: zenactions program running; pid=8506

Daemon: zentrap program running; pid=8604

Daemon: zenmodeler program running; pid=8609

Daemon: zenrender program running; pid=8644

Daemon: zenperfsnmp program running; pid=8668

Daemon: zencommand program running; pid=8702

Daemon: zenprocess program running; pid=8729

Daemon: zenwin program running; pid=8766

Daemon: zeneventlog program running; pid=8804

Daemon: zenjmx program running; pid=8833

And here's the output from the collector:

[zenoss@localhost ~]$ zenoss status

Daemon: zenping program running; pid=4713

Daemon: zensyslog program running; pid=4756

Daemon: zenstatus program running; pid=4758

Daemon: zentrap program running; pid=4796

Daemon: zenmodeler program running; pid=4836

Daemon: zenrender program running; pid=4862

Daemon: zenperfsnmp program running; pid=4895

Daemon: zencommand program running; pid=4921

Daemon: zenprocess program running; pid=4947

Daemon: zenwin program running; pid=4982

Daemon: zeneventlog program running; pid=5018

Daemon: zenjmx not running

Note that I've had the problem with zenjmx before, and fixed it by installing java and adding hubhost and monitor argments to /opt/zenoss/etc/zenjmx.conf, but I wanted to show the output as a fresh install exactly the way you described.