Hardware specs

Up to Discussions in zenoss-users

96135 Views 13 Replies Latest reply: Sep 25, 2009 9:14 AM by jmp242

mosburn

78 posts since
Feb 21, 2008

Currently Being Moderated

Dec 5, 2008 2:37 PM

I am working on a large install of Zenoss and am trying to figure out the ratio of hardware resources/number of devices. My current server is a beefy dual core 2.4 Ghz, 8gb ram and more then enough disk space that is constantly pegged with 1400 devices being monitored. To relieve some of the load we have set up remote collectors at all of our sites and this helped some, but I regularly see loads of 9 and 10 on the main server. Has any one figured out the correct requirements for how beefy a server needs to be before I turn on syslog?

Thanks,

Michael

Like (0)

Guest

Currently Being Moderated

1. Dec 5, 2008 2:47 PM (in response to Guest )
Hardware specs

mosburn wrote:

I am working on a large install of Zenoss and am trying to figure out the ratio of hardware resources/number of devices. My current server is a beefy dual core 2.4 Ghz, 8gb ram and more then enough disk space that is constantly pegged with 1400 devices being monitored. To relieve some of the load we have set up remote collectors at all of our sites and this helped some, but I regularly see loads of 9 and 10 on the main server. Has any one figured out the correct requirements for how beefy a server needs to be before I turn on syslog?

As with any performance related issue for any application, you must
first determine *what* is causing the load. Just throwing hardware or
new collectors at it may be more trouble than it is worth.

If you have determined that your disks are the bottleneck, you next need
to determine what process is hogging the disks. You can use something
like iotop for this. http://guichaz.free.fr/iotop/

Once you've determined what process is hogging all the disk, you can
than look at the type of data it is reading/writing. Is the data
valuable? If not, put it on tmpfs. Is it hogging the disk based on
throughput, seek times or latency? Then you can optimize a storage
solution based upon the characteristics of your data.

Your solution might be as simple as adjusting some caching parameters,
but you won't know until you know why exactly the disks are being slammed.

Nathaniel
_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users

Like (0)
Guest

Currently Being Moderated

2. Dec 5, 2008 2:47 PM (in response to mosburn)
Hardware specs

mosburn wrote:

I am working on a large install of Zenoss and am trying to figure out the ratio of hardware resources/number of devices. My current server is a beefy dual core 2.4 Ghz, 8gb ram and more then enough disk space that is constantly pegged with 1400 devices being monitored. To relieve some of the load we have set up remote collectors at all of our sites and this helped some, but I regularly see loads of 9 and 10 on the main server. Has any one figured out the correct requirements for how beefy a server needs to be before I turn on syslog?

As with any performance related issue for any application, you must
first determine *what* is causing the load. Just throwing hardware or
new collectors at it may be more trouble than it is worth.

If you have determined that your disks are the bottleneck, you next need
to determine what process is hogging the disks. You can use something
like iotop for this. http://guichaz.free.fr/iotop/

Once you've determined what process is hogging all the disk, you can
than look at the type of data it is reading/writing. Is the data
valuable? If not, put it on tmpfs. Is it hogging the disk based on
throughput, seek times or latency? Then you can optimize a storage
solution based upon the characteristics of your data.

Your solution might be as simple as adjusting some caching parameters,
but you won't know until you know why exactly the disks are being slammed.

Nathaniel
_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users

Like (0)
jbaird
166 posts since
Sep 18, 2007

Currently Being Moderated

3. Dec 5, 2008 3:31 PM (in response to Guest )
RE: Hardware specs

I would definitely say that you are pushing it with one dual core CPU and 8GB of ram with 1400 devices. Is this machine swapping? It more than likely definitely needs more physical memory -- and with 1400 devices, I would say that you need more CPU horsepower as well.

How is the I/O wait on the machine? What kind of filesystem and RAID are you running?

Josh

Report Abuse

Like (0)
Matt Ray
2,484 posts since
Apr 5, 2008

Currently Being Moderated

4. Dec 5, 2008 4:00 PM (in response to jbaird)
Hardware specs

Of course Nathaniel gives you the well-reasoned approach, I'll give
you the half-cocked answer. 8 gigs of RAM sounds low. From the
thread "How big is your Zenoss (community) install?" (http://forums.zenoss.com/viewtopic.php?t=6230
) it looks like the largest install was around 1500 devices on a box
with 32 gigs of RAM.

Thanks,
Matt Ray
Zenoss Community Manager
community.zenoss.com
mray@zenoss.com

_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users

Report Abuse

Like (0)
Guest

Currently Being Moderated

5. Dec 5, 2008 4:18 PM (in response to Matt Ray)
Hardware specs

I should also add that the difficulty of just going with the number of
devices is that some devices are just being pinged and some are
collecting hundreds of datapoints. It is certainly possible to ping
1500 devices on 8gb of ram. However, how much ram you'll need will
depend on your environment.

iotop will show you if your disk load is being caused by swaping, you
will then know that the issue is RAM.

Nathaniel

Matt Ray wrote:

Of course Nathaniel gives you the well-reasoned approach, I'll give
you the half-cocked answer. 8 gigs of RAM sounds low. From the
thread "How big is your Zenoss (community) install?" (http://forums.zenoss.com/viewtopic.php?t=6230
) it looks like the largest install was around 1500 devices on a box
with 32 gigs of RAM.

Thanks,
Matt Ray
Zenoss Community Manager
community.zenoss.com
mray@zenoss.com

_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users

_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users

Like (0)
mosburn
78 posts since
Feb 21, 2008

Currently Being Moderated

6. Dec 8, 2008 12:41 PM (in response to Guest )
RE: Hardware specs

All of the servers that I monitor do full snmp monitoring and only about 50 are ping only. Watching top we are constantly in danger of swapping and jump right in when a full model happens. Thanks for the insight on what I need to order.

Report Abuse

Like (0)
Andrea Consadori
863 posts since
Feb 11, 2008

Currently Being Moderated

7. Dec 8, 2008 2:41 PM (in response to mosburn)
RE: Hardware specs

i use a virtual machine.

i monitor 450 device via snmp and some script , 50 device ping only.

Next week i saw that my server was swapping and i've to increase memory from 1.5 gb to 2 gb to stop it swapping.

i read in a post (i don't remember where) that zenoss collector limit is set to 1000 - 1500 because with more device zenoss is not capable of check all before another cicle)

Report Abuse

Like (0)
Matt Ray
2,484 posts since
Apr 5, 2008

Currently Being Moderated

8. Dec 8, 2008 4:06 PM (in response to Andrea Consadori)
Hardware specs

I'm not aware of a limit enforced by the application, the only limits
I'm aware of are the speed of the software on hardware. So in theory
the bigger the machine the bigger the install you can support.

Thanks,
Matt Ray
Zenoss Community Manager
community.zenoss.com
mray@zenoss.com

On Dec 8, 2008, at 1:41 PM, konsa79 wrote:

i use a virtual machine.

i monitor 450 device via snmp and some script , 50 device ping only.

Next week i saw that my server was swapping and i've to increase
memory from 1.5 gb to 2 gb to stop it swapping.

i read in a post (i don't remember where) that zenoss collector
limit is set to 1000 - 1500 because with more device zenoss is not
capable of check all before another cicle)

_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users

_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users

Report Abuse

Like (0)
chardy
23 posts since
Nov 18, 2008

Currently Being Moderated

9. Dec 8, 2008 9:02 PM (in response to Matt Ray)
RE: Hardware specs

Well, it's important to take the data from the "How big is your install" and tune the heck out of my.cnf, zope.conf and zopectl.conf.

By default I'd say ZenOSS Core is configured for a small to medium sized install.

I'd also make sure the modeller and or other daemon's aren't spewing out errors, since that seems to cause zen to go into wait states.

Report Abuse

Like (0)
Andrea Consadori
863 posts since
Feb 11, 2008

Currently Being Moderated

10. Dec 10, 2008 4:31 PM (in response to chardy)
RE: Hardware specs

admin manual - page 9

Scaling Zenoss

it says that after 1000-1200 device you've to add another collector because it became harder to end the cicle.

Report Abuse

Like (0)
jenkinskj
330 posts since
Jul 30, 2009

Currently Being Moderated

11. Sep 24, 2009 2:49 PM (in response to mosburn)
Re: Hardware specs

Team,

I was hoping to gather some updated information regarding Zenoss physical server specifications and use of virtual servers.

Hardware Specifications ->

- Is it advised to use physical servers for Zenoss?
- Should we install the Zenoss Network Management Server Zenoss application on one server and MySQL DB on another?
- Are specifications and sizing recommendations available?
- How well does Zenoss work on virtual servers (VMWare / ESX)?
- Is it recommended that virtual servers be installed with Zenoss in a production environment?

Distributed Data Collectors for Zenoss Core ->

- Is there a cost to use distributed data collector with Zenoss Core?
- Can distributed data collectors be installed on virtual servers?
- Are there specifications to consider when installing a data collector using a virtual (CPU, Memory, Disk, etc.)?

Thank you,
- Ken

Report Abuse

Like (0)
Andrea Consadori
863 posts since
Feb 11, 2008

Currently Being Moderated

12. Sep 24, 2009 5:47 PM (in response to jenkinskj)
Re: Hardware specs

My experience and opinion

Hardware Specifications ->
>
- Is it advised to use physical servers for Zenoss?

I use virtual one for managing more than 600 hosts

- Should we install the Zenoss Network Management Server Zenoss
application on one server and MySQL DB on another?

same server

- Are specifications and sizing recommendations available?

in my envirorment ibm quad core 1.39 ghz and 3 gb of ram

- How well does Zenoss work on virtual servers (VMWare / ESX)?

before I've installed zenoss on centos 32bit guest on vmware server and
works but I've some preformance issue (no vmware bare metal, windows 2003
server with installed vmware server)

this summer I've time to backup zenoss, format the server and install xen
server (bare metal) and I create a new centos 32 bit guest machine and
restore my installation....

it's a colplete different world.. same hardware... better performance

- Is it recommended that virtual servers be installed with Zenoss in a
production environment?

I use it in production because using core version I've no maintenance
contract with zenoss team so when I make update sometimes I can have some
issue, so I can revert back to my old virtual machine (loosing 10 minutes of
rrd's).

>
Distributed Data Collectors for Zenoss Core ->
>
- Is there a cost to use distributed data collector with Zenoss Core?

no, you've to pay to have global dashboard (distribuite data collectors only
grab performance info's and send info to zenoss core master server I
think -- never try).

- Can distributed data collectors be installed on virtual servers?

why not

- Are there specifications to consider when installing a data collector
using a virtual (CPU, Memory, Disk, etc.)?
>
Thank you,
- Ken
>

Report Abuse

Like (0)
jmp242
4,060 posts since
Mar 7, 2007

Currently Being Moderated

13. Sep 25, 2009 9:14 AM (in response to jenkinskj)
Re: Hardware specs

I'll respond in line with my comments.

jenkinskj wrote, On 9/24/2009 2:50 PM:
Team,

I was hoping to gather some updated information regarding Zenoss physical server specifications and use of virtual servers.

Hardware Specifications ->

- Is it advised to use physical servers for Zenoss?
I prefer physical servers for performance, but if you run your own Linux
install a decently performing VM should work. I/O is going to be the
limiting factor I should think with VMs, so if you're not running fast
local storage...
- Should we install the Zenoss Network Management Server Zenoss application on one server and MySQL DB on another?
It's much easier to set up with both on the same server. I haven't
actually seen any indications from the Community or Zenoss that there is
a reason to separate the servers.
- Are specifications and sizing recommendations available?
Not as such, but see:
docs/DOC-2445#WhatisthemaximumnumberofdevicesZenosscanmonitor
- How well does Zenoss work on virtual servers (VMWare / ESX)?
- Is it recommended that virtual servers be installed with Zenoss in a production environment?
I wouldn't use the provided appliance, and there isn't any
recommendation either way that I've seen. My personal preference is for
physical servers.

Distributed Data Collectors for Zenoss Core ->

- Is there a cost to use distributed data collector with Zenoss Core?
The answer is no, if you can hack it together to get it working, though
you do need to check some of the old forum threads to understand the pain.

If you want it to "just work", you have to purchase Enterprise.
- Can distributed data collectors be installed on virtual servers?
Sure.
- Are there specifications to consider when installing a data collector using a virtual (CPU, Memory, Disk, etc.)?
Disk I/O is what is probably the gotcha.

--
James Pulver
Information Technology Area Supervisor
LEPP Computer Group
Cornell University

Report Abuse

Like (0)

Go to original post

Dec 5, 2008 2:37 PM

Hardware specs

Actions

More Like This

Incoming Links