Archived community.zenoss.org | full text search
Skip navigation
96135 Views 13 Replies Latest reply: Sep 25, 2009 9:14 AM by jmp242 RSS
mosburn Rank: White Belt 78 posts since
Feb 21, 2008
Currently Being Moderated

Dec 5, 2008 2:37 PM

Hardware specs

I am working on a large install of Zenoss and am trying to figure out the ratio of hardware resources/number of devices. My current server is a beefy dual core 2.4 Ghz, 8gb ram and more then enough disk space that is constantly pegged with 1400 devices being monitored. To relieve some of the load we have set up remote collectors at all of our sites and this helped some, but I regularly see loads of 9 and 10 on the main server. Has any one figured out the correct requirements for how beefy a server needs to be before I turn on syslog?

Thanks,

Michael
  • Currently Being Moderated
    1. Dec 5, 2008 2:47 PM (in response to Guest )
    Hardware specs
    mosburn wrote:

     

     

    I am working on a large install of Zenoss and am trying to figure out the ratio of hardware resources/number of devices. My current server is a beefy dual core 2.4 Ghz, 8gb ram and more then enough disk space that is constantly pegged with 1400 devices being monitored. To relieve some of the load we have set up remote collectors at all of our sites and this helped some, but I regularly see loads of 9 and 10 on the main server. Has any one figured out the correct requirements for how beefy a server needs to be before I turn on syslog?

    As with any performance related issue for any application, you must
    first determine *what* is causing the load. Just throwing hardware or
    new collectors at it may be more trouble than it is worth.

    If you have determined that your disks are the bottleneck, you next need
    to determine what process is hogging the disks. You can use something
    like iotop for this. http://guichaz.free.fr/iotop/

    Once you've determined what process is hogging all the disk, you can
    than look at the type of data it is reading/writing. Is the data
    valuable? If not, put it on tmpfs. Is it hogging the disk based on
    throughput, seek times or latency? Then you can optimize a storage
    solution based upon the characteristics of your data.

    Your solution might be as simple as adjusting some caching parameters,
    but you won't know until you know why exactly the disks are being slammed.

    Nathaniel
    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • Currently Being Moderated
    2. Dec 5, 2008 2:47 PM (in response to mosburn)
    Hardware specs
    mosburn wrote:

     

     

    I am working on a large install of Zenoss and am trying to figure out the ratio of hardware resources/number of devices. My current server is a beefy dual core 2.4 Ghz, 8gb ram and more then enough disk space that is constantly pegged with 1400 devices being monitored. To relieve some of the load we have set up remote collectors at all of our sites and this helped some, but I regularly see loads of 9 and 10 on the main server. Has any one figured out the correct requirements for how beefy a server needs to be before I turn on syslog?

    As with any performance related issue for any application, you must
    first determine *what* is causing the load. Just throwing hardware or
    new collectors at it may be more trouble than it is worth.

    If you have determined that your disks are the bottleneck, you next need
    to determine what process is hogging the disks. You can use something
    like iotop for this. http://guichaz.free.fr/iotop/

    Once you've determined what process is hogging all the disk, you can
    than look at the type of data it is reading/writing. Is the data
    valuable? If not, put it on tmpfs. Is it hogging the disk based on
    throughput, seek times or latency? Then you can optimize a storage
    solution based upon the characteristics of your data.

    Your solution might be as simple as adjusting some caching parameters,
    but you won't know until you know why exactly the disks are being slammed.

    Nathaniel
    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • jbaird Rank: Green Belt 166 posts since
    Sep 18, 2007
    Currently Being Moderated
    3. Dec 5, 2008 3:31 PM (in response to Guest )
    RE: Hardware specs
    I would definitely say that you are pushing it with one dual core CPU and 8GB of ram with 1400 devices. Is this machine swapping? It more than likely definitely needs more physical memory -- and with 1400 devices, I would say that you need more CPU horsepower as well.

    How is the I/O wait on the machine? What kind of filesystem and RAID are you running?

    Josh
  • Matt Ray Rank: Zen Master 2,484 posts since
    Apr 5, 2008
    Currently Being Moderated
    4. Dec 5, 2008 4:00 PM (in response to jbaird)
    Hardware specs
    Of course Nathaniel gives you the well-reasoned approach, I'll give
    you the half-cocked answer. 8 gigs of RAM sounds low. From the
    thread "How big is your Zenoss (community) install?" (http://forums.zenoss.com/viewtopic.php?t=6230
    ) it looks like the largest install was around 1500 devices on a box
    with 32 gigs of RAM.

    Thanks,
    Matt Ray
    Zenoss Community Manager
    community.zenoss.com
    mray@zenoss.com

    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • Currently Being Moderated
    5. Dec 5, 2008 4:18 PM (in response to Matt Ray)
    Hardware specs
    I should also add that the difficulty of just going with the number of
    devices is that some devices are just being pinged and some are
    collecting hundreds of datapoints. It is certainly possible to ping
    1500 devices on 8gb of ram. However, how much ram you'll need will
    depend on your environment.

    iotop will show you if your disk load is being caused by swaping, you
    will then know that the issue is RAM.

    Nathaniel

    Matt Ray wrote:

     

     

    Of course Nathaniel gives you the well-reasoned approach, I'll give
    you the half-cocked answer. 8 gigs of RAM sounds low. From the
    thread "How big is your Zenoss (community) install?" (http://forums.zenoss.com/viewtopic.php?t=6230
    ) it looks like the largest install was around 1500 devices on a box
    with 32 gigs of RAM.

    Thanks,
    Matt Ray
    Zenoss Community Manager
    community.zenoss.com
    mray@zenoss.com

    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users


    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • Andrea Consadori ZenossMaster 863 posts since
    Feb 11, 2008
    Currently Being Moderated
    7. Dec 8, 2008 2:41 PM (in response to mosburn)
    RE: Hardware specs
    i use a virtual machine.

    i monitor 450 device via snmp and some script , 50 device ping only.

    Next week i saw that my server was swapping and i've to increase memory from 1.5 gb to 2 gb to stop it swapping.

    i read in a post (i don't remember where) that zenoss collector limit is set to 1000 - 1500 because with more device zenoss is not capable of check all before another cicle)
  • Matt Ray Rank: Zen Master 2,484 posts since
    Apr 5, 2008
    Currently Being Moderated
    8. Dec 8, 2008 4:06 PM (in response to Andrea Consadori)
    Hardware specs
    I'm not aware of a limit enforced by the application, the only limits
    I'm aware of are the speed of the software on hardware. So in theory
    the bigger the machine the bigger the install you can support.

    Thanks,
    Matt Ray
    Zenoss Community Manager
    community.zenoss.com
    mray@zenoss.com



    On Dec 8, 2008, at 1:41 PM, konsa79 wrote:

     

     

    i use a virtual machine.

    i monitor 450 device via snmp and some script , 50 device ping only.

    Next week i saw that my server was swapping and i've to increase
    memory from 1.5 gb to 2 gb to stop it swapping.

    i read in a post (i don't remember where) that zenoss collector
    limit is set to 1000 - 1500 because with more device zenoss is not
    capable of check all before another cicle)







    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users


    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • chardy Rank: White Belt 23 posts since
    Nov 18, 2008
    Currently Being Moderated
    9. Dec 8, 2008 9:02 PM (in response to Matt Ray)
    RE: Hardware specs
    Well, it's important to take the data from the "How big is your install" and tune the heck out of my.cnf, zope.conf and zopectl.conf.

    By default I'd say ZenOSS Core is configured for a small to medium sized install.

    I'd also make sure the modeller and or other daemon's aren't spewing out errors, since that seems to cause zen to go into wait states.
  • Andrea Consadori ZenossMaster 863 posts since
    Feb 11, 2008
    Currently Being Moderated
    10. Dec 10, 2008 4:31 PM (in response to chardy)
    RE: Hardware specs
    admin manual - page 9

    Scaling Zenoss

    it says that after 1000-1200 device you've to add another collector because it became harder to end the cicle.
  • jenkinskj Rank: Green Belt 330 posts since
    Jul 30, 2009
    Currently Being Moderated
    11. Sep 24, 2009 2:49 PM (in response to mosburn)
    Re: Hardware specs

    Team,

     

    I was hoping to gather some updated information regarding Zenoss physical server specifications and use of virtual servers.

     

    Hardware Specifications ->

     

    - Is it advised to use physical servers for Zenoss?

    - Should we install the Zenoss Network Management Server Zenoss application on one server and MySQL DB on another?

    - Are specifications and sizing recommendations available?

    - How well does Zenoss work on virtual servers (VMWare / ESX)?

    - Is it recommended that virtual servers be installed with Zenoss in a production environment?

     

    Distributed Data Collectors for Zenoss Core ->

     

    - Is there a cost to use distributed data collector with Zenoss Core?

    - Can distributed data collectors be installed on virtual servers?

    - Are there specifications to consider when installing a data collector using a virtual (CPU, Memory, Disk, etc.)?

     

    Thank you,

    - Ken

  • Andrea Consadori ZenossMaster 863 posts since
    Feb 11, 2008
    Currently Being Moderated
    12. Sep 24, 2009 5:47 PM (in response to jenkinskj)
    Re: Hardware specs

    My experience and opinion

     

    Hardware Specifications ->

    >

    - Is it advised to use physical servers for Zenoss?

     

    I use virtual one for managing more than 600 hosts

     

    - Should we install the Zenoss Network Management Server Zenoss

    application on one server and MySQL DB on another?

     

    same server

     

    - Are specifications and sizing recommendations available?

     

    in my envirorment ibm quad core 1.39 ghz and 3 gb of ram

     

    - How well does Zenoss work on virtual servers (VMWare / ESX)?

     

    before I've installed zenoss on centos 32bit guest on vmware server and

    works but I've some preformance issue (no vmware bare metal, windows 2003

    server with installed vmware server)

     

    this summer I've time to backup zenoss, format the server and install xen

    server (bare metal) and I create a new centos 32 bit guest machine and

    restore my installation....

     

    it's a colplete different world.. same hardware... better performance

     

    - Is it recommended that virtual servers be installed with Zenoss in a

    production environment?

     

    I use it in production because using core version I've no maintenance

    contract with zenoss team so when I make update sometimes I can have some

    issue, so I can revert back to my old virtual machine (loosing 10 minutes of

    rrd's).

     

    >

    Distributed Data Collectors for Zenoss Core ->

    >

    - Is there a cost to use distributed data collector with Zenoss Core?

     

    no, you've to pay to have global dashboard (distribuite data collectors only

    grab performance info's and send info to zenoss core master server I

    think -- never try).

     

    - Can distributed data collectors be installed on virtual servers?

     

    why not

     

    - Are there specifications to consider when installing a data collector

    using a virtual (CPU, Memory, Disk, etc.)?

    >

    Thank you,

    - Ken

    >

  • jmp242 ZenossMaster 4,060 posts since
    Mar 7, 2007
    Currently Being Moderated
    13. Sep 25, 2009 9:14 AM (in response to jenkinskj)
    Re: Hardware specs

    I'll respond in line with my comments.

     

     

     

     

    jenkinskj wrote, On 9/24/2009 2:50 PM:

    Team,

     

    I was hoping to gather some updated information regarding Zenoss physical server specifications and use of virtual servers.

     

    Hardware Specifications ->

     

    - Is it advised to use physical servers for Zenoss?

    I prefer physical servers for performance, but if you run your own Linux

    install a decently performing VM should work. I/O is going to be the

    limiting factor I should think with VMs, so if you're not running fast

    local storage...

    - Should we install the Zenoss Network Management Server Zenoss application on one server and MySQL DB on another?

    It's much easier to set up with both on the same server. I haven't

    actually seen any indications from the Community or Zenoss that there is

    a reason to separate the servers.

    - Are specifications and sizing recommendations available?

    Not as such, but see:

    docs/DOC-2445#WhatisthemaximumnumberofdevicesZenosscanmonitor

    - How well does Zenoss work on virtual servers (VMWare / ESX)?

    - Is it recommended that virtual servers be installed with Zenoss in a production environment?

    I wouldn't use the provided appliance, and there isn't any

    recommendation either way that I've seen. My personal preference is for

    physical servers.

     

    Distributed Data Collectors for Zenoss Core ->

     

    - Is there a cost to use distributed data collector with Zenoss Core?

    The answer is no, if you can hack it together to get it working, though

    you do need to check some of the old forum threads to understand the pain.

     

    If you want it to "just work", you have to purchase Enterprise.

    - Can distributed data collectors be installed on virtual servers?

    Sure.

    - Are there specifications to consider when installing a data collector using a virtual (CPU, Memory, Disk, etc.)?

    Disk I/O is what is probably the gotcha.

     

    --

    James Pulver

    Information Technology Area Supervisor

    LEPP Computer Group

    Cornell University

More Like This

  • Retrieving data ...