I did some testing over the last couple of days on how to tune the Zenoss Web Gui experience. Our Zenoss Core server is currently running on a recycled 1U server that was never designed for high I/O loads. Especially zenperfsnmp can be very I/O bound and easily saturates our internal raid controller which only provides disk mirroring and doesn't have any write cache.

In the original configuration this lead to a situation where the Zenoss Web Gui became practically unusable (wait times of up to 1 minute or more) every 5 minutes. Because zenperfsnmp starved the rest of the system to dead. The suggested modification didn't improve overall performance. But they managed to push the maximum Zenoss page load time into a much more enjoyable time range of 5-10 seconds.

Latency tuning involves a loot of Voodoo and can be very hardware dependent. So please take this advice with a big grain of salt:

Use Firefox 3.0:
Firefox 3.0 has a greatly improved caching and javascript engine. Using Firefox 3 speeds up Zenoss a lot without having to do any low level kernel stuff.

Use a battery buffered write cache:
Much has been said about Zenoss CPU + Memory requirements. But if you can don't forget to either buy a storage controller that supports a battery buffered write cache or use a high performance storage systems. If you don't you easily end spending a lot of CPU cycles in I/O wait state.

Use the Linux Deadline I/O scheduler:
This can be done by adding "elevator=deadline" as a kernel parameter in your grub bootloader configuration. The Deadline scheduler can hurt your overall performance, but improves latency for processes that mainly read.

Tune your Virtual Memory configuration:
Buffer and Virtual Memory tuning is an area of much debate and wild guessing, but the following parameters in /etc/syctl.conf improved latency on our system:

ProxyPass / http://localhost:8080/ retry=5
ProxyPassReverse / http://localhost:8080/

Use Apache mod_proxy and mod_cache:
Using mod_proxy can be nice, because it allows you to easily redirect request to Zenoss:

CacheEnable mem / 

But mod_proxy alone doesn't add any performance gains and the same functionality could be gained using Apache rewrite rules. What can improve latency is using mod_proxy in combination with mod_cache and memory web caching:

vm.vfs_cache_pressure = 5
vm.swappiness = 5
vm.overcommit_memory = 2
vm.dirty_background_ratio = 5
vm.dirty_ratio = 80
vm.dirty_writeback_centisecs = 3000
vm.dirty_expire_centisecs = 9000 

net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.ipv4.tcp_no_metrics_save=1
net.ipv4.tcp_congestion_control=bic

This list is far from complete and different hardware probably will lead to different results. So feel free to comment.