I'm a bit annoyed by the recent tendency of my linux to swap out a lot of memory from my apps while a lot of memory is used only as cache.
Since I have 4Gb ram and the only really memconsuming application I use is VMware workstation, I did an experiment and swapoff for 2 days: ok, some apps like chrome or firefox will die on start with 2 VM's running, but overall the system is far more responsive without swapin/swapout on my slow sata disk!
I googled for a way to tune swap behaviour on linux and find vm.swappiness.
Swappiness can be read at /proc/sys/vm/swappiness, and mine was set to 60 out of a range between 0 and 100. From what I understand, low swappiness will cause cache to loose the memory reclamation battle, while high swappiness will cause unused pages to loose, then
low swappines = less cache, less swap
hi swappiness = more cache, more swap
My swappiness was set to 60.
Experimentally, I will set a swappiness of 10 for a while, swapon() and see what happens.
sudo sysctl -w vm.swappiness=10