I was admitted there last year and attended their fly-out. My impression is that they admit a pool of people who have very well-defined research interests already, at least in terms of the field they want to do. If you are interested in micro, neuro, experimental, or political economy, it's a great place. However, make sure you are really certain about doing one of these fields because other fields (except for a bit of finance) are virtually non-existing. Of course you can also take classes at USC and UCLA, but that won't substitute having an advisor in your field at Caltech.
I think the small size of the program is an upside, as is their generous funding, and the California weather :tup:. I think some of their research also tends to be more high risk than at some other places, and hence it could be very rewarding if you're that type of researcher.