Testing response times throughout the project lifecycle can help as an early indicator of performance issues.
Production-like environments are expensive to use - this can lead to limited load testing, and the resultant risks.
When benchmarking performance testing, don't change the codebase and the environment/infrastructure at the same time - freeze one whilst modifying the other.
Stress testing - finding where the app breaks. Load testing - testing how the app deals with a typical/estimated load
A degree of performance testing can be done at each CI build cycle, but not always to the degree that will show system-level performance issues.
Some photos of Stuart Moncrieff's Agile Performance Testing slidepack are available here: www.jds.net.au/news/melbourne-citcon-2008/