I think the biggest problem with A/B Testing is the "single winner" fallacy that most a/b testing and multivariate testing tools perpetrate.
In my latest post on the Performable blog I explain how you can prevent falling into this trap and how you can continuously improve your business using A/B Testing.
Remember this: Each visitor to your site is different and brings a different context. Some are fans who follow your company on Twitter, others are first-timers who have never heard of you, and others are potential customers in the research stage, comparing you feature-by-feature to one of your competitors.
Interested in improving your marketing performance? Read the rest on the Performable Blog: Avoiding the biggest problem with A/B Testing