I think in many instances the test screening is important in weeding out the kinks in a film, but there has to be a give and take with the director, and films subjected to test screenings should be screened to appropriate audiences. They also shouldn't be treated as a guideline in shaping a film. Two films destroyed by test screenings spring to mind:
Payback - Screened to a general audience, and tested poorly, with responses all over the place. Paramount panicked, and Helgeland's bleak, 70's revenge flick was cut to pieces. The whole color palette was re-timed, the final act was re-shot, new characters and plot points were introduced, and the film lost most of its bite. The "Straight up" version, featuring Helgeland's original cut annihilates the theatrical version in just about every way imaginable. If the film had been shown to a more savvy crowd, they'd have likely dialed into the film's 70's tone, and been considerably more positive. Then again, Paramount could have also given Helgeland the benefit of the doubt, as his version was considerably better received in the long run.
Legend - Hot on the heels of the theatrical fiasco that was Blade runner, Ridley tested his fantasy flick with a crowd typically known to embrace such types of films. The story about a bunch of potheads guffawing at the film's more fairy-tale like moments, and thus changing the course of the production has become a sort of "test screening" parable in and of itself in the years since. Again the screening dictated massive alterations, and a severe distilling of the original product. The theatrical cut of legend is disjointed and nonsensical, the director's cut is a much stronger, full bodied flick.
As the ancient Tibetan philosophy states:"Don't start none... won't be none...".