Since standardized testing is receiving such a hammering in this forum, we might as well ask about automated testing. Writing in LISP, SCHEME, or PROLOGUE it is straightforward to create programs that create dialogues that are eerily human sounding. It is also possible to explore a person's knowledge in a particular area in a systematic way by asking questions of increasing difficulty until the rate of incorrect answers becomes unacceptable. The program then switches to another subarea of content and begins anew.
Is there any good reason that automated testing should not replace standardized tests? Consider a seventh grader sitting down for a test. A few integer addition items might be followed by rationals, followed by reals, followed by algebraic expressions with mixed ops. Satisfied that was adequate understanding for a seventh grader, the program could switch to a hierarchy operations questions. A different seventh grader might be stymied by the reals and be tested next on subtraction. Thus in a given amount of time a much wider range of test questions can be considered.
Should a student spend the whole year dealing with a machine? No. Should an impartial assessment be made once a year to complement the biased daily assessments of teachers? Yes. How would the results of a test like this fit in a portfolio? Nicely, I think. It's not the whole story but its more complete than a standardized test especially at both tails of the distribution.