
Re: An independent integration test suite
Posted:
Jul 15, 2013 4:16 PM


On Monday, July 15, 2013 3:07:38 AM UTC4, Albert Rich wrote: > On Sunday, July 14, 2013 12:58:04 AM UTC10, Nasser M. Abbasi wrote: > > > > > I think the way to do this is by automation. The way I was > > > doing it so far is not practical. It should instead by > > > done by writing a script to run through all the test cases > > > automatically, otherwise it will take me another 2 years to > > > finish. But the problem comes when one wants to combine > > > results of many CAS'es in one document. > > > > I agree completely that automating the testing of systems AND grading their results is essential for any testsuite of significant size. However, I do not think it necessary or practical to combine the raw test results of multiple systems into a single document. > > > > Instead a system's testsuite program should compare its solution for each problem with the optimal solution in the testsuite, and assign it a numerical grade. Then the grades of all the systems for each of the problems, rather than the raw results, should be combined into a single table. > > > > This was how I compiled the table of Charlwood Fifty test results for 7 systems that was posted in another sci.math.symbolic thread. Although the grading system used was relatively coarse (2 for an optimal antiderivative, 1 for a nonoptimal antiderivative, and 0 if unable to integrate), the table's bottomline made it easy to compare the relative performance of the systems on this small testsuite. Obviously, it would be even more useful on a large testsuite with all the major systems tested... > > > > Albert
For some of the integration problems Axiom finds more than one result. For instance, t1:=sqrt(a+b*x)/x
++ \b x + a (1)  x Type: Expression(Integer) (2) > integrate(t1,x)
(2) ++ ++ ++  2\a \b x + a + b x + 2a ++ [\a log() + 2\b x + a , x ++ ++ \b x + a ++  2\ a atan() + 2\b x + a ] ++ \ a Type: Union(List(Expression(Integer)),...)
The posted suite only shows one answer. How should a test suite handle this?
Tim

