Date: Jul 15, 2013 4:16 PM
Author: daly@axiom-developer.org
Subject: Re: An independent integration test suite
On Monday, July 15, 2013 3:07:38 AM UTC-4, Albert Rich wrote:

> On Sunday, July 14, 2013 12:58:04 AM UTC-10, Nasser M. Abbasi wrote:

>

>

>

> > I think the way to do this is by automation. The way I was

>

> > doing it so far is not practical. It should instead by

>

> > done by writing a script to run through all the test cases

>

> > automatically, otherwise it will take me another 2 years to

>

> > finish. But the problem comes when one wants to combine

>

> > results of many CAS'es in one document.

>

>

>

> I agree completely that automating the testing of systems AND grading their results is essential for any test-suite of significant size. However, I do not think it necessary or practical to combine the raw test results of multiple systems into a single document.

>

>

>

> Instead a system's test-suite program should compare its solution for each problem with the optimal solution in the test-suite, and assign it a numerical grade. Then the grades of all the systems for each of the problems, rather than the raw results, should be combined into a single table.

>

>

>

> This was how I compiled the table of Charlwood Fifty test results for 7 systems that was posted in another sci.math.symbolic thread. Although the grading system used was relatively coarse (2 for an optimal antiderivative, 1 for a nonoptimal antiderivative, and 0 if unable to integrate), the table's bottom-line made it easy to compare the relative performance of the systems on this small test-suite. Obviously, it would be even more useful on a large test-suite with all the major systems tested...

>

>

>

> Albert

For some of the integration problems Axiom finds more than one result.

For instance,

t1:=sqrt(a+b*x)/x

+-------+

\|b x + a

(1) ----------

x

Type: Expression(Integer)

(2) -> integrate(t1,x)

(2)

+-+ +-------+

+-+ - 2\|a \|b x + a + b x + 2a +-------+

[\|a log(----------------------------) + 2\|b x + a ,

x

+-------+

+---+ \|b x + a +-------+

- 2\|- a atan(----------) + 2\|b x + a ]

+---+

\|- a

Type: Union(List(Expression(Integer)),...)

The posted suite only shows one answer.

How should a test suite handle this?

Tim