Date: Jul 15, 2013 3:36 PM
Author: daly@axiom-developer.org
Subject: Re: An independent integration test suite

On Monday, July 15, 2013 3:07:38 AM UTC-4, Albert Rich wrote:
> On Sunday, July 14, 2013 12:58:04 AM UTC-10, Nasser M. Abbasi wrote:
>
>
>

> > I think the way to do this is by automation. The way I was
>
> > doing it so far is not practical. It should instead by
>
> > done by writing a script to run through all the test cases
>
> > automatically, otherwise it will take me another 2 years to
>
> > finish. But the problem comes when one wants to combine
>
> > results of many CAS'es in one document.
>
>
>
> I agree completely that automating the testing of systems AND grading their results is essential for any test-suite of significant size. However, I do not think it necessary or practical to combine the raw test results of multiple systems into a single document.
>
>
>
> Instead a system's test-suite program should compare its solution for each problem with the optimal solution in the test-suite, and assign it a numerical grade. Then the grades of all the systems for each of the problems, rather than the raw results, should be combined into a single table.
>
>
>
> This was how I compiled the table of Charlwood Fifty test results for 7 systems that was posted in another sci.math.symbolic thread. Although the grading system used was relatively coarse (2 for an optimal antiderivative, 1 for a nonoptimal antiderivative, and 0 if unable to integrate), the table's bottom-line made it easy to compare the relative performance of the systems on this small test-suite. Obviously, it would be even more useful on a large test-suite with all the major systems tested...
>
>
>
> Albert

As mentioned, the antiderivative given is of the form
1
--
99
----
99
x

whereas Axiom gets the equivalent result of

1
-----
99
99x

Axiom builds a "type tower" for expressions. Fractions are of typ
FRACTION(INTEGER). If a polynomial has fractional coefficients you get
POLYNOMIAL(FRACTION(INTEGER)). This is the type of the Axiom expression
above. However, the suggested antiderivative is the ratio of two
polynomials with fractional coefficients leading to a type of
FRACTION(POLYNOMIAL(FRACTION(INTEGER))) which is rather more complicated.

In order to ensure that the answers of the integration differ by no
more than a constant I've been differencing the expected answer from
the Axiom answer and then taking the derivative.