
Re: An independent integration test suite
Posted:
Jul 15, 2013 3:36 PM


On Monday, July 15, 2013 3:07:38 AM UTC4, Albert Rich wrote: > On Sunday, July 14, 2013 12:58:04 AM UTC10, Nasser M. Abbasi wrote: > > > > > I think the way to do this is by automation. The way I was > > > doing it so far is not practical. It should instead by > > > done by writing a script to run through all the test cases > > > automatically, otherwise it will take me another 2 years to > > > finish. But the problem comes when one wants to combine > > > results of many CAS'es in one document. > > > > I agree completely that automating the testing of systems AND grading their results is essential for any testsuite of significant size. However, I do not think it necessary or practical to combine the raw test results of multiple systems into a single document. > > > > Instead a system's testsuite program should compare its solution for each problem with the optimal solution in the testsuite, and assign it a numerical grade. Then the grades of all the systems for each of the problems, rather than the raw results, should be combined into a single table. > > > > This was how I compiled the table of Charlwood Fifty test results for 7 systems that was posted in another sci.math.symbolic thread. Although the grading system used was relatively coarse (2 for an optimal antiderivative, 1 for a nonoptimal antiderivative, and 0 if unable to integrate), the table's bottomline made it easy to compare the relative performance of the systems on this small testsuite. Obviously, it would be even more useful on a large testsuite with all the major systems tested... > > > > Albert
As mentioned, the antiderivative given is of the form 1  99  99 x
whereas Axiom gets the equivalent result of
1  99 99x
Axiom builds a "type tower" for expressions. Fractions are of typ FRACTION(INTEGER). If a polynomial has fractional coefficients you get POLYNOMIAL(FRACTION(INTEGER)). This is the type of the Axiom expression above. However, the suggested antiderivative is the ratio of two polynomials with fractional coefficients leading to a type of FRACTION(POLYNOMIAL(FRACTION(INTEGER))) which is rather more complicated.
In order to ensure that the answers of the integration differ by no more than a constant I've been differencing the expected answer from the Axiom answer and then taking the derivative.
One curious pattern is that your answers differ from Axiom's answers by nonzero constants. I found that the same thing happens with Maple and Axiom. It appears that we're using different branch cuts caused by different trig rewrite formulas. Axiom and Mathematica tend to agree if memory serves me. I can't check because the Mathematica version I bought won't run due to system changes.
Perhaps there has to be a wider discussion of the choice of these simplification formulas. Is there some reason to choose one over the other?
Tim

