Date: Jul 22, 2013 10:33 PM
Author: daly@axiom-developer.org
Subject: Re: An independent integration test suite

On Monday, July 22, 2013 9:28:14 PM UTC-4, Albert Rich wrote:
> On Monday, July 22, 2013 11:53:11 AM UTC-10, da...@axiom-developer.org wrote:
>
>
>

> > On Sunday, July 21, 2013 8:04:31 PM UTC-4, Albert Rich wrote:
>
>
>

> >> Sorry, I don't want to wade through megabytes of raw Axiom test results. What I, and I think others, want to know is what percentage of the problems did Axiom get the optimal antiderivative, what percentage did it get valid but non-optimal antiderivatives, and what percentage is it unable to integrate.
>
>
>

> > Perfectly reasonable. However, it takes a while to create and process
>
> > 40,000+ integrals. I'll make a second pass when I have created the test
>
> > suite to evaluate the results. So far it takes about 3-4 days to create
>
> > one of the test suites and you appear to have posted 80 files. The raw
>
> > numbers you seek probably won't exist until the end of the year at this
>
> > rate.
>
> >
>
> > Rather than the raw numbers I'm more interested in the classes of problems
>
> > that Axiom does not handle. My long term plan for this test suite is to
>
> > use your patterns to form these classes and then use your patterns to write
>
> > pattern-based integration routines to handle them.
>
>
>
> Glad to hear Rubi and its test-suite will be of use in your efforts to improve Axiom's symbolic integrator. Since you have apparently written a program to run the test-suite, seems like extending it to compare the results with the optimal antiderivatives and assign a grade should be relatively easy...
>
>
>
> Albert


On a per-file basis this information is readily available. Every integral
is a 5 step process where the 5th step compares the derivative of the
difference between your "optimal integral" and Axiom's answer. If the 5th
step is 0 then they have the same answer. If they differ by a non-constant
value then one of us has the wrong answer. This could be decided by taking
the derivative of the "optimal integral". If that is non-constant then the
supplied result is wrong.

There were a couple of examples where Rubi's answer did not differentiate
to a constant value. These are mentioned in a previous post.

To make life easier I'll update the abstract at the start of the file to
include these statistics (on a per-file basis).

Tim