Date: Jul 22, 2013 10:33 PM
Author: daly@axiom-developer.org
Subject: Re: An independent integration test suite
On Monday, July 22, 2013 9:28:14 PM UTC-4, Albert Rich wrote:

> On Monday, July 22, 2013 11:53:11 AM UTC-10, da...@axiom-developer.org wrote:

>

>

>

> > On Sunday, July 21, 2013 8:04:31 PM UTC-4, Albert Rich wrote:

>

>

>

> >> Sorry, I don't want to wade through megabytes of raw Axiom test results. What I, and I think others, want to know is what percentage of the problems did Axiom get the optimal antiderivative, what percentage did it get valid but non-optimal antiderivatives, and what percentage is it unable to integrate.

>

>

>

> > Perfectly reasonable. However, it takes a while to create and process

>

> > 40,000+ integrals. I'll make a second pass when I have created the test

>

> > suite to evaluate the results. So far it takes about 3-4 days to create

>

> > one of the test suites and you appear to have posted 80 files. The raw

>

> > numbers you seek probably won't exist until the end of the year at this

>

> > rate.

>

> >

>

> > Rather than the raw numbers I'm more interested in the classes of problems

>

> > that Axiom does not handle. My long term plan for this test suite is to

>

> > use your patterns to form these classes and then use your patterns to write

>

> > pattern-based integration routines to handle them.

>

>

>

> Glad to hear Rubi and its test-suite will be of use in your efforts to improve Axiom's symbolic integrator. Since you have apparently written a program to run the test-suite, seems like extending it to compare the results with the optimal antiderivatives and assign a grade should be relatively easy...

>

>

>

> Albert

On a per-file basis this information is readily available. Every integral

is a 5 step process where the 5th step compares the derivative of the

difference between your "optimal integral" and Axiom's answer. If the 5th

step is 0 then they have the same answer. If they differ by a non-constant

value then one of us has the wrong answer. This could be decided by taking

the derivative of the "optimal integral". If that is non-constant then the

supplied result is wrong.

There were a couple of examples where Rubi's answer did not differentiate

to a constant value. These are mentioned in a previous post.

To make life easier I'll update the abstract at the start of the file to

include these statistics (on a per-file basis).

Tim