
Re: An independent integration test suite
Posted:
Jul 22, 2013 10:33 PM


On Monday, July 22, 2013 9:28:14 PM UTC4, Albert Rich wrote: > On Monday, July 22, 2013 11:53:11 AM UTC10, da...@axiomdeveloper.org wrote: > > > > > On Sunday, July 21, 2013 8:04:31 PM UTC4, Albert Rich wrote: > > > > >> Sorry, I don't want to wade through megabytes of raw Axiom test results. What I, and I think others, want to know is what percentage of the problems did Axiom get the optimal antiderivative, what percentage did it get valid but nonoptimal antiderivatives, and what percentage is it unable to integrate. > > > > > Perfectly reasonable. However, it takes a while to create and process > > > 40,000+ integrals. I'll make a second pass when I have created the test > > > suite to evaluate the results. So far it takes about 34 days to create > > > one of the test suites and you appear to have posted 80 files. The raw > > > numbers you seek probably won't exist until the end of the year at this > > > rate. > > > > > > Rather than the raw numbers I'm more interested in the classes of problems > > > that Axiom does not handle. My long term plan for this test suite is to > > > use your patterns to form these classes and then use your patterns to write > > > patternbased integration routines to handle them. > > > > Glad to hear Rubi and its testsuite will be of use in your efforts to improve Axiom's symbolic integrator. Since you have apparently written a program to run the testsuite, seems like extending it to compare the results with the optimal antiderivatives and assign a grade should be relatively easy... > > > > Albert
On a perfile basis this information is readily available. Every integral is a 5 step process where the 5th step compares the derivative of the difference between your "optimal integral" and Axiom's answer. If the 5th step is 0 then they have the same answer. If they differ by a nonconstant value then one of us has the wrong answer. This could be decided by taking the derivative of the "optimal integral". If that is nonconstant then the supplied result is wrong.
There were a couple of examples where Rubi's answer did not differentiate to a constant value. These are mentioned in a previous post.
To make life easier I'll update the abstract at the start of the file to include these statistics (on a perfile basis).
Tim

