PDF (letter size)
PDF (legal size)

Computer Algebra Independent Integration Tests

Nasser M. Abbasi

June 14, 2018 compiled on — Thursday June 14, 2018 at 06:54 PM [public]

These reports and the web pages themselves were written in LATEX using TeXLive distribution and compiled to HTML using TeX4ht.

FULL CAS integration tests

These reports show the result of using full CAS test suite on different CAS’s at same time. Only Maple, Mathematica and Rubi are used. This compares how one CAS performes against another.

The test suite used is the full CAS intergation tests. This table gives the links to the tests and percentage solved by each CAS.

Test # integrals Rubi Mathematica Maple date

Rubi 4.15.2, Mathematica 11.3, Maple 2018 In progress In progress In progress In progress Started June, 6, 2018

Rubi 4.14.8, Mathematica 11.3, Maple 2018 70,530 99.9% 96.77% 83.61% April 4, 2018

LITE CAS integration tests

These reports show the result of using same test suite on different versions of same CAS. Only Maple, Mathematica and Rubi are used. This is for regression testing to find how different version of the same CAS improved over the years. The test suite used is the lite version of the full CAS intergation tests. It contains smaller number of integrals compared to the full test suite used above.

  1. Mathematica This zip file contains the raw input tests used which was obtained from Rubi web site.
  2. Rubi Uses same zip file as above for input.
  3. Maple This zip file contains the raw input tests used which was obtained from Rubi web site.

Special builds

These are special/custom builds using other CAS software.

  1. Rubi 4.14.5, Mathematica 11.2, Maple 2017.3, FriCAS 1.3.2 [1846 integrals] Completed January 2018. Contains only the first 12 test files. Proof of concept build to try FriCAS.
  2. Due to slow computation time with sympy and to a lesser extent mupad, I will not be including sympy and mupad for the time being in future builds of the test cases. The build below includes sympy and mupad for the first 16 test cases.

    Rubi 4.11, Mathematica 11.1, Maple 2017, Mupad 7.0, Sympy 1.0 [9615 integrals. Done]

Note on build system

The following diagram gives a high level view of the current test build system.


The following are the steps to run one test suite

  1. open Maple and run convert_maple_to_sympy.mw This translates all Maple input files to sympy syntax. This needs to be done once only. This coverts all Maple (177) test files to sympy syntax.
  2. load Rubi package
  3. open main.nb and definitions.nb. Evaluate definitions.nb. Then in main.nb, increment counter to run the test number (under RUN THE TESTS). This will run both Mathematica and Rubi test and grade and create the result tables in the test folder.
  4. In main.nb, increment counter for Rubi rules and run. This will create Rubi rules used for each integral.
  5. Now run Maple. open master_file_maple.mw and adjust the counter to same test number and evaluate. This will create Maple result table and will also call grading function to grade the result. This uses GradeAntiderivative.mpl
  6. remove all comments from Maple input files before the next step, since mupad does not like Maple comments. Use the command

    grep -v '^ *#'   file_name.txt > new_file.txt
    mv new_file.txt file_name.txt
  7. Now open Matlab and load mupad.mn and increment the counter to same test number and run it. This will create mupad result table.
  8. Open Maple again and load grade_mupad.mw and adjust the counter to the test number. Before running this, make sure to backup mupad result table file. Now run it. This will read mupad result table and grade it.
  9. Edit sympy_main.py which is the Python driver to run the sympy tests and adjust the counter, then type (on Linux only) python sympy_main.py This will create sympy result table. Wait few days for this to complete.
  10. Now go back to main.nb and run command makeReportLatexDriver[n] where n is the test number. This will create the test report Latex document. Then run makeIndexLatexReportDriver[m] command, where m is the total completed tests so far. This will build the main index latex document.
  11. Now compile everything on Linux using the command CAS_integration_tests>make -I$HOME all
  12. after many hrs it should be done.
  13. When build is done, upload to site.

My PC during running the tests

I really need a faster PC with much more RAM !


This below shows example of CAS suddenly consuming all RAM in PC, and I had to terminate the process, since it did not time out as instructed, and just hanged.


my cheat sheet notes


To use Fricas online, go to http://axiom-wiki.newsynthesis.org/FriCASIntegration Go to bottom of page, type


Then click the preview button

to send email to supprt group use fricas-devel@googlegroups.com

To download go to https://sourceforge.net/projects/fricas

To install, see http://fricas.sourceforge.net/doc/INSTALL.txt

To build

#install sbcl
#sudo apt-get install sbcl #do not use this. bug
sudo apt-get install clisp #This lisp works
sudo apt-get install dvipng
sudo apt-get install auctex
bzip2 -dk fricas-1.3.2-full.tar.bz2
tar -xf fricas-1.3.2-full.tar

Now build it

make clean
fricas-1.3.2>./configure --with-lisp=/usr/bin/clisp
checking build system type... x86_64-linux-gnu
checking host system type... x86_64-linux-gnu
checking target system type... x86_64-linux-gnu
checking for make... make
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a BSD-compatible install... /usr/bin/install -c
checking for touch... touch
checking for mktemp... mktemp
checking for gawk... gawk
checking for gtar... no
checking for tar... tar
checking for ranlib... ranlib
checking for ar... ar
checking for latex... /usr/local/texlive/2017/bin/x86_64-linux/latex
checking for makeindex... makeindex
checking PREGENERATED... "/home/me/data/fricas/fricas-1.3.2/pre-generated"
checking for sbcl... /usr/bin/sbcl
checking Lisp implementation... This is SBCL 1.3.1.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* sbcl
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking dirent.h usability... yes
checking dirent.h presence... yes
checking for dirent.h... yes
checking whether closedir is declared... yes
checking whether opendir is declared... yes
checking whether readdir is declared... yes
checking whether dirfd is declared... yes
checking whether fchdir is declared... yes
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking whether sigaction is declared... yes
checking for sys/stat.h... (cached) yes
checking for unistd.h... (cached) yes
checking whether getuid is declared... yes
checking whether geteuid is declared... yes
checking whether getgid is declared... yes
checking whether getegid is declared... yes
checking whether kill is declared... yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking util.h usability... no
checking util.h presence... no
checking for util.h... no
checking pty.h usability... yes
checking pty.h presence... yes
checking for pty.h... yes
checking whether openpty is declared... yes
checking for openpty in -lc... no
checking for openpty in -lutil... yes
checking sys/wait.h usability... yes
checking sys/wait.h presence... yes
checking for sys/wait.h... yes
checking whether wait is declared... yes
checking whether fork is declared... yes
checking for X... no
configure: The Graphics and HyperDoc components are disabled.
configure: WARNING: Aldor interface will not be built.
configure: creating ./config.status
config.status: creating src/clef/Makefile
config.status: creating src/sman/Makefile
config.status: creating src/hyper/Makefile
config.status: creating src/doc/Makefile
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating src/lib/Makefile
config.status: creating src/lisp/Makefile
config.status: creating src/boot/Makefile
config.status: creating src/interp/Makefile
config.status: creating src/algebra/Makefile
config.status: creating src/input/Makefile
config.status: creating src/etc/Makefile
config.status: creating src/aldor/Makefile
config.status: creating src/aldor/Makefile2
config.status: creating src/aldor/Makefile3
config.status: creating contrib/emacs/Makefile
config.status: creating config/fricas_c_macros.h
extracting list of SPAD type definitions
Type 'make' (without quotes) to build FriCAS



Add these to .bashrc

export PATH=/home/me/data/fricas/fricas-1.3.2/target/x86_64-linux-gnu/bin:$PATH
export AXIOM=/home/me/data/fricas/fricas-1.3.2/target/x86_64-linux-gnu
export DAASE=/home/me/data/fricas/fricas-1.3.2/target/x86_64-linux-gnu

Example of integral which fails in Fricas

For Fricas, use these commands to get 1D plain text output

Otherwise the output will go to console in 2D. To get Latex output do

And this will give

                                         (√ -------)2
                                   arcsin----− x2-+-1--
(19)                             −         2

To record console session to file, use (from https://github.com/daly/axiom/blob/master/faq)

Like this

To send Latex output to file do

To turn off, just do )set output tex off. Make sure to tex on first. To send back to console, do )set output tex console

To read commands from file do

To read an input file into Axiom, use the )read system command. read For example, you can read a file in a particular directory by issuing

)read /spad/src/input/matrix.input

To make record

Some hints below thanks to Waldek Hebisch http://mathforum.org/kb/message.jspa?messageID=9791385

copied here so easy for me to get to:

1. How to measure CPU time used from the int call?
(3) -> )set messages time on
(3) -> integrate(x^(3/2)/sqrt(1 + x^5), x)

Type: Union(Expression(Integer),...)
Time: 0.06 (EV) + 0.00 (OT) = 0.06 sec

That is command ')set messages time on' tell FriCAS to print
time needed to execute following command. The (EV) part means
actual computation (OT) printing and at the end there is total

2. How to measure leaf count/size of result? (Maple and Mathematica have
build in function to do this)

That is a bit tricky. FriCAS (like Axiom) has quite different
representation of expressions tham Maple or Mathematica.
In FriCAS expression is a quotient of two polnomials, with
variables beeing kernels:

(13) -> f := exp(x^3 + 1/x) + 1
(14) -> numer(f)

x + 1

(15) -> denom(f)

(16) -> variables(numer(f))

x + 1
(16) [%e ]
Type: List(Kernel(Expression(Integer)))
Time: 0.00 (OT) = 0.00 sec

From FriCAS point of view natural measure of size is number
of monomials in numerator and denominator:

(17) -> numberOfMonomials(numer(f))

(17) 2
(18) -> numberOfMonomials(denom(f))
(18) 1

but this may underestimate size because kernels may be big.
Also internally given kernel is stored only once, but in
printed output it may appear several times.

It is possible to traverse expression is somewhat tree like
manner, so it is possible to give better approximation
to say Maple result, but that would require investigating
what Maple is doing. I personally never used Maple node
count so I do not know it this is simple or if there are
some traps.

3. How to check if int passed or failed? Aborted? nil result? etc..

integrate may produce an error (in particular it will do so
if it can not decide if integral is elementary). Or may
produce unevaluated integral. Or normal result. At programistic
level it is possible to catch errors, but a bit tricky,
If you look at printed output, than errors are reasonably
easy to match via a regex:

(22) -> integrate(1/sqrt(exp(x) - x + 1), x)
integrate: implementation incomplete (constant residues)

the '>> Error' part indicates error. Unevaluated integrals
always have integral sign at top level. FriCAS never returns
partial results, so text match is relatively easy. Or you
may use code like:

test_int(f) ==
res : Union(Expression Integer, LE)
res := integrate(f, 'x)
res0 : List Expression Integer :=
res case Expression Integer =>
[res::(Expression Integer)]@(List Expression Integer)
res case List Expression Integer =>
res::(List Expression Integer)
error "test_int: impossible 1"
for ri in res0 repeat
#(kernels ri) > 0 and is?(operator first kernels ri, 'integral) =>
print("Unevaluated integral"::OutputForm)

Note1: this is part extracted from bigger test script, I did not
test it alone.

Note2: FriCAS may either return list of results or a single result.
Middle part converts this to have always a list (typically of
length 1). The last part is a loop so that all solutions can
be examined. It you only want to know if result is evaluated, than
loop is not needed: more than one result means that integral
is evaluated.

Note3: In FriCAS testsuite I use a bit different test for
evaluated integrals. Namely, FriCAS can do useful computations
on unevaluated integrals and in particular unevaluated integrals
may appear in the argument to integrate. Such unevaluated
integrals of course may propagate form input to the output.
The test above may misclasify such integral as unevaluated,
while better test checks that unevaluated integral came from
the input.

4. And most importantly, how to export the result (if it passed) to a
plain text file in _Latex_ format?

(24) -> )set output tex on
(24) -> integrate(exp(x)*exp(1/(exp(x)+1)-x), x)

Here one have to decide what to do with errors. Without error
trapping error will abort currently executing function
and propagate to top level (up to command line).

After doing:

)set break resume

after error FriCAS will continue executing file from next
command (but still abort current command). If you need
real loop then I can provide error catcher, but it is
a bit more complicated than snippets above.
Waldek Hebisch

To get type of value in fricas do  typeOf(r)

To clear everything do

)clear all

)clear completely




To install python package do conda install package-name to update do conda   update package for example conda   update spyder

To integrate the command is

from sympy import *


import sympy  #but now have to add sympy. to each call

import os
import math
init_printing(use_unicode=False, wrap_line=False, no_global=True)
x = symbols('x', real=True)
text_file = open("python.txt", "w")
text_file.write("\"%s\"" % r0)

To check if integral fails?

r=integrate(f*g, (x, L/2, L))
type(r) is integrals.Integral

Out[41]: True


isinstance(r, integrals.Integral)
Out[48]: True

type(r) is integrals.Integral
Out[43]: False

if isinstance(r, integrals.Integral):
    result = 0

To get cwd


To get list of folder in cwd do





f = open('Hebisch_Problems.txt','r')

how to do timeout? simple timeouts using signal.alarm module check http://stackoverflow.com/questions/492519/timeout-on-a-function-call for some code. There is also talk about it here https://groups.google.com/forum/#!topic/sympy/qsPImy6WqcI code from above is

def _timeout(self, function, timeout):
         def callback(x, y):
             raise Skipped("Timeout")
         signal.signal(signal.SIGALRM, callback)
         signal.alarm(timeout)  # Set an alarm with a given timeout
         signal.alarm(0)  # Disable the alarm

elementry functions http://docs.sympy.org/latest/modules/functions/elementary.html and http://docs.sympy.org/dev/modules/functions/special.html

and http://docs.sympy.org/latest/modules/functions/index.html

Need to change

AppellF1 -->             #hypergeometric function of two variables
arctanh -->atanh
arctan --> atan
Pi --> pi.
arcsin -> asin
arccos -> acos
hypergeom -> hyper,
arccoth -> acoth
GAMMA -> uppergamma()
arccsc -> acsc
arcsec -> asec
arccot -> acot,
EllipticF -> elliptic_f
EllipticE -> elliptic_e
Li -> Li
Si -> Si
Ci -> Ci
Ei -> Ei
FresnelS -> fresnels
FresnelC -> fresnelc

I also removed 4th field that contains integrate(...) in it as it hangs the reading of the input file.

To run python tests do

cd /media/data/public_html/my_notes/CAS_integration_tests/reports/rubi_4_11
python sympy_main.py

To add new test, or run more tests, edit sympy_main.py and change the line

for n in range(0,1):  #change the last number to the number of tests to do

To update conda and python do

conda update conda
conda update python

To update sympy do

pip list | grep sympy
pip install --upgrade sympy

To find version, from inside python, type

import sympy

from github https://github.com/sympy/sympy/wiki/SymPy-vs.-Sage

To obtain a Rational in SymPy, one of these methods must be used:

>>> from sympy import Rational
>>> Rational(2, 7)

In python/sympy, to loop using index 1, can use this:

>>> for num,val in enumerate(sol,start=1):
...   print("root number {} is {}".format(num,val))

For timeout in recent python, on windows, this seems to work

from subprocess import STDOUT, check_output
output = check_output('dir', shell=True, stderr=STDOUT, timeout=5)

To download development sympy

sudo git clone git://github.com/sympy/sympy.git
sudo git clone https://github.com/HPAC/matchpy.git

To install using conda sudo conda install multiset

packages are here /home/me/anaconda3/lib/python3.6/site-packages