## Chapter 1Introduction

This report gives the result of running the computer algebra independent integration problems. The listing of the problems are maintained by and can be downloaded from https://rulebasedintegration.org

The number of integrals in this report is [ 1059 ]. This is test number [ 185 ].

### 1.1 Listing of CAS systems tested

The following systems were tested at this time.

1. Mathematica 12.1 (64 bit) on windows 10.
2. Rubi 4.16.1 in Mathematica 12 on windows 10.
3. Maple 2020 (64 bit) on windows 10.
4. Maxima 5.43 on Linux. (via sagemath 8.9)
5. Fricas 1.3.6 on Linux (via sagemath 9.0)
6. Sympy 1.5 under Python 3.7.3 using Anaconda distribution.
7. Giac/Xcas 1.5 on Linux. (via sagemath 8.9)

Maxima, Fricas and Giac/Xcas were called from inside SageMath. This was done using SageMath integrate command by changing the name of the algorithm to use the diﬀerent CAS systems.

Sympy was called directly using Python.

### 1.2 Results

Important note: A number of problems in this test suite have no antiderivative in closed form. This means the antiderivative of these integrals can not be expressed in terms of elementary, special functions or Hypergeometric2F1 functions. RootSum and RootOf are not allowed.

If a CAS returns the above integral unevaluated within the time limit, then the result is counted as passed and assigned an A grade.

However, if CAS times out, then it is assigned an F grade even if the integral is not integrable, as this implies CAS could not determine that the integral is not integrable in the time limit.

If a CAS returns an antiderivative to such an integral, it is assigned an A grade automatically and this special result is listed in the introduction section of each individual test report to make it easy to identify as this can be important result to investigate.

The results given in in the table below reﬂects the above.

 System solved Failed Rubi % 100. ( 1059 ) % 0. ( 0 ) Mathematica % 99.24 ( 1051 ) % 0.76 ( 8 ) Maple % 88.39 ( 936 ) % 11.61 ( 123 ) Maxima % 69.88 ( 740 ) % 30.12 ( 319 ) Fricas % 91.41 ( 968 ) % 8.59 ( 91 ) Sympy % 24.83 ( 263 ) % 75.17 ( 796 ) Giac % 73.56 ( 779 ) % 26.44 ( 280 )

The table below gives additional break down of the grading of quality of the antiderivatives generated by each CAS. The grading is given using the letters A,B,C and F with A being the best quality. The grading is accomplished by comparing the antiderivative generated with the optimal antiderivatives included in the test suite. The following table describes the meaning of these grades.

 grade description A Integral was solved and antiderivative is optimal in quality and leaf size. B Integral was solved and antiderivative is optimal in quality but leaf size is larger than twice the optimal antiderivatives leaf size. C Integral was solved and antiderivative is non-optimal in quality. This can be due to one or more of the following reasons antiderivative contains a hypergeometric function and the optimal antiderivative does not. antiderivative contains a special function and the optimal antiderivative does not. antiderivative contains the imaginary unit and the optimal antiderivative does not. F Integral was not solved. Either the integral was returned unevaluated within the time limit, or it timed out, or CAS hanged or crashed or an exception was raised.

Grading is implemented for all CAS systems. Based on the above, the following table summarizes the grading for this test suite.

 System % A grade % B grade % C grade % F grade Rubi 100. 0. 0. 0. Mathematica 83.76 5.95 9.54 0.76 Maple 57.51 24.74 6.14 11.61 Maxima 42.87 27.01 0. 30.12 Fricas 29.56 55.62 6.23 8.59 Sympy 22.1 2.74 0. 75.17 Giac 48.63 24.55 0.38 26.44

The following is a Bar chart illustration of the data in the above table. The ﬁgure below compares the CAS systems for each grade level. ### 1.3 Performance

The table below summarizes the performance of each CAS system in terms of CPU time and leaf size of results.

 System Mean time (sec) Mean size Normalized mean Median size Normalized median Rubi 0.13 67.91 0.92 50. 1. Mathematica 2.48 186.14 2.23 52. 1. Maple 0.08 202.49 2.2 64.5 1.38 Maxima 1.18 120.83 2.64 82. 1.85 Fricas 2.07 1339.59 16.39 455. 9.4 Sympy 15.07 211.46 3.79 61. 1.77 Giac 1.07 117.85 2.11 77. 1.82

### 1.4 list of integrals that has no closed form antiderivative

{334, 339, 340, 341, 346, 347, 348, 353, 354, 355, 360, 361, 362, 367, 368, 369, 374, 375, 376, 381, 382, 383, 388, 389, 390, 395, 396, 397, 402, 403, 404, 409, 410, 411, 416, 417, 424, 429, 430, 431, 436, 437, 438, 443, 444, 445, 450, 451, 452, 457, 458, 459, 464, 465, 466, 471, 472, 473, 478, 479, 480, 485, 486, 487, 492, 493, 494, 499, 500, 501, 505, 506, 507, 512, 513, 514, 519, 520, 521, 526, 527, 870, 1014, 1015, 1016, 1017}

Rubi
{}
Mathematica
{}
Maple
{}
Maxima
{}
Fricas
{}
Sympy
{}
Giac
{}

### 1.6 list of integrals solved by CAS but failed veriﬁcation

The following are integrals solved by CAS but the veriﬁcation phase failed to verify the anti-derivative produced is correct. This does not mean necessarily that the anti-derivative is wrong, as additional methods of veriﬁcation might be needed, or more time is needed (3 minutes time limit was used). These integrals are listed here to make it easier to do further investigation to determine why it was not possible to verify the result produced.

Rubi
{}
Mathematica
{219, 337, 349, 364, 372, 393, 405, 406, 425, 433, 439, 446, 462, 474, 475, 488, 490, 503, 531, 547, 556, 590, 592, 594, 622, 761, 762, 763, 764, 765, 766, 767, 768, 769, 774, 775, 776, 777, 854, 911, 918, 930, 935, 943, 944, 966}
Maple
Veriﬁcation phase not implemented yet.
Maxima
Veriﬁcation phase not implemented yet.
Fricas
Veriﬁcation phase not implemented yet.
Sympy
Veriﬁcation phase not implemented yet.
Giac
Veriﬁcation phase not implemented yet.

### 1.7 Timing

The command AboluteTiming[] was used in Mathematica to obtain the elapsed time for each integrate call. In Maple, the command Usage was used as in the following example

cpu_time := Usage(assign ('result_of _int',int(expr,x)),output='realtime'

For all other CAS systems, the elapsed time to complete each integral was found by taking the diﬀerence between the time after the call has completed from the time before the call was made. This was done using Python’s time.time() call.

All elapsed times shown are in seconds. A time limit of 3 minutes was used for each integral. If the integrate command did not complete within this time limit, the integral was aborted and considered to have failed and assigned an F grade. The time used by failed integrals due to time out is not counted in the ﬁnal statistics.

### 1.8 Veriﬁcation

A veriﬁcation phase was applied on the result of integration for Rubi and Mathematica. Future version of this report will implement veriﬁcation for the other CAS systems. For the integrals whose result was not run through a veriﬁcation phase, it is assumed that the antiderivative produced was correct.

Veriﬁcation phase has 3 minutes time out. An integral whose result was not veriﬁed could still be correct. Further investigation is needed on those integrals which failed veriﬁcations. Such integrals are marked in the summary table below and also in each integral separate section so they are easy to identify and locate.

### 1.9 Important notes about some of the results

#### 1.9.1 Important note about Maxima results

Since these integrals are run in a batch mode, using an automated script, and by using sagemath (SageMath uses Maxima), then any integral where Maxima needs an interactive response from the user to answer a question during evaluation of the integral in order to complete the integration, will fail and is counted as failed.

The exception raised is ValueError. Therefore Maxima result below is lower than what could result if Maxima was run directly and each question Maxima asks was answered correctly.

The percentage of such failures were not counted for each test ﬁle, but for an example, for the Timofeev test ﬁle, there were about 30 such integrals out of total 705, or about 4 percent. This pecrentage can be higher or lower depending on the speciﬁc input test ﬁle.

Such integrals can be indentiﬁed by looking at the output of the integration in each section for Maxima. If the output was an exception ValueError then this is most likely due to this reason.

Maxima integrate was run using SageMath with the following settings set by default

'besselexpand : true'
'display2d : false'
'domain : complex'
'keepfloat : true'

SageMath loading of Maxima abs_integrate was found to cause some problem. So the following code was added to disable this eﬀect.

from sage.interfaces.maxima_lib import maxima_lib
maxima_lib.set('extra_definite_integration_methods', '[]')
maxima_lib.set('extra_integration_methods', '[]')

#### 1.9.2 Important note about FriCAS and Giac/XCAS results

There are Few integrals which failed due to SageMath not able to translate the result back to SageMath syntax and not because these CAS system were not able to do the integrations.

These will fail With error Exception raised: NotImplementedError

The number of such cases seems to be very small. About 1 or 2 percent of all integrals.

Hopefully the next version of SageMath will have complete translation of FriCAS and XCAS syntax and I will re-run all the tests again when this happens.

#### 1.9.3 Important note about ﬁnding leaf size of antiderivative

For Mathematica, Rubi and Maple, the buildin system function LeafSize is used to ﬁnd the leaf size of each antiderivative.

The other CAS systems (SageMath and Sympy) do not have special buildin function for this purpose at this time. Therefore the leaf size is determined as follows.

For Fricas, Giac and Maxima (all called via sagemath) the following code is used

#see https://stackoverflow.com/questions/25202346/how-to-obtain-leaf-count-expression-size-in-sage

def tree(expr):
if expr.operator() is None:
return expr
else:
return [expr.operator()]+map(tree, expr.operands())

try:
# 1.35 is a fudge factor since this estimate of leaf count is bit lower than
#what it should be compared to Mathematica's
leafCount = round(1.35*len(flatten(tree(anti))))
except Exception as ee:
leafCount =1

For Sympy, called directly from Python, the following code is used

try:
# 1.7 is a fudge factor since it is low side from actual leaf count
leafCount = round(1.7*count_ops(anti))

except Exception as ee:
leafCount =1

When these cas systems have a buildin function to ﬁnd the leaf size of expressions, it will be used instead, and these tests run again.

### 1.10 Design of the test system

The following diagram gives a high level view of the current test build system. 