\[ \int \frac {\coth ^{-1}(a x)}{\left (c+d x^2\right )^{5/2}} \, dx \]
Optimal antiderivative \[ \frac {x \,\mathrm {arccoth}\left (a x \right )}{3 c \left (d \,x^{2}+c \right )^{\frac {3}{2}}}-\frac {\left (3 a^{2} c +2 d \right ) \arctanh \left (\frac {a \sqrt {d \,x^{2}+c}}{\sqrt {a^{2} c +d}}\right )}{3 c^{2} \left (a^{2} c +d \right )^{\frac {3}{2}}}+\frac {a}{3 c \left (a^{2} c +d \right ) \sqrt {d \,x^{2}+c}}+\frac {2 x \,\mathrm {arccoth}\left (a x \right )}{3 c^{2} \sqrt {d \,x^{2}+c}} \]
command
integrate(arccoth(a*x)/(d*x^2+c)^(5/2),x, algorithm="giac")
Giac 1.9.0-11 via sagemath 9.6 output
\[ \frac {1}{3} \, a {\left (\frac {{\left (3 \, a^{2} c + 2 \, d\right )} \arctan \left (\frac {\sqrt {d x^{2} + c} a}{\sqrt {-a^{2} c - d}}\right )}{{\left (a^{2} c^{3} + c^{2} d\right )} \sqrt {-a^{2} c - d} a} + \frac {1}{{\left (a^{2} c^{2} + c d\right )} \sqrt {d x^{2} + c}}\right )} + \frac {x {\left (\frac {2 \, d x^{2}}{c^{2}} + \frac {3}{c}\right )} \log \left (-\frac {\frac {1}{a x} + 1}{\frac {1}{a x} - 1}\right )}{6 \, {\left (d x^{2} + c\right )}^{\frac {3}{2}}} \]
Giac 1.7.0 via sagemath 9.3 output
\[ \int \frac {\operatorname {arcoth}\left (a x\right )}{{\left (d x^{2} + c\right )}^{\frac {5}{2}}}\,{d x} \]________________________________________________________________________________________