{
"info": {
"author": "Philippe Remy",
"author_email": "",
"bugtrack_url": null,
"classifiers": [],
"description": "# ARMA w. Scipy\nEstimating coefficients of ARMA models with the Scipy package.\n\nInstallation\n```bash\npip install arma-scipy\n```\n\nPython import\n```python\nfrom arma_scipy import fit\n```\n\n## Motivation\n\nARMA models in general can be, after choosing p and q, fitted by least\nsquares regression to find the values of the parameters which minimize\nthe error term. It is generally considered good practice to find the\nsmallest values of p and q which provide an acceptable fit to the data.\nFor a pure AR model, the Yule-Walker equations may be used to provide a\nfit. \"Least squares\" means that the overall solution minimizes the sum of the\nsquares of the residuals made in the results of every single equation.\n\nThe reasons behind this Scipy fit implementation are twofold:\n- provide an alternative when the score function is not the MSE - **more important**\n- provide a way to compare both fit methods (stat theory vs optimization) - less important\n\nYou can fit the coefficients of an `ARMA(4,4)` that way:\n```bash\npython generate_arma_process.py\npython scipy_fit_data.py\n```\n\nHere is an example of such a fit:\n```\n################################################################################\nOptimization terminated successfully.\n Current function value: 1.432208\n Iterations: 508\n Function evaluations: 788\nEstimation of the coefficients with the scipy package:\n[ 0.2235 -0.5872 0.3143 -0.1805 0.167 -0.0464 0.6528 0.224 ]\nEstimation of the coefficients with the statsmodels.tsa (least squares) package:\n[ 0.237 -0.4998 0.3467 -0.128 0.1542 -0.1467 0.6244 0.2245]\nTrue ARMA coefficients:\n[ 0.25 -0.5 0.35 -0.15 0.5 -0.4 0.78 0.32]\n```\n\n## Comparison\n\n- It is not a surprise that the score function is minimized by the fit of the `statsmodels` package. Indeed, the maximum likelihood estimation is guaranteed to yield the lowest mean squared error score on the train set.\n- The Scipy minimize function is doing a relatively good job at being close to this minimum. However, due to the randomness nature of this optimization and the crucial choice of x0 (initial values of the coefficients to optimize), several runs are necessary to guarantee to be close enough to this global minimum. It's clear enough that there is a strong variability across the runs. A significant proportion of runs do not get even closer to this minimum. On 200 runs, the average hovers ~10% above the minimum expected value, then starts to overfit. The best run, however, reaches 1.41807, a score extremely close to the target minimum score of 1.4179.\n\n
\n \n
\n\nThe conclusion of this comparison is such that there it makes little sense to fit the coefficients of an ARMA model with a numerical method like `scipy.minimize`. A traditional fit using `statsmodels` is preferable. Yet, `scipy.minimize` could be used if the score function was not the mean squared error. Finding a good x0 is challenging and one way to have a smooth optimization could lie in the estimation of x0 by `statsmodels`. In other words, fit the coefficients with `statsmodels` then optimize with `scipy.minimize` and your custom score function.\n\n## Custom score function\n\nLets consider the custom score function that assigns:\n- 1 if the prediction has the same sign as the true value.\n- 0 otherwise.\n\n```python\ndef score_function(p, t):\n return 1 - np.mean(((p > 0) & (t > 0)) | ((p < 0) & (t < 0)))\n```\n\n- The optimization on the ARMA coefficients is done first with the `statsmodels`. This is our `x0`.\n- The second optimization is done with `scipy` and the custome score function.\n\nThe second fit seems to add some value as the overall score decreases from 0.2090 to 0.2084. This represents a reduction of ~0.29% of the custom score. Concretely, after the first fit, the model correctly predicts the sign of the true value 79.10% of the time. After the second fit, this percentage goes up to 79.16%, thanks to Scipy.\n\n
![](\"misc/arma_44_fit.png\")
\n![](\"misc/custom_score_function.png\")
\n