{ "info": { "author": "Caleb Everett", "author_email": "mail@calebeverett.io", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Scientific/Engineering :: Information Analysis" ], "description": "pysgd\n=====\n\nThe ``pysgd`` package contains a function that performs various\nstochastic gradient descent algorithms. The function accepts data, an\nobjective function, a gradient descent adaptation and algorithm\nhyperparameters as its arguments.\n\nThe intention of this package is to present reasonably efficient,\nworking algorithms that are easy to understand.\n\nThe code is structured with one main function in the main module,\n``sgd``, one class, ``Objective``, and modules for adaptations and\nobjective functions. ``sgd`` creates an ``Objective`` instance using the\nadaptation and objective function modules and any data specified. The\nobjective modules include both gradient and cost functions. The gradient\nfunctions accept ``theta`` and ``batch`` as arguments and the cost\nfunctions accept ``theta`` and ``data``. The adaptations are set up as\ndecorated generator functions that accept ``params`` and ``grad_fun`` as\narguments. ``params`` is a dict of hyperparameters and ``grad_fun`` is a\ngradient function. ``Objective`` creates a gradient generator and a cost\nfunction (injecting batch generator and data as required) that each\naccept ``theta`` as a single argument. These functions are then used in\nthe main gradient descent algorithm in ``sgd``.\n\nThe package is structured to facilitate the inclusion of additional\nalgorithms with minimal additional boilerplate. Additional objective\nfunctions and gradient adaptations can be added by following the basic\nform of the included ones and placing them in their respective folders.\n\n::\n\n pysgd\n | `-- __init__.py\n |-- adaptations\n | |-- __init__.py\n | |-- adagrad.py\n | |-- adam.py\n | `-- constant.py\n |-- objectives\n | |-- __init__.py\n | |-- linear.py\n | |-- logistic.py\n | `-- stab_tang.py\n `-- tests\n\nGradient Descent\n~~~~~~~~~~~~~~~~\n\nGradient descent is a method for minimizing an objective function. In\nmachine learning applications the objective function to be minimized is\nthe error (or cost), ``J``, of a predictive model. A predictive model\nconsists of a parameters, ``theta``, that are applied to inputs, ``X``,\n(also called training samples, features, observations or independent\nvariables) in order to estimate an output, ``y_hat`` (also called a\nlabel or dependent variable). Gradient descent attempts to determine the\nparameters that when applied to a set of inputs result in the lowest\ntotal error (the difference between the actual outcome and the one\npredicted by the model). Below is the basic predictive formula.\n\n``H(X,theta) = y_hat``\n\nAnd here is an illustrative formula for determining the total error of a\nmodel.\n\n``J(theta) = sum(|h_i(theta,x_i) - y_i| for each training observation, i)``\n\nDifferent formulas for computing cost are used depending on the\napplication, but the formula above expresses the essence of predicting\nactual outcomes as closely as possible.\n\nIn order to minimze ``J`` with respect to ``theta``, the algorithm\nstarts with an abitrary value of ``theta``, determines the \"direction\"\nthat would result in the fastest decrease in cost (called the\n``gradient``), updates ``theta`` in that direction by a small amount\n(called the learning rate or ``alpha``) and then repeats until cost\n``J`` has been minimized.\n\n``theta_(j+1) = theta_j - alpha * gradient_j``\n\nAPI\n~~~\n\nThe package has one main function, ``sgd``, that returns a ``j x (n+2)``\narray, where ``j`` is the number of iterations and ``n`` is the number\nof features. ``theta_j`` is in the first ``n+1`` columns and the cost\n``J_j`` in the last column.\n\n\\|Argument \\|Definition \\|\n\\|-------------------\\|----------------------------------------------------------------------------------------------\\|\n\\|\\ ``theta0`` \\|Starting value of ``theta`` in the form of an\n``1 x (n+1)`` array. \\| \\|\\ ``obj='stab_tang'`` \\|Objective function to\nbe minimized in the form of a string with a value of ``stab_tang``,\n``linear`` or ``logistic``. ``stab_tang`` is for the `Stablinsky-Tang\nfunction `__,\nincluded for testing and illustrative purposes. \\|\n\\|\\ ``adapt='constant'`` \\|Gradient descent adaptation in the form of a\nstring with a value of ``constant``, ``adagrad`` or ``adam``.\n\n.. raw:: html\n\n \n\n::\n\n |\n\n\\|\\ ``data=np.array.(closed brackets)``\\ \\|Data in the form of an\n``m x (n+1)`` array, including ``ones`` in the first column, if\nnecessary, where ``m`` is the number of training observations. \\|\n\\|\\ ``size=50`` \\|Batch size in the form of an integer between ``1`` and\n``m``. Batches are generated contiguously over the data until theta has\nconverged or all observations have been included in a batch, at which\npoint the data is shuffled before additional batches are used.\\|\n\\|\\ ``alpha=.01`` \\|Learning rate ``alpha`` in the form of a floating\npoint integer. \\| \\|\\ ``epsilon=10**-8`` \\|Hyperparameter used by\n``adagrad`` and ``adam`` for smoothing. \\| \\|\\ ``beta1=0.9``\n\\|Hyperparamter used by ``adam`` that controls the decay rates of the\nmoving gradient averages. \\| \\|\\ ``beta2=0.999`` \\|Hyperparamter used by\n``adam`` that controls the decay rates of the moving gradient averages.\n\\| \\|\\ ``delta_min=10**-6`` \\|Maximum change in all elements of\n``theta`` required to establish convergence, in the form of a floating\npoint integer.\\| \\|\\ ``iters=1000`` \\|Maximum number of batches to\nevaluate if convergence is not achieved in fewer iterations. \\|\n\nTests\n^^^^^\n\nTests are are performed with\n`pytest `__ and cover 100%\nof the code.\n\nIn addition to sample data sets, the `Stablinsky-Tang\nfunction `__\nis used for testing. This function is non-convex with straightforward\ngradient computations that makes it possible to compare the values\nproduced by the algorithms with known values. By using two dimensional\ninputs all of the possible values of ``J`` within a range of ``theta``\nvalues can be plotted as a surface. ``J(theta)`` for each iteration of\nthe algorithm can then be plotted on the surface in order to visualize\nthe gradient descent.\n\nThe color scale of the surfaces in the plots in the included notebook\ncorresponds to the value of ``J``. The color scale of the points on the\nsurface, which represent ``J_(theta_j)`` at each iteration, corresponds\nto the iteration of the algorithm.", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "machine-learning gradient-descent", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pysgd", "package_url": "https://pypi.org/project/pysgd/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/pysgd/", "project_urls": null, "release_url": "https://pypi.org/project/pysgd/0.0.1a2/", "requires_dist": [ "numpy", "py-cov; extra == 'test'", "pyplot; extra == 'test'", "pytest; extra == 'test'" ], "requires_python": "", "summary": "Stochastic gradient descent algorithms", "version": "0.0.1a2" }, "last_serial": 2556718, "releases": { "0.0.1a1": [ { "comment_text": "", "digests": { "md5": "9db85614fd1fd962665fa6a6186dd038", "sha256": "a3a1309d067f714a65fb93bb95d322148cf9eb15ae878580b2f88ea1865f0d83" }, "downloads": -1, "filename": "pysgd-0.0.1a1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9db85614fd1fd962665fa6a6186dd038", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 13488, "upload_time": "2017-01-04T11:13:40", "url": "https://files.pythonhosted.org/packages/b6/9a/ffd7ad50a659c126e343b8414a07d7771df29117383a72fbc76739a83df6/pysgd-0.0.1a1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a928f7687e3ac0e3165beeb93d478a63", "sha256": "c15f5aa30fdf13c9ef0f30a245be42b70f76f75128baa847faa68b86c4486a77" }, "downloads": -1, "filename": "pysgd-0.0.1a1.tar.gz", "has_sig": false, "md5_digest": "a928f7687e3ac0e3165beeb93d478a63", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6910, "upload_time": "2017-01-04T11:13:42", "url": "https://files.pythonhosted.org/packages/fa/c3/255f5262c0fecc4dec4c45fdbce509da4ccc823237ccfcea3acb792353eb/pysgd-0.0.1a1.tar.gz" } ], "0.0.1a2": [ { "comment_text": "", "digests": { "md5": "0dc4ed4d18597a69cbd8e8f2ec40cbcf", "sha256": "bfb72e1c81afc46c0a6d10fcb3001b377d73cfbce171e6d6d50b66fa6f0bf605" }, "downloads": -1, "filename": "pysgd-0.0.1a2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0dc4ed4d18597a69cbd8e8f2ec40cbcf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 13794, "upload_time": "2017-01-05T23:14:05", "url": "https://files.pythonhosted.org/packages/51/cd/36522ed8161f46ae6ea874d4c72eeb0119039f671f447a88c3d85341cf62/pysgd-0.0.1a2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7e2999d51060df4b088567bdbcfd1d1a", "sha256": "6a50e08400cf4a116875c5ecf946b3b24f4c6d0d38513467ee5327235c31cc2f" }, "downloads": -1, "filename": "pysgd-0.0.1a2.tar.gz", "has_sig": false, "md5_digest": "7e2999d51060df4b088567bdbcfd1d1a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7756, "upload_time": "2017-01-05T23:14:07", "url": "https://files.pythonhosted.org/packages/b6/ac/01fbba88babad0e45e9652ffa32d90f9934e36f662f172599736b5d04423/pysgd-0.0.1a2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "0dc4ed4d18597a69cbd8e8f2ec40cbcf", "sha256": "bfb72e1c81afc46c0a6d10fcb3001b377d73cfbce171e6d6d50b66fa6f0bf605" }, "downloads": -1, "filename": "pysgd-0.0.1a2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "0dc4ed4d18597a69cbd8e8f2ec40cbcf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 13794, "upload_time": "2017-01-05T23:14:05", "url": "https://files.pythonhosted.org/packages/51/cd/36522ed8161f46ae6ea874d4c72eeb0119039f671f447a88c3d85341cf62/pysgd-0.0.1a2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7e2999d51060df4b088567bdbcfd1d1a", "sha256": "6a50e08400cf4a116875c5ecf946b3b24f4c6d0d38513467ee5327235c31cc2f" }, "downloads": -1, "filename": "pysgd-0.0.1a2.tar.gz", "has_sig": false, "md5_digest": "7e2999d51060df4b088567bdbcfd1d1a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7756, "upload_time": "2017-01-05T23:14:07", "url": "https://files.pythonhosted.org/packages/b6/ac/01fbba88babad0e45e9652ffa32d90f9934e36f662f172599736b5d04423/pysgd-0.0.1a2.tar.gz" } ] }