{ "info": { "author": "Valentin Thorey", "author_email": "v.thorey@gmail.com", "bugtrack_url": null, "classifiers": [ "Programming Language :: Python", "Programming Language :: Python :: 3" ], "description": "\n# benderopt\n\nbenderopt is a black box optimization library.\n\nFor asynchronous use, a web client using this library is available in open access at [bender.dreem.com](https://bender.dreem.com)\n\nThe algorithm implemented \"parzen_estimator\" is similar to TPE described in:\n[Bergstra, James S., et al. \u201cAlgorithms for hyper-parameter optimization.\u201d Advances in Neural Information Processing Systems.](https://www.lri.fr/~kegl/research/PDFs/BeBaBeKe11.pdf)\n\n# Demo\nHere is a comparison on 200 evaluations of a function we want to minimize. First a random estimator is used to select random evaluation point. Then the parzen_estimator implemented in benderopt is used to select evaluation points.\n\nThe function to minimize is the following: `cos(x) + cos(2 * x + 1) + cos(y)`.\n\nThe red point correspond to the location of the global minima between 0 and 2pi for x and y.\n\n

\n \n

\n\nThe code to generate the video can be found in `benchmark/benchmark_sinus2D`\n\nWe can observe on this example that the parzen estimator tends to explore more the local minimum than the random approach. This might lead to a better optimization given a fixed number of evaluations.\n\n# The goal\nIn Black box optimization, we have a function to optimize but cannot compute the gradient, and evaluation is expensive in term of time / ressource. So we want to find a good exploration-exploitation trade off to get the best hyperparameters in as few evaluations as possible.\nUse case are:\n- Optimization of a machine learning model (number of layers of a neural network, function of activation, etc.\n- Business optimization (marketing campain, a/b testing)\n- Large scale clinical studies\n\n# Code Minimal Example\n\nOne of the advantage of benderopt is that it uses JSON-like object representation making it easier for a user to define parameters to optimize. This also allows an easy to integratation with an asynchrounous system such as [bender.dreem.com](https://bender.dreem.com).\n\nHere is a minimal example.\n\n```\nfrom benderopt import minimize\nimport numpy as np\n\n# We want to minimize the sinus function between 0 and 2pi\ndef f(x):\n return np.sin(x)\n\n# We define the parameters we want to optimize:\noptimization_problem_parameters = [\n {\n \"name\": \"x\", \n \"category\": \"uniform\",\n \"search_space\": {\n \"low\": 0,\n \"high\": 2 * np.pi,\n }\n }\n]\n\n# We launch the optimization\nbest_sample = minimize(f, optimization_problem_parameters, number_of_evaluation=50)\n\nprint(best_sample[\"x\"], 3 * np.pi / 2)\n\n\n> 4.710390692396651 4.71238898038469\n```\n\n# Minimal Documentation:\n# Optimization Problem\n\nAn optimization problem contains:\n\n- A list of parameters (i.e. parameters with their search space)\n- A list of observation (i.e. values for each parameter of the list and a corresponding loss)\n\nWe use JSON representation for each of them e.g.\n```\noptimization_problem_data = {\n \"parameters\": [\n {\n \"name\": \"parameter_1\", \n \"category\": \"uniform\",\n \"search_space\": {\"low\": 0, \"high\": 2 * np.pi, \"step\": 0.1}\n },\n {\n \"name\": \"parameter_2\", \n \"category\": \"categorical\",\n \"search_space\": {\"values\": [\"a\", \"b\", \"c\"]}\n }\n ],\n \"observations\": [\n {\n \"sample\": {\"parameter_1\": 0.4, \"parameter_2\": \"a\"},\n \"loss\": 0.1\n },\n {\n \"sample\": {\"parameter_1\": 3.4, \"parameter_2\": \"a\"},\n \"loss\": 0.1\n },\n {\n \"sample\": {\"parameter_1\": 4.1, \"parameter_2\": \"c\"},\n \"loss\": 0.1\n },\n ]\n}\n\n```\n## Optimizer\nAn optimizer takes an optimization problem and suggest new_predictions.\nIn other words, an optimizer takes a list of parameters with their search space and a history of past evaluations to suggest a new one.\n\nUsing the `optimization_problem_data` from the previous example:\n```\nfrom benderopt.base import OptimizationProblem, Observation\nfrom benderopt.optimizer import optimizers\n\noptimization_problem = OptimizationProblem.from_json(optimization_problem_data)\noptimizer = optimizers[\"parzen_estimator\"](optimization_problem)\nsample = optimizer.suggest()\n\nprint(sample)\n\n> {\"parameter_1\": 3.9, \"parameter_2\": \"b\"}\n```\nOptimizers currently available are `random` and `parzen_estimator`.\n\nBenderopt allows to add a new optimizer really easily by inheriting an optimizer from `BaseOptimizer` class.\n\nYou can check `benderopt/optimizer/random.py` for a minimal example.\n\n## Minimize function\nMinimize function shown above in the `minimal example` section implementation is quite strateforward:\n```\noptimization_problem = OptimizationProblem.from_list(optimization_problem_parameters)\noptimizer = optimizers[\"parzen_estimator\"](optimization_problem)\nfor _ in range(number_of_evaluation):\n sample = optimizer.suggest()\n loss = f(**sample)\n observation = Observation.from_dict({\"loss\": loss, \"sample\": sample})\n optimization_problem.add_observation(observation)\n```\nThe optimization_problem's observation history list is extended with a new observations at each iteration. This allows the optimizer to take them into account for the next suggestion.\n\n## Uniform Parameter\n| parameter | type | default | comments | \n|-----------|-----------|---------|---------------------------------------------------------------------------------------------------------------------| \n| low | mandatory | - | lowest possible value: all values will be greater than or equal to low | \n| high | mandatory | - | highest value: all values will be stricly less than high | \n| step | optionnal | None | discretize the set of possible values: all values will follow 'value = low + k * step with k belonging to [0, K]' | \n\ne.g.\n```\n {\n \"name\": \"x\", \n \"category\": \"uniform\",\n \"search_space\": {\n \"low\": 0,\n \"high\": 2 * np.pi,\n # \"step\": np.pi / 8\n }\n }\n```\n\n## Log-Uniform Parameter\n| parameter | type | default | comments | \n|-----------|-----------|---------|---------------------------------------------------------------------------------------------------------------------| \n| low | mandatory | - | lowest possible value: all values will be greater than or equal to low | \n| high | mandatory | - | highest value: all values will be stricly less than high | \n| step | optionnal | None | \"discretize the set of possible values: all values will follow 'value = low + k * step with k belonging to [0, K]'\" | \n| base | optional | 10 | logarithmic base to use | \n\ne.g.\n```\n {\n \"name\": \"x\", \n \"category\": \"loguniform\",\n \"search_space\": {\n \"low\": 1e-4,\n \"high\": 1e-2,\n # \"step\": 1e-5,\n # \"base\": 10,\n }\n }\n```\n\n## Normal Parameter\n| parameter | type | default | comments | \n|-----------|-----------|---------|---------------------------------------------------------------------------------------------------------------------| \n| low | optionnal | -inf | lowest possible value: all values will be greater than or equal to low | \n| high | optionnal | inf | highest value: all values will be stricly less than high | \n| mu | mandatory | - | mean value: all values will be initially drawn following a gaussian centered at mu with sigma variance | \n| sigma | mandatory | - | sigma value: all values will be initially drawn following a gaussian centered at mu with sigma variance | \n| step | optionnal | None | \"discretize the set of possible values: all values will follow 'value = low + k * step with k belonging to [0, K]'\" | \n\n\ne.g.\n```\n {\n \"name\": \"x\", \n \"category\": \"normal\",\n \"search_space\": {\n # \"low\": 0,\n # \"high\": 10,\n \"mu\": 5,\n \"sigma\": 1\n # \"step\": 0.01,\n }\n }\n```\n## Log-Normal Parameter\n| parameter | type | default | comments | \n|-----------|-----------|---------|---------------------------------------------------------------------------------------------------------------------| \n| low | optionnal | -inf | lowest possible value: all values will be greater than or equal to low | \n| high | optionnal | inf | highest value: all values will be stricly less than high | \n| mu | mandatory | - | mean value: all values will be initially drawn following a gaussian centered at mu with sigma variance | \n| sigma | mandatory | - | sigma value: all values will be initially drawn following a gaussian centered at mu with sigma variance | \n| step | optionnal | None | \"discretize the set of possible values: all values will follow 'value = low + k * step with k belonging to [0, K]'\" | \n| base | optional | 10 | logarithmic base to use | \n\ne.g.\n```\n {\n \"name\": \"x\", \n \"category\": \"lognormal\",\n \"search_space\": {\n # \"low\": 1e-6,\n # \"high\": 1e0,\n \"mu\": 1e-3,\n \"sigma\": 1e-2\n # \"step\": 1e-7,\n # \"base\": 10,\n }\n }\n```\n## Categorical Parameter\n| parameter | type | default | comments | \n|---------------|-----------|-------------------------------------------|-------------------------------------------------------------------------------------------| \n| values | mandatory | - | list of categories: all values will be sampled from this list | \n| probabilities | optionnal | number_of_values * [1 / number_of_values] | list of probabilities: all values will be initially drawn following this probability list | \n\ne.g.\n```\n {\n \"name\": \"x\", \n \"category\": \"categorical\",\n \"search_space\": {\n \"values\": [\"a\", \"b\", \"c\", \"d\"],\n # \"probabilities\": [0.1, 0.2, 0.2, 0.2, 0.3]\n }\n }\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Dreem-Organization/benderopt", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "benderopt", "package_url": "https://pypi.org/project/benderopt/", "platform": "", "project_url": "https://pypi.org/project/benderopt/", "project_urls": { "Homepage": "https://github.com/Dreem-Organization/benderopt" }, "release_url": "https://pypi.org/project/benderopt/1.3.1/", "requires_dist": [ "numpy (>=1.15.4)", "scipy (>=1.1.0)" ], "requires_python": "", "summary": "Black Box optimization library.", "version": "1.3.1" }, "last_serial": 4673122, "releases": { "1.3.1": [ { "comment_text": "", "digests": { "md5": "dfaa218339f41b96f81844ddc399a043", "sha256": "0e7ee571755eb105d9ff5eb045ff2178631c7244574c691755eec7958ad63343" }, "downloads": -1, "filename": "benderopt-1.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "dfaa218339f41b96f81844ddc399a043", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 40702, "upload_time": "2019-01-08T14:19:02", "url": "https://files.pythonhosted.org/packages/72/cd/dee36fc04f32b650baf11437314dd82600f572dbf3a85833cd64a11892cc/benderopt-1.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "75e569db4748e64cd9b21a7e4728358d", "sha256": "88bb2a1da55621c1ad3f3a294b618a37d20bd956bbb9d6ae3710da7deb4ecd1f" }, "downloads": -1, "filename": "benderopt-1.3.1.tar.gz", "has_sig": false, "md5_digest": "75e569db4748e64cd9b21a7e4728358d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16389, "upload_time": "2019-01-08T14:19:04", "url": "https://files.pythonhosted.org/packages/c2/1d/c0204a753b94a6731932ae995a4bb97ebd37fa7ae562c505833cb1e224a2/benderopt-1.3.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "dfaa218339f41b96f81844ddc399a043", "sha256": "0e7ee571755eb105d9ff5eb045ff2178631c7244574c691755eec7958ad63343" }, "downloads": -1, "filename": "benderopt-1.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "dfaa218339f41b96f81844ddc399a043", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 40702, "upload_time": "2019-01-08T14:19:02", "url": "https://files.pythonhosted.org/packages/72/cd/dee36fc04f32b650baf11437314dd82600f572dbf3a85833cd64a11892cc/benderopt-1.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "75e569db4748e64cd9b21a7e4728358d", "sha256": "88bb2a1da55621c1ad3f3a294b618a37d20bd956bbb9d6ae3710da7deb4ecd1f" }, "downloads": -1, "filename": "benderopt-1.3.1.tar.gz", "has_sig": false, "md5_digest": "75e569db4748e64cd9b21a7e4728358d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16389, "upload_time": "2019-01-08T14:19:04", "url": "https://files.pythonhosted.org/packages/c2/1d/c0204a753b94a6731932ae995a4bb97ebd37fa7ae562c505833cb1e224a2/benderopt-1.3.1.tar.gz" } ] }