{ "info": { "author": "Hao Wang and Bas van Stein", "author_email": "wangronin@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering :: Artificial Intelligence" ], "description": "

\n

\n\n                 \n![Python](https://img.shields.io/pypi/pyversions/mipego.svg)\n![Python](https://img.shields.io/pypi/status/mipego.svg)\n![Dependencies](https://img.shields.io/badge/dependencies-up%20to%20date-brightgreen.svg)\n[![GitHub Issues](https://img.shields.io/github/issues/wangronin/MIP-EGO.svg)](https://github.com/wangronin/MIP-EGO/issues)\n![Contributions welcome](https://img.shields.io/badge/contributions-welcome-orange.svg)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n\n## Overview\n\n**MiP-EGO** *(Mixed integer, Parallel - Efficient Global Optimization)* is an optimization package that can be used to optimize Mixed integer optimization problems. A mixed-integer problem is one where some of the decision variables are constrained to be integer values or categorical values. \nNext to the classical mixed integer problems, Algorithm selection or algorithm parameter optimization can also be seen as a complex mixed-integer problem.\n\nThe advantage of MiP-EGO is that it uses a surrogate model (the EGO part) to learn from the evaluations it has made so far. Instead of *Gaussian Process Regression* like in standard *EGO*, the MiP-EGO uses Random Forests instead, since Random Forests can handle mixed integer data by default. \nThe P in MiP-EGO stands for parallel, as this implementation has the additional feature that it can evaluate several solutions in parallel, which is extremely handy when an evaluation takes a long time and several machines are available.\n\nFor example, one use case would be to optimize an expensive (in time) simulation. There are four simulation licenses, so four simulations can be run at the same time. With MiP-EGO, all these four licenses can be fully utilized, speeding up the optimization procedure. Using a novel infill-criteria, the Moment Generating Function Based criterium, multiple points can be selected as candidate solutions. See the following paper for more detail about this criterium: \nWANG, Hao, et al. *A new acquisition function for Bayesian optimization based on the moment-generating function.* In: Systems, Man, and Cybernetics (SMC), 2017 IEEE International Conference on. IEEE, 2017. p. 507-512.\n\n\n#### Async Parallel Optimization of Neural Network Architectures\n\nMiP-EGO also supports asynchronous parallel optimization, currently this feature is in *Beta* and being used to optimize the architecture and parameters of deep neural networks. See Example 2 for more details.\n\n\n## Install\n```python\npip install mipego\n```\n\n## Usage\n\nTo use the optimizer you need to define an objective function, the search space and configure the optimizer. Below are two examples that describe most of the functionality.\n\n### Example - Optimizing A Black-Box Function\nIn this example we optimize a mixed integer black box problem.\n\n```python\nimport os\nimport numpy as np\n\n#import our package, the surrogate model and the search space classes\nfrom mipego import mipego\nfrom mipego.surrogate import RandomForest\nfrom mipego.SearchSpace import ContinuousSpace, NominalSpace, OrdinalSpace\n\n# The \"black-box\" objective function\ndef obj_func(x):\n x_r, x_i, x_d = np.array([x['C_0'],x['C_1']]), x['I'], x['N']\n if x_d == 'OK':\n tmp = 0\n else:\n tmp = 1\n return np.sum(x_r ** 2.) + abs(x_i - 10) / 123. + tmp * 2.\n\n\n# First we need to define the Search Space\n# the search space consists of two continues variable\n# one ordinal (integer) variable\n# and one categorical.\nC = ContinuousSpace([-5, 5],'C') * 2 \n#here we defined two variables at once using the same lower and upper bounds.\n#One with label C_0, and the other with label C_1\nI = OrdinalSpace([-100, 100],'I') # one integer variable with label I\nN = NominalSpace(['OK', 'A', 'B', 'C', 'D', 'E'], 'N')\n\n#the search space is simply the product of the above variables\nsearch_space = C * I * N\n\n#next we define the surrogate model and the optimizer.\nmodel = RandomForest(levels=search_space.levels)\nopt = mipego(search_space, obj_func, model, \n minimize=True, #the problem is a minimization problem.\n max_eval=500, #we evaluate maximum 500 times\n max_iter=500, #we have max 500 iterations\n infill='EI', #Expected improvement as criteria\n n_init_sample=10, #We start with 10 initial samples\n n_point=1, #We evaluate every iteration 1 time\n n_job=1, # with 1 process (job).\n optimizer='MIES', #We use the MIES internal optimizer.\n verbose=False, random_seed=None)\n\n\n#and we run the optimization.\nincumbent, stop_dict = opt.run()\n```\n\n\n\n### Example 2 - Optimizing A Neural Network\nIn this example we optimize a neural network architecture on the MNIST dataset.\nThe objective function in this case is [this file](https://github.com/wangronin/BayesianOptimization/blob/master/all-cnn.py) from the root repository directory. \nIn the objective file the neural network architecture is defined and evaluated on the MNIST dataset. \nThe code below shows how to set up the optimizer for this purpose using 4 GPUs asynchronously. \n\n```python\nimport os\nimport numpy as np\nimport subprocess, sys\nfrom subprocess import STDOUT, check_output\n\n#import our package, the surrogate model and the search space classes\nfrom mipego import mipego\nfrom mipego.Surrogate import RandomForest\nfrom mipego.SearchSpace import ContinuousSpace, NominalSpace, OrdinalSpace\n\n#some help packages\nimport re\nimport traceback\nimport time\n\n#first lets define our objective function, \n#this is basically calling the file (all-cnn.py) and processes its output.\nclass obj_func(object):\n def __init__(self, program):\n self.program = program\n\n def __call__(self, cfg, gpu_no):\n print(\"calling program with gpu \"+str(gpu_no))\n cmd = ['python3', self.program, '--cfg', str(cfg), str(gpu_no)]\n outs = \"\"\n outputval = 0\n try:\n #we use a timeout to cancel very long evaluations.\n outs = str(check_output(cmd,stderr=STDOUT, timeout=40000)) \n outs = outs.split(\"\\\\n\")\n\n outputval = 0\n for i in range(len(outs)-1,1,-1):\n if re.match(\"^\\d+?\\.\\d+?$\", outs[-i]) is not None:\n print(outs[-i])\n outputval = -1 * float(outs[-i])\n if np.isnan(outputval):\n outputval = 0 #default to 0.\n except subprocess.CalledProcessError as e:\n #exception handling\n traceback.print_exc()\n print (e.output)\n except:\n print (\"Unexpected error:\")\n traceback.print_exc()\n outputval = 0\n return outputval\n\n\n\nobjective = obj_func('./all-cnn.py')\nactivation_fun = [\"softmax\",\"sigmoid\"] #activation function of the last layer.\nactivation_fun_conv = [\"elu\",\"relu\",\"tanh\",\"sigmoid\",\"selu\"]\n\n#Next we define the search space.\nfilters = OrdinalSpace([10, 600], 'filters') * 7\nkernel_size = OrdinalSpace([1, 6], 'k') * 7\nstrides = OrdinalSpace([1, 5], 's') * 3\nstack_sizes = OrdinalSpace([1, 5], 'stack') * 3\nactivation = NominalSpace(activation_fun_conv, \"activation\") \nactivation_dense = NominalSpace(activation_fun, \"activ_dense\") \n\n# to use step decay or not\nstep = NominalSpace([True, False], \"step\") \n#to use global pooling in the end or not.\nglobal_pooling = NominalSpace([True, False], \"global_pooling\")\n\ndrop_out = ContinuousSpace([1e-5, .9], 'dropout') * 4 \nlr_rate = ContinuousSpace([1e-4, 1.0e-0], 'lr') #learning rate\nl2_regularizer = ContinuousSpace([1e-5, 1e-2], 'l2') # l2_regularizer\n\nsearch_space = stack_sizes * strides * filters * kernel_size * activation * activation_dense * drop_out * lr_rate * l2_regularizer * step * global_pooling \n\n#We will use the first 4 GPU's of the system.\navailable_gpus = [0,1,2,3] \n\n# use random forest as the surrogate model \nmodel = RandomForest(levels=search_space.levels)\n\n#now define the optimizer.\nopt = mipego(search_space, objective, model, \n minimize=True, max_eval=None, max_iter=500, \n infill='MGFI', n_init_sample=10, \n n_point=4, n_job=4, \n #4 GPU's, all evaluating 1 point at a time.\n wait_iter=3, optimizer='MIES', \n verbose=False, random_seed=None,\n available_gpus=available_gpus)\n\n\n#run\nincumbent, stop_dict = opt.run()\n```\n\n\n\n## Contributing\nPlease take a look at our [contributing](https://github.com/wangronin/BayesianOptimization/blob/master/CONTRIBUTING.md) guidelines if you're interested in helping!\n\n#### Beta Features\n- Async GPU execution\n- Intermediate files to support restarts / resumes.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/wangronin/MIP-EGO", "keywords": "bayesian optimization ego mixed-integer", "license": "", "maintainer": "", "maintainer_email": "", "name": "mipego", "package_url": "https://pypi.org/project/mipego/", "platform": "", "project_url": "https://pypi.org/project/mipego/", "project_urls": { "Homepage": "https://github.com/wangronin/MIP-EGO" }, "release_url": "https://pypi.org/project/mipego/1.0.2/", "requires_dist": [ "pandas", "numpy", "scipy", "scikit-learn", "joblib", "dill" ], "requires_python": "", "summary": "Mixed Integer Parallel - Efficient Global Optimization with GPU support", "version": "1.0.2" }, "last_serial": 3847705, "releases": { "0.9.8": [ { "comment_text": "", "digests": { "md5": "17ae000196574fa80886bb059e07d787", "sha256": "813bcf5b28eead4db7b42380776cfe6bbcfeb40f1421cb6fa0553fc4502dd62c" }, "downloads": -1, "filename": "mipego-0.9.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "17ae000196574fa80886bb059e07d787", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12210, "upload_time": "2018-05-08T12:08:44", "url": "https://files.pythonhosted.org/packages/fa/af/2d73f7588a289af8c7432766c475506e414dee1f1bbb9b9ecbe6b2e9c61b/mipego-0.9.8-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9878e32a271e8fa309fdd63e31c7ca85", "sha256": "2127f9d8ef84e761880193f35d5b3cc70c34f06a3e13a53a1755f774a5dea641" }, "downloads": -1, "filename": "mipego-0.9.8.tar.gz", "has_sig": false, "md5_digest": "9878e32a271e8fa309fdd63e31c7ca85", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13438, "upload_time": "2018-05-08T12:08:45", "url": "https://files.pythonhosted.org/packages/b8/0c/ef3bb069d99ff6bf666a2f3d35ce3fa8ad2a791afa5788be071a83ff289e/mipego-0.9.8.tar.gz" } ], "0.9.9": [ { "comment_text": "", "digests": { "md5": "66206a2939e4275fc4ce1888e863b4a2", "sha256": "9c450cd6d9f18caf82bdbe658bb40e7a4e9836ca7b20a65e976f9754169f7644" }, "downloads": -1, "filename": "mipego-0.9.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "66206a2939e4275fc4ce1888e863b4a2", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12222, "upload_time": "2018-05-08T12:22:56", "url": "https://files.pythonhosted.org/packages/18/08/a2c49a0cced1709c514ff379f40169df15d1a1211c3c872af3049fae65a1/mipego-0.9.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "02c841b2f216e28c0ab266812a8090f4", "sha256": "71204eed7f28435df6febfb12ad73694ea6f0c46399b964718f3a226d3ef9b5f" }, "downloads": -1, "filename": "mipego-0.9.9.tar.gz", "has_sig": false, "md5_digest": "02c841b2f216e28c0ab266812a8090f4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13439, "upload_time": "2018-05-08T12:22:58", "url": "https://files.pythonhosted.org/packages/c2/3c/b4c500dfe8d821fd2929b96a313ddb814650cfd164ee3869e07a68651bac/mipego-0.9.9.tar.gz" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "88b108f327b653c57174c208aaa7d378", "sha256": "f85508edf920b571974bac747da7dc1ad0764e1f377743166ba2fb95d4bba9f0" }, "downloads": -1, "filename": "mipego-1.0.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "88b108f327b653c57174c208aaa7d378", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 36032, "upload_time": "2018-05-09T10:01:23", "url": "https://files.pythonhosted.org/packages/30/35/9945f633f956eeb751b33a07eaa7436ff668bc295c9885463f3a8104446f/mipego-1.0.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "34d9eb9023cc67d42264c1487ebf8bec", "sha256": "8e385cc04087b55e31c809e410983bca027cb3e968d6db7564544a0223177869" }, "downloads": -1, "filename": "mipego-1.0.0.tar.gz", "has_sig": false, "md5_digest": "34d9eb9023cc67d42264c1487ebf8bec", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34646, "upload_time": "2018-05-09T10:05:25", "url": "https://files.pythonhosted.org/packages/b9/ee/0ef710f9dd10346a1a79ea91b21fb924581d60169bf9fa7d91c8f063ba3a/mipego-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "a7315b52e4b08eab4b2e6a46526ea332", "sha256": "e235d846ab3fece6bd4fc6a1ac504f0aa0c6ea38371da551c917e24ce65697ff" }, "downloads": -1, "filename": "mipego-1.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a7315b52e4b08eab4b2e6a46526ea332", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 36086, "upload_time": "2018-05-09T13:04:31", "url": "https://files.pythonhosted.org/packages/08/a3/36be9a62dc5020c4b939696a612eb3c1647e151b8f7655a2a4af3cdac941/mipego-1.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c9911931298e47ff0663bacb758f437a", "sha256": "f265964915af1b966d421f0d3bb870b91cefdbd378333ba449a8769b138fa3b7" }, "downloads": -1, "filename": "mipego-1.0.1.tar.gz", "has_sig": false, "md5_digest": "c9911931298e47ff0663bacb758f437a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34687, "upload_time": "2018-05-09T13:04:33", "url": "https://files.pythonhosted.org/packages/05/e4/e5d545382de494ab3e157e9c8228a28617086e7606b34679a4e911ad75a7/mipego-1.0.1.tar.gz" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "a03fd4bbdfadf0abd53778ef50f402f4", "sha256": "219085178ea28eb7440e64402da358d2030b662b6131b9499d582110928bed77" }, "downloads": -1, "filename": "mipego-1.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a03fd4bbdfadf0abd53778ef50f402f4", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 36221, "upload_time": "2018-05-09T14:25:44", "url": "https://files.pythonhosted.org/packages/9d/de/05201dfdfd5ee9317962b6ac21180d175897f49dedceadd211fcfcb6dd9b/mipego-1.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e8c83f5e77871962b507785eb4efd8b1", "sha256": "71825336c0843046b0a7a7dcc2b31a382f071985f54449a327e40c30bc57291e" }, "downloads": -1, "filename": "mipego-1.0.2.tar.gz", "has_sig": false, "md5_digest": "e8c83f5e77871962b507785eb4efd8b1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34820, "upload_time": "2018-05-09T14:25:46", "url": "https://files.pythonhosted.org/packages/a9/0d/fcb620b68cc3d58357ceb585bbf2ccf9ad1b56a908d49873d5674dbc844b/mipego-1.0.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a03fd4bbdfadf0abd53778ef50f402f4", "sha256": "219085178ea28eb7440e64402da358d2030b662b6131b9499d582110928bed77" }, "downloads": -1, "filename": "mipego-1.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a03fd4bbdfadf0abd53778ef50f402f4", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 36221, "upload_time": "2018-05-09T14:25:44", "url": "https://files.pythonhosted.org/packages/9d/de/05201dfdfd5ee9317962b6ac21180d175897f49dedceadd211fcfcb6dd9b/mipego-1.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e8c83f5e77871962b507785eb4efd8b1", "sha256": "71825336c0843046b0a7a7dcc2b31a382f071985f54449a327e40c30bc57291e" }, "downloads": -1, "filename": "mipego-1.0.2.tar.gz", "has_sig": false, "md5_digest": "e8c83f5e77871962b507785eb4efd8b1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34820, "upload_time": "2018-05-09T14:25:46", "url": "https://files.pythonhosted.org/packages/a9/0d/fcb620b68cc3d58357ceb585bbf2ccf9ad1b56a908d49873d5674dbc844b/mipego-1.0.2.tar.gz" } ] }