{ "info": { "author": "Faculty", "author_email": "info@faculty.ai", "bugtrack_url": null, "classifiers": [], "description": "\n\n# faculty-xval\n\nCross validation of machine-learning models on Faculty platform. At present, the\npackage mostly offers a way to cross validate models in parallel by means of\nFaculty jobs. To access the functionality one makes use of the class:\n\n```python\nfaculty_xval.validation.JobsCrossValidator\n```\n\nAdditional information is found in the example notebooks provided. Please have a\nlook at the section `Try out the examples` below.\n\nThe package supports `keras` and `sklearn` models. Whilst one can write custom\nmodels that are compatible with `faculty-xval`, no guarantee is given that the\npackage handles these situations correctly, in particular because of issues\nconcerning the randomisation of weights.\n\nTwo sets of installation instructions are provided below:\n\n- If you would like to simply use `faculty-xval`, please follow the\n `User installation instructions`.\n- If you would like to develop `faculty-xval` further, please follow the\n `Developer installation instructions`.\n\n## User installation instructions\n\n### Create an environment\n\nIn your project on Faculty platform, create an environment named `faculty_xval`.\nIn the `PYTHON` section, select `Python 3` and `pip` from the dropdown menus.\nThen, type `faculty-xval` in the text box, and click on the `ADD` button.\n\nThe environment installs the package `faculty-xval`, and should be applied on\nevery server that you create; this includes both interactive servers and job\nservers, as explained next.\n\n### Create a job definition\n\nCreate a new job definition named `cross_validation`. In the `COMMAND` section,\npaste the following:\n\n`faculty_xval_jobs_xval $IN_PATHS`\n\nThen, add a `PARAMETER` with the name `IN_PATHS`, and ensure that the\n`Make field mandatory` box is checked.\n\nFinally, under `SERVER SETTINGS`, add `faculty_xval` to the `ENVIRONMENTS`\nsection.\n\nFor cross-validation jobs that are computationally intensive, we recommend using\ndedicated servers as opposed to running on shared infrastructure. To achieve\nthis, click on `Large and GPU servers` under `SERVER RESOURCES`, and select an\nappropriate server type from the dropdown menu.\n\nRemember to click `SAVE` when you are finished.\n\n## Developer installation instructions\n\n### Select a username\n\nBefore beginning the installation process, pick an appropriate username, such as\n`foo`. This does not necessarily need to match your Faculty platform username.\nIn the following instructions, your selected username will be referred to as\n``.\n\n##### Clone the repository\n\nCreate the folder `/project/`. Then, run the commands:\n\n```bash\ncd /project/\ngit clone https://github.com/facultyai/faculty-xval.git\n```\n\n### Create an environment\n\nNext, create an environment in your project named `faculty_xval_`.\n\nIn this environment, under `SCRIPTS`, paste in the following code to the `BASH`\nsection, remembering to change the `USER_NAME` definition on the second line to\nyour selected ``:\n\n```bash\n# Remember to change username!\nUSER_NAME=\n\n# Install faculty-xval from local repository.\npip install /project/$USER_NAME/faculty-xval/\n\n# Turn USER_NAME into an environment variable.\necho \"export USER_NAME=$USER_NAME\" > /etc/faculty_environment.d/app.sh\nif [[ -d /etc/service/jupyter ]] ; then\n sudo sv restart jupyter\nfi\n```\n\nThis environment should be applied on every server that you create; this\nincludes both 'normal' interactive servers and job servers, as explained next.\n\n### Create a job definition\n\nNext, create a new job definition named `cross_validation_`. In the\n`COMMAND` section, paste the following:\n\n`faculty_xval_jobs_xval $in_paths`\n\nThen, add a `PARAMETER` with the name `in_paths`, and ensure that the\n`Make field mandatory` box is checked.\n\nFinally, under `SERVER SETTINGS`, add `faculty_xval_` to the\n`ENVIRONMENTS` section.\n\nFor cross-validation jobs that are computationally intensive, we recommend using\ndedicated servers as opposed to running in the cluster. To achieve this, click\non `Large and GPU servers` under `SERVER RESOURCES`, and select an appropriate\nserver type from the dropdown menu.\n\nRemember to click `SAVE` when you are finished.\n\n## Try out the examples\n\nPlease clone this repository. Examples of cross validation with `faculty-xval`\nfor the different types of model are provided in the directories\n`examples/keras` and `examples/sklearn`. Usage instructions are then divided in\ntwo notebooks:\n\n- `jobs_cross_validator_run.ipynb` loads the data, instantiates the model, and\n starts a Faculty job that carries out the cross validation.\n- `jobs_cross_validator_analyse.ipynb` gathers the results from the cross\n validation, reloads the target data, and calculates the model accuracy over\n multiple train-test splits.\n\nNote that the example notebooks must be run in the order just defined.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/facultyai/faculty-xval", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "faculty-xval", "package_url": "https://pypi.org/project/faculty-xval/", "platform": "", "project_url": "https://pypi.org/project/faculty-xval/", "project_urls": { "Homepage": "https://github.com/facultyai/faculty-xval" }, "release_url": "https://pypi.org/project/faculty-xval/0.1.0/", "requires_dist": [ "click", "faculty", "keras (>=2.2.4)", "numpy", "scikit-learn", "tensorflow" ], "requires_python": "", "summary": "Cross validation of machine-learning models on Faculty platform.", "version": "0.1.0" }, "last_serial": 5155975, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "84bf48a0e1df63b6220c14fcb21b624d", "sha256": "dbb7c7ab966c0c218246c05ce8f71173a9274ea8a742a3f333827e9f4b3f33b7" }, "downloads": -1, "filename": "faculty_xval-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "84bf48a0e1df63b6220c14fcb21b624d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15887, "upload_time": "2019-04-17T16:13:03", "url": "https://files.pythonhosted.org/packages/99/a4/92c7ca217b9c0efed2fb4a4afeeb5d4acbe11d56593ad467fe4d2e94b6a4/faculty_xval-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f0017269bcfcfcee37db731e87a60f07", "sha256": "69a3d4cded33f5dd5cd97edec17d91c2112a51bd9ae4e8e7bfff2df37bd39188" }, "downloads": -1, "filename": "faculty-xval-0.1.0.tar.gz", "has_sig": false, "md5_digest": "f0017269bcfcfcee37db731e87a60f07", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11709, "upload_time": "2019-04-17T16:13:05", "url": "https://files.pythonhosted.org/packages/9e/a3/8cc96fdd6a7623a517272584ae94545a76e9aa676cac1bc2f8ed477920f9/faculty-xval-0.1.0.tar.gz" } ], "0.1.0rc1": [ { "comment_text": "", "digests": { "md5": "962b986f357fd97868a2ed21f9f990a2", "sha256": "656daa116f942e4edbef99d711862dfa96d2714233dc800d654c637b95b25068" }, "downloads": -1, "filename": "faculty_xval-0.1.0rc1-py3-none-any.whl", "has_sig": false, "md5_digest": "962b986f357fd97868a2ed21f9f990a2", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15810, "upload_time": "2019-04-16T18:37:00", "url": "https://files.pythonhosted.org/packages/48/cf/7c3013ce3b7a7181e1e3990542c991df644ff26c08945b2add5543d1c41c/faculty_xval-0.1.0rc1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "40a15887e420e2d145f07e97af022d75", "sha256": "1ce71afeff7b7cf69e3e533d5d27d4de1cd506b73cf81458e114a2bf26a4b294" }, "downloads": -1, "filename": "faculty-xval-0.1.0rc1.tar.gz", "has_sig": false, "md5_digest": "40a15887e420e2d145f07e97af022d75", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11270, "upload_time": "2019-04-16T18:37:02", "url": "https://files.pythonhosted.org/packages/be/5c/9fc40b11a13f8d49a642e98c3f605c783b7ff5e6a11a29805bf4cfe9d0ba/faculty-xval-0.1.0rc1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "84bf48a0e1df63b6220c14fcb21b624d", "sha256": "dbb7c7ab966c0c218246c05ce8f71173a9274ea8a742a3f333827e9f4b3f33b7" }, "downloads": -1, "filename": "faculty_xval-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "84bf48a0e1df63b6220c14fcb21b624d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 15887, "upload_time": "2019-04-17T16:13:03", "url": "https://files.pythonhosted.org/packages/99/a4/92c7ca217b9c0efed2fb4a4afeeb5d4acbe11d56593ad467fe4d2e94b6a4/faculty_xval-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f0017269bcfcfcee37db731e87a60f07", "sha256": "69a3d4cded33f5dd5cd97edec17d91c2112a51bd9ae4e8e7bfff2df37bd39188" }, "downloads": -1, "filename": "faculty-xval-0.1.0.tar.gz", "has_sig": false, "md5_digest": "f0017269bcfcfcee37db731e87a60f07", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11709, "upload_time": "2019-04-17T16:13:05", "url": "https://files.pythonhosted.org/packages/9e/a3/8cc96fdd6a7623a517272584ae94545a76e9aa676cac1bc2f8ed477920f9/faculty-xval-0.1.0.tar.gz" } ] }