{ "info": { "author": "Jie Zhu", "author_email": "zhujie@genomics.cn", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Scientific/Engineering :: Bio-Informatics" ], "description": "# **metapi**\n\nhello, metagenomics!\n\n## brother project\n\n* [anvio](https://github.com/merenlab/anvio)\n* [atlas](https://github.com/pnnl/atlas)\n* [sunbeam](https://github.com/sunbeam-labs/sunbeam)\n\n## motivation\n\n we all need a metagenomics pipeline for academic research.\n\n## principle\n\n* bind intelligense together\n * [github](https://github.com/search?q=metagenomics)\n * why we here?\n* do not make wheels\n * [biopython](https://github.com/biopython/biopython)\n * [bioperl](http://bioperl.org)\n * [seqan](https://github.com/seqan/seqan)\n* make full use of pipeline execution engine\n * [make](https://www.gnu.org/software/make/manual/make.html)\n * [snakemake](https://bitbucket.org/snakemake/snakemake)\n * [common workflow language](https://github.com/common-workflow-language/common-workflow-language)\n * [workflow definition language](https://software.broadinstitute.org/wdl/)\n* make full use of awesome bioinformatics tools\n * [bwa](https://github.com/lh3/bwa)\n * [samtools](https://github.com/samtools/samtools)\n * [spades](https://github.com/ablab/spades)\n * [idba](https://github.com/loneknightpy/idba)\n * [megahit](https://github.com/voutcn/megahit)\n * [metabat](https://bitbucket.org/berkeleylab/metabat)\n * [checkm](https://github.com/Ecogenomics/CheckM)\n* robust and module, extensible, update\n * one rule, one module\n * one module, one analysis\n* welcome to PR\n\n## design\n\n* execution module\n ```python\n # Snakefile\n include: \"rules/step.smk\"\n include: \"rules/simulation.smk\"\n include: \"rules/fastqc.smk\"\n include: \"rules/trimming.smk\"\n include: \"rules/rmhost.smk\"\n include: \"rules/assembly.smk\"\n include: \"rules/alignment.smk\"\n include: \"rules/binning.smk\"\n include: \"rules/cobinning.smk\"\n include: \"rules/checkm.smk\"\n include: \"rules/dereplication.smk\"\n include: \"rules/classification.smk\"\n include: \"rules/annotation.smk\"\n include: \"rules/profilling.smk\"\n ```\n\n* analysis module\n * raw data report\n * quality control\n * remove host sequences\n * assembly\n * assembly evaluation\n * binning\n * checkm\n * dereplication\n * bins profile\n * taxonomy classification\n * genome annotation\n * function annotation\n\n* test module\n * execution test\n * analysis test\n\n## install\n\n* install dependencies*\n * [snakemake](https://snakemake.readthedocs.io)\n * [pigz](https://zlib.net/pigz/)\n * [ncbi-genome-download](https://github.com/kblin/ncbi-genome-download)\n * [InSilicoSeq](https://github.com/HadrienG/InSilicoSeq)\n * [OAFilter](https://github.com/Scelta/OAFilter)\n * [sickle](https://github.com/najoshi/sickle)\n * [fastp](https://github.com/OpenGene/fastp)\n * [MultiQC](https://github.com/ewels/MultiQC)\n * [bwa](https://github.com/lh3/bwa)\n * [samtools](https://github.com/samtools/samtools)\n * [spades](https://github.com/ablab/spades)\n * [idba](https://github.com/loneknightpy/idba)\n * [megahit](https://github.com/voutcn/megahit)\n * [quast](https://sourceforge.net/projects/quast/)\n * [MetaBat](https://bitbucket.org/berkeleylab/metabat)\n * [MaxBin2](http://downloads.jbei.org/data/microbial_communities/MaxBin/MaxBin.html)\n * [CheckM](https://github.com/Ecogenomics/CheckM)\n * [drep](https://github.com/MrOlm/drep)\n * [prokka](https://github.com/tseemann/prokka)\n * [metaphlan2](https://bitbucket.org/biobakery/metaphlan2)\n\n ```bash\n # in python3 environment\n conda install snakemake pigz ncbi-genome-download sickle-trim fastp bwa samtools \\\n bbmap spades idba megahit maxbin2 prokka metabat2 drep quast checkm-genome\n pip install insilicoseq\n\n # in python2 envrionment\n conda install metaphlan2\n\n # database configuration\n wget https://data.ace.uq.edu.au/public/CheckM_databases/checkm_data_2015_01_16.tar.gz\n mkdir checkm_data\n cd checkm_data\n tar -xzvf ../checkm_data_2015_01_16.tar.gz\n cd ..\n ln -s checkm_data checkm_data_latest\n\n # activate python3 environment where checkm in\n checkm data setRoot checkm_data_latest\n ```\n\n* install metapi\n\n ```bash\n # recommand\n git clone https://github.com/ohmeta/metapi\n # or (maybe not latest)\n pip install metapi\n ```\n\n## example\n\n* snakemake\u4e86\u89e3\u4e00\u4e0b:)\n\n ```python\n rule bwa_mem:\n input:\n r1 = \"fastq/sample_1.fq.gz\",\n r2 = \"fastq/sample_2.fq.gz\",\n ref = \"ref/ref.index\n output:\n bam = \"sample.sort.bam\",\n stat = \"sample_flagstat.txt\"\n threads:\n 8\n shell:\n '''\n bwa mem -t {threads} {input.ref} {input.r1} {input.r2} | \\\n samtools view -@{threads} -hbS - | \\\n tee >(samtools flagstat -@{threads} - > {output.stat}) | \\\n samtools sort -@{threads} -o {output.bam} -\n '''\n ```\n\n* a simulated metagenomics data test(uncomplete)\n\n ```bash\n # in metapi/example/basic_test directory\n cd example/basic_test\n\n # look\n snakemake --dag | dot -Tsvg > dat.svg\n ```\n \n\n ```bash\n # run on local\n snakemake\n\n # run on SGE cluster\n snakemake \\\n --jobs 80 \\\n --cluster \"qsub -S /bin/bash -cwd -q {queue} -P {project} -l vf={mem},p={cores} -binding linear:{cores}\"\n ```\n\n* a real world metagenomics data process(uncomplete)\n\n ```bash\n # in metapipe directory\n # look\n cd metapi\n snakemake --dag | dot -Tsvg > ../docs/dat.svg\n ```\n \n\n ```bash\n # run on local\n snakemake \\\n --cores 8 \\\n --snakefile metapi/Snakefile \\\n --configfile metapi/metaconfig.yaml \\\n --until all\n\n # run on SGE cluster\n snakemake \\\n --snakefile metapi/Snakefile \\\n --configfile metapi/metaconfig.yaml \\\n --cluster-config metapi/metacluster.yaml \\\n --jobs 80 \\\n --cluster \"qsub -S /bin/bash -cwd -q {cluster.queue} -P {cluster.project} -l vf={cluster.mem},p={cluster.cores} -binding linear:{cluster.cores} -o {cluster.output} -e {cluster.error}\"\n --latency-wait 360 \\\n --until all\n ```\n\n## metapi command line interface\n\n* init\n\n ```bash\n metapi --help\n\n usage: metapi [subcommand] [options]\n\n metapi, a metagenomics data process pipeline\n\n optional arguments:\n -h, --help show this help message and exit\n -v, --version print software version and exit\n\n available subcommands:\n\n init a metagenomics project initialization\n simulation a simulation on metagenomics data\n workflow a workflow on real metagenomics data\n ```\n\n\n please supply samples.tsv \n format \n | id | fq1 | fq2 | \n |----|------------|------------| \n | s1 | s1.1.fq.gz | s1.2.fq.gz | \n | s2 | s2.1.fq.gz | s2.2.fq.gz | \n\n\n ```bash\n python /path/to/metapi/metapi/metapi.py init -d . -s samples.tsv -b raw -a metaspades\n ```\n\n* list\n\n ```bash\n snakemake --snakefile /path/to/metapi/metapi/Snakefile --configfile metaconfig.yaml --list\n ``` \n\n* debug\n\n ```bash \n snakemake --snakefile /path/to/metapi/metapi/Snakefile \\\n --configfile metaconfig.yaml \\\n -p -r -n --debug-dag \\\n --until checkm_lineage_wf\n ```\n\n* simulation\n\n* workflow\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ohmeta/metapi", "keywords": "", "license": "GPLv3", "maintainer": "", "maintainer_email": "", "name": "metapi", "package_url": "https://pypi.org/project/metapi/", "platform": "", "project_url": "https://pypi.org/project/metapi/", "project_urls": { "Homepage": "https://github.com/ohmeta/metapi" }, "release_url": "https://pypi.org/project/metapi/0.6.4/", "requires_dist": [ "pandas", "ruamel.yaml" ], "requires_python": ">=3.5", "summary": "a pipeline to construct a genome catalogue from metagenomics data", "version": "0.6.4" }, "last_serial": 5153063, "releases": { "0.1.3": [ { "comment_text": "", "digests": { "md5": "180a88a7d6e412cd8b9e2ecc6bc09d78", "sha256": "ff1f22ad258eb6b5e7561a5ad63da35e1b1700d2c27379a5cdaecee4b2e07e12" }, "downloads": -1, "filename": "metapi-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "180a88a7d6e412cd8b9e2ecc6bc09d78", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 17881, "upload_time": "2018-08-05T07:03:16", "url": "https://files.pythonhosted.org/packages/ed/26/0636ffdac508499dc3162d6bd75d7822b9dc2801cf182a500d6a6f40226e/metapi-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c40167b32b5a0f1f62dd6754779469dc", "sha256": "a077a6814043977c221cc792b76d7b81245e607ee2085c4078a3bb41b076505a" }, "downloads": -1, "filename": "metapi-0.1.3.tar.gz", "has_sig": false, "md5_digest": "c40167b32b5a0f1f62dd6754779469dc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14378, "upload_time": "2018-08-05T07:03:18", "url": "https://files.pythonhosted.org/packages/af/66/b3bf8e7cf7e962bf88059171381e3039ad9b0e927ecbe574533324be1960/metapi-0.1.3.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "c2b4ee3d00f075f74b997eb08b48eb1d", "sha256": "6cc8e3e684f2adc1bc99643ba430e71a1aa60ea022a7887d32f7386b5cfaaef5" }, "downloads": -1, "filename": "metapi-0.5.0-py3-none-any.whl", "has_sig": false, "md5_digest": "c2b4ee3d00f075f74b997eb08b48eb1d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 41590, "upload_time": "2019-04-15T13:11:06", "url": "https://files.pythonhosted.org/packages/6f/78/13af5535fc353e24565365c27a94d70c502ee0d86b78b22df5a7f0cd050d/metapi-0.5.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "809eb05a5c94e0a95056032b57999ae9", "sha256": "f3d22826d2909416055ef0b28332d0bf5a78f7f71fcb4afd70831a73464028f6" }, "downloads": -1, "filename": "metapi-0.5.0.tar.gz", "has_sig": false, "md5_digest": "809eb05a5c94e0a95056032b57999ae9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24612, "upload_time": "2019-04-15T13:11:08", "url": "https://files.pythonhosted.org/packages/68/70/46ce984eeeb87044a551e3c39c429b097f90655ac9e3ea9ceef664a1503b/metapi-0.5.0.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "7178842710fe542be771a5414aeda1bc", "sha256": "f5b21850f45602823ce61cb731e9fabc72f5085beb4378be56eb0c54141b23d1" }, "downloads": -1, "filename": "metapi-0.5.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7178842710fe542be771a5414aeda1bc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 41884, "upload_time": "2019-04-15T13:15:02", "url": "https://files.pythonhosted.org/packages/70/ed/39c234968772b66012a3db59a641b2a3ea77f835683260888ae1bf11fb82/metapi-0.5.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f70d0ff5d893eddf00744b4efca7d07a", "sha256": "8c4ba1335b51a824edcd6a8ee0066b3ec41d46da9c4357ce383a49d4783a8878" }, "downloads": -1, "filename": "metapi-0.5.1.tar.gz", "has_sig": false, "md5_digest": "f70d0ff5d893eddf00744b4efca7d07a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24821, "upload_time": "2019-04-15T13:15:06", "url": "https://files.pythonhosted.org/packages/51/cd/9c23c746403a7657dec3b0139635e887b578a3f3eaa7c2746fcaf840afd2/metapi-0.5.1.tar.gz" } ], "0.5.2": [ { "comment_text": "", "digests": { "md5": "cfbc95b5385d0d9c9e791aaa5cd1e679", "sha256": "3b7430594f81f4a5fe040c166ec427712f1127807febf090d6d1f761f5e924a4" }, "downloads": -1, "filename": "metapi-0.5.2-py3-none-any.whl", "has_sig": false, "md5_digest": "cfbc95b5385d0d9c9e791aaa5cd1e679", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 41883, "upload_time": "2019-04-15T13:18:01", "url": "https://files.pythonhosted.org/packages/ed/d8/5d5df4a1bc767e4fa05a808d94e8390bf129fc928e1921bbeaf33f1477d0/metapi-0.5.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "66f70964f77af8506d1261ab38e6e0fb", "sha256": "e87ed0b4e5c28998d87500dce48daf98c1c912875f5cd7e260dd4b3e5f51f77d" }, "downloads": -1, "filename": "metapi-0.5.2.tar.gz", "has_sig": false, "md5_digest": "66f70964f77af8506d1261ab38e6e0fb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24810, "upload_time": "2019-04-15T13:18:03", "url": "https://files.pythonhosted.org/packages/98/b1/6677394455ccd38f4ae9bccd3086222707b1c048578c78ca1e7d5f3e4289/metapi-0.5.2.tar.gz" } ], "0.6.0": [ { "comment_text": "", "digests": { "md5": "a94fab10542f3a44c4e0bbcfcd530e40", "sha256": "7ab7a27fdc8d756f5033d0ef4a5378703f8ff3440ce87a1e45374ec0c99cbd42" }, "downloads": -1, "filename": "metapi-0.6.0-py3-none-any.whl", "has_sig": false, "md5_digest": "a94fab10542f3a44c4e0bbcfcd530e40", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 41738, "upload_time": "2019-04-15T14:48:54", "url": "https://files.pythonhosted.org/packages/0b/61/47bae5303dc6d1d8c5aaac11422126aa5c1f9ca4e81d08720c99a175a716/metapi-0.6.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ceecd5f3349225d90a6d8a279c5d8917", "sha256": "cba7601862a8f9efcf67bcbe31f21b7b42e62ced8d124de8dfaff47ba6efdc2d" }, "downloads": -1, "filename": "metapi-0.6.0.tar.gz", "has_sig": false, "md5_digest": "ceecd5f3349225d90a6d8a279c5d8917", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23963, "upload_time": "2019-04-15T14:48:56", "url": "https://files.pythonhosted.org/packages/0e/66/50eb4b7df3fe81cc1672789c1e5f81f2b015f9621ed3fa286f3356eb7e9b/metapi-0.6.0.tar.gz" } ], "0.6.1": [ { "comment_text": "", "digests": { "md5": "56a78b8b2ddece902e70d7bac2ca8ee1", "sha256": "d8a07c4ef99931c3d8e700775e88a8d79a9d1072a6f07a419a8697f554decb76" }, "downloads": -1, "filename": "metapi-0.6.1-py3-none-any.whl", "has_sig": false, "md5_digest": "56a78b8b2ddece902e70d7bac2ca8ee1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 41738, "upload_time": "2019-04-15T15:02:50", "url": "https://files.pythonhosted.org/packages/23/63/2415bac0d3347e1a0dea9a0ec18c30d1f2cc8ce0e20f139adcf7d5e44df8/metapi-0.6.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b63ce79dec58b8542b800ff68788092c", "sha256": "256dc3eb12ed2bc7365b39f629e3bef28d6f1cf5c731298ff2aadd5ecfb52e66" }, "downloads": -1, "filename": "metapi-0.6.1.tar.gz", "has_sig": false, "md5_digest": "b63ce79dec58b8542b800ff68788092c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24897, "upload_time": "2019-04-15T15:02:52", "url": "https://files.pythonhosted.org/packages/e8/bd/3e6375bb8545771110834d066d93ee147effc2392d1ecd9d4dd814f0d2a9/metapi-0.6.1.tar.gz" } ], "0.6.2": [ { "comment_text": "", "digests": { "md5": "95ded06995ffe9e45f71c95c137b4d8f", "sha256": "02433db2970dce4329166a85cb1fd0aac840d6221ab51ee3efb49e2642de290e" }, "downloads": -1, "filename": "metapi-0.6.2-py3-none-any.whl", "has_sig": false, "md5_digest": "95ded06995ffe9e45f71c95c137b4d8f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 43045, "upload_time": "2019-04-16T00:58:04", "url": "https://files.pythonhosted.org/packages/f8/59/ab998a395d41cedfcc3668c16b2e1e86d9ffbfa0b99d0945dccd13825d31/metapi-0.6.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1e7d7fbb8b2b6dd0140da5363adef519", "sha256": "f2c3d29dc576dc4564901e9e8e07cc4c43eed05c5eb91a4a676d331cadd0d48f" }, "downloads": -1, "filename": "metapi-0.6.2.tar.gz", "has_sig": false, "md5_digest": "1e7d7fbb8b2b6dd0140da5363adef519", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 5118904, "upload_time": "2019-04-16T00:58:23", "url": "https://files.pythonhosted.org/packages/df/c3/45a67c3ad778728313abbf6ad7ff4798a465604e25b5d057bb2bae5ca1c8/metapi-0.6.2.tar.gz" } ], "0.6.3": [ { "comment_text": "", "digests": { "md5": "74b4b9b3c3056de184eed8a5c0646518", "sha256": "3e49dd34980077b5319bef4156774ddf64e9289475a909a71d0a01b09bb48c6b" }, "downloads": -1, "filename": "metapi-0.6.3-py3-none-any.whl", "has_sig": false, "md5_digest": "74b4b9b3c3056de184eed8a5c0646518", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 43044, "upload_time": "2019-04-16T01:24:18", "url": "https://files.pythonhosted.org/packages/d1/c8/9b337c0f4db087878e441f9c42112dbe72b03a8fbf61c597937fc2bb9ce9/metapi-0.6.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "251f2ed45062f6d3105e89f900ad898f", "sha256": "f39f2d57e539ac0fba78a6cc6faca99d10a984d057d7019a160613fe00031b07" }, "downloads": -1, "filename": "metapi-0.6.3.tar.gz", "has_sig": false, "md5_digest": "251f2ed45062f6d3105e89f900ad898f", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 5118855, "upload_time": "2019-04-16T01:24:38", "url": "https://files.pythonhosted.org/packages/71/26/fc5fd9d6bd66d2c615c41b4565fee25586dc7f79b8ed76923d17b38d5dc6/metapi-0.6.3.tar.gz" } ], "0.6.4": [ { "comment_text": "", "digests": { "md5": "551c8febf5890228a81fd1e76a234274", "sha256": "20a2eb9877bafb9b17a42fa89faa5b170f717a4f48d1031d55ceb772f61877b7" }, "downloads": -1, "filename": "metapi-0.6.4-py3-none-any.whl", "has_sig": false, "md5_digest": "551c8febf5890228a81fd1e76a234274", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 43106, "upload_time": "2019-04-17T03:27:05", "url": "https://files.pythonhosted.org/packages/3b/be/69bed4732b463eeb3abc6f7fb07265b72dfe98b3f782d63509a983578ca9/metapi-0.6.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9ab61df7f3c2a70fe81be810d9670501", "sha256": "859b6b046dd561496f5e82b8f901ffb76ada59d85ea333f5e33e0d0a7ec38a30" }, "downloads": -1, "filename": "metapi-0.6.4.tar.gz", "has_sig": false, "md5_digest": "9ab61df7f3c2a70fe81be810d9670501", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 5121386, "upload_time": "2019-04-17T03:27:43", "url": "https://files.pythonhosted.org/packages/05/b3/d82260d4ab4969713daa313fb3c2884b44da61acb7b078cbab09c5c7b82f/metapi-0.6.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "551c8febf5890228a81fd1e76a234274", "sha256": "20a2eb9877bafb9b17a42fa89faa5b170f717a4f48d1031d55ceb772f61877b7" }, "downloads": -1, "filename": "metapi-0.6.4-py3-none-any.whl", "has_sig": false, "md5_digest": "551c8febf5890228a81fd1e76a234274", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5", "size": 43106, "upload_time": "2019-04-17T03:27:05", "url": "https://files.pythonhosted.org/packages/3b/be/69bed4732b463eeb3abc6f7fb07265b72dfe98b3f782d63509a983578ca9/metapi-0.6.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9ab61df7f3c2a70fe81be810d9670501", "sha256": "859b6b046dd561496f5e82b8f901ffb76ada59d85ea333f5e33e0d0a7ec38a30" }, "downloads": -1, "filename": "metapi-0.6.4.tar.gz", "has_sig": false, "md5_digest": "9ab61df7f3c2a70fe81be810d9670501", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 5121386, "upload_time": "2019-04-17T03:27:43", "url": "https://files.pythonhosted.org/packages/05/b3/d82260d4ab4969713daa313fb3c2884b44da61acb7b078cbab09c5c7b82f/metapi-0.6.4.tar.gz" } ] }