{
    "info": {
        "author": "JL Peyret",
        "author_email": "jpeyret@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 2 - Pre-Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "![https://pypi.python.org/pypi/pip-stripper](https://img.shields.io/pypi/v/pip-stripper.svg)  ![https://travis-ci.org/jpeyret/pip-stripper](https://img.shields.io/travis/jpeyret/pip-stripper.svg) [![Coverage Status](https://coveralls.io/repos/github/jpeyret/pip-stripper/badge.svg?branch=master)](https://coveralls.io/github/jpeyret/pip-stripper?branch=master)\n[![Downloads](https://pepy.tech/badge/pip-stripper)](https://pepy.tech/project/pip-stripper)\n\n# TLDR:  requirements without *unnecessary* `pip` packages.\n\n~~~\nFor the purpose of this exercise, an unnecessary pip package is any package \nthat is not being imported by YOUR own Python code.  \nUnless you've provided a configuration override stating that you want it.\n~~~\n\n## Simplest example: initialize, scan and build in current directory\n\n````\n$ pipstripper --init --build\n\npip-stripper configuration generated @ sample/tst.seedworkdir01/pip-stripper.yaml\n\nbuild phase - generating requirements at:\n sample/tst.seedworkdir01/requirements.dev.txt\n sample/tst.seedworkdir01/requirements.txt\n\n$cat requirements.txt\nDjango==2.2\nJinja2==2.10.1\ncelery==4.3.0\ncx-Oracle==6.4.1\npsycopg2==2.8.2\npython-dateutil==2.8.0\n\n$ cat requirements.dev.txt\npyquery==1.4.0\n\n````\n\n\n\n# How it works\n\n\nLots of your packages may not be needed.  Let's say you've installed `black 18.9b0`.  \nA linter and autoformatter has no need to be on a server:\n\n`pip-stripper` most likely won't find `import black` anywhere, so it will not put it in `requirements`\n\n\n## options\n\n````\nusage: pipstripper [-h] [--config CONFIG] [--noscan] [--build] [--init]\n                   [--workdir WORKDIR] [--verbose]\n\noptional arguments:\n  -h, --help         show this help message and exit\n  --config CONFIG    config file. if not provided will look for pip-\n                     stripper.yaml in --workdir, current directory\n  --noscan           don't scan to classify packages. build phase will re-use\n                     existing pip-stripper.scan.yaml. [False].\n  --build            read pip-stripper.scan.yaml to create\n                     requirements.prod/dev.txt [False]\n  --init             initialize the config file (as pip-stripper.yaml) if it\n                     doesn't exist\n  --workdir WORKDIR  work directory [defaults to config file's value or\n                     current directory]\n  --verbose          verbose mode. adds extra zzz_debug: entry to pip-\n                     stripper.scan.yaml [False]\n````\n\n## Three phases, `initialization`, `scan` and `build`.\n\n\n### Initialization (defaults to False)\n\nThe first option `--init` will create **pip-stripper.yaml**, the configuration file for pip-stripper.\n\n**This is the only file you should edit manually!!!**\n\n\n### Scan phase (will run unless you specify `--noscan`)\n\nThis will scan your Python source files in `--workdir` and use it to create **pip-stripper.scan.yaml**.  \n\nThis is the file that contains instructions for the build phase.  \n\n**Don't edit pip-stripper.scan.yaml!**  \n\nInstead:\n - adjust the configuration in **pip-stripper.yaml**\n - re-run the scan\n\nScanning also creates 2 work files, `tmp.pip-stripper.freeze.rpt` and  `tmp.pip-stripper.imports.rpt`, tracking pip packages and its best guesses at python imports, respectively.\n\n\n### Build (defaults to False)\n\n`--build` takes what it finds in **pip-stripper.scan.yaml** and uses it to populate \n`requirements.txt` and `requirements.dev.txt`.\n\nIf those requirements files don't suit you, you may need to edit **pip-stripper.yaml**.\n\n\n## Editing `pip-stripper.yaml`\n\nThis allows you to specify:\n\n- `pip` vs `import` aliases\n- specify which packages are just *workstation*-level and shouldn't go into requirements.\n- hardcode packages that need to go into either.\n- Associating your source directories to either `prod` or `dev`.\n\n### Aliases\n\nYou may have to enter pip to python import alias names manually (alias matching is something that needs work). \n\n````\nhardcoded_aliases:\n  PyYAML: yaml\n````\n\n\n### Hardcoding package to requirement mapping:\n\nBecause `psycopg2` is typically never really imported in a Django or SQLAlchemy context, but rather derived from the configuration, you need to specify it yourself as below.  Same thing with `django-redis-cache` which is configured in django's `settings.py` as package path rather than an import.  \n\n````\nClassifierPip:\n\n  #the following are used to \"hardcode\" package names to given buckets.\n  buckets:\n    prod:\n      - psycopg2\n      - django-redis-cache\n\n    tests:\n      - nose\n      - pytest\n\n    workstation:\n      # that's a workstation only package, so it's held back\n      - black\n````\n\n#### Associating Python directories to requirements:\n\nThis is a typical regex-based configuration telling which *buckets* the directories count as:\n\n`prod` is the default outcome.  First match wins, and `tests` is the only one needed.\n\n````\nClassifierImport:\n  regex_dirs:\n    workstation: []\n    dev: []\n    tests:\n      - \"/tests/\"\n    prod: []\n\n  default_bucket: \"prod\"\n\n````\n\n### Walkthrough:\n\n#### case 1: hardcoding\n\nGiven a `pip freeze` line like \n\n````\npsycopg2==2.7.7\n````\n\n\nFirst, it looks for a matching entry in `ClassifierPip:buckets` in **pip-stripper.yaml**, (basically a hardcoded decision by the user of where to put it).\n        \n````\nClassifierPip:\n  buckets:\n    workstation:\n      - black\n    prod:\n      - psycopg2\n````\n\nThis will result in `psycopg2==2.7.7` going into **requirements.prod.txt** (when needed, requirements lines are always copied from the `pip freeze` output).\n\n#### case 2: import classification.\n\n`grep`-ing the Python code found this line:\n\n````\n./tests/helper_pyquery.py:57:    from pyquery import PyQuery\n````\n\nEach file path is run against the regex specified by `ClassifierImport:regex_dirs`, so pyquery ends up in the `tests` bucket.\n\n````\nClassifierImport:\n  regex_dirs:\n    tests:\n      - \"/tests/\"\n    prod: []\n  default_bucket: \"prod\"\n````\n\nfinally, the `--build` pass looks at where buckets get mapped:\n\n````\nBuilder:\n  req_mapper:\n    dev:\n      buckets:\n        - dev\n        - tests   \n    prod:\n      buckets:\n        - prod\n````\n\nwhich puts `pyquery` in `requirements.dev.txt`.\n\n### case 3 multiple imports\n\n````\n./tests/helper_pyquery.py:57:    from pyquery import PyQuery\n./myserver/foobar.py:22:    from pyquery import PyQuery\n````\n\nAs before, `pyquery` is put in `tests` and `prod` buckets. But also in `prod` as  `myserver` did not match any `ClassifierImport:regex_dirs`, meaning that `default_bucket: \"prod\"` was used.\n\nenter *bucket precedence*\n\n````\nClassifierPip:\n  bucket_precedence:\n    - prod\n    - tests\n    - dev\n    - workstation\n````\n\n`prod` beats `tests` so `pyquery` ends up only in `prod` bucket.\n\n### case 4.  no import match was found and nothing was hardcoded.\n\n`Babel==2.6.0`\n\nwill get left out of requirements.  Which is not to say that it won't end up `pip installed` on your server if it is a dependency of some other package ([`pipdeptree`](https://pypi.org/project/pipdeptree/) can help you there).\n\n````\nBabel==2.6.0\n  - pytz [required: >=0a, installed: 2018.9]\n\n````\n\n## Build Phase - the result of the `--scan` phase gets put into **pip-stripper.scan.yaml**:\n\nNotice our friend `black`?  We've explicitly classified it as *workstation*, so the scan didn't label it as *unknown*.\n\n````\npips:\n  buckets:\n    dev:\n    \t....\n    tests:\n    - pyquery\n    prod:\n    - psycopg2\n    unknown:\n    - Babel\n    workstation:\n    - black        \n````\n\nLook at the end for the `warnings:` section.  In this case, `repr` was used with Python 2.7 but isn't necessary with Python 3, so I won't worry about it.  A typical reason for a missing import is that automatic aliasing to link the pip name and import name didn't work. \n\n````\nwarnings:\n- missing import:repr\n````\n\n\n\n### And that's it.  The outcome?\n\nmy raw pip freeze weighs in at 158 packages:\n\n````\n$ wc -l requirements.freeze_raw.txt\n158 requirements.freeze_raw.txt\n````\n\nmy stripped down requirements ended up with 24 packages total:\n\n````\n$ wc -l requirements*txt | egrep 'prod|dev'\n6 requirements.dev.txt\n18 requirements.prod.txt\n````\n\nOn my test environment, `pip install -r requirements.prod.txt -r requirements.dev.txt` got me 48 packages, after dependencies were pulled in.\n\n````\n$ pip freeze | wc -l\n48\n````\n\n# This is all very nice, but hopefully you have sufficient tests to allow you to be confident you didn't miss anything!\n\nThe way you can test is to create a new virtualenv, pip install both requirements files and then run your tests.\n\n### WARNING:  be cautious in hardcoding package associations in dev/tests buckets rather than prod.  Your tests could run to success, but production would still fail.\n\nThese are pretty conservative as nose and pytest are really only testing tools.\n\n````\nClassifierPip:\n  buckets:\n    tests:\n      - nose\n      - nose2\n      - pytest\n````",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/jpeyret/pip-stripper",
        "keywords": "pip_stripper",
        "license": "MIT license",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pip-stripper",
        "package_url": "https://pypi.org/project/pip-stripper/",
        "platform": "",
        "project_url": "https://pypi.org/project/pip-stripper/",
        "project_urls": {
            "Homepage": "https://github.com/jpeyret/pip-stripper"
        },
        "release_url": "https://pypi.org/project/pip-stripper/0.3.0/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "strip out unnecessary pip packages from requirements",
        "version": "0.3.0"
    },
    "last_serial": 5175488,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "35825e275639faca860ef3d28d2057df",
                    "sha256": "b3c625da9e1e4a04a995299c0cbb478a644cdf15733baefe137ca523174bbd62"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "35825e275639faca860ef3d28d2057df",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20465,
                "upload_time": "2019-04-13T01:05:25",
                "url": "https://files.pythonhosted.org/packages/10/5c/7e62430eafd4020a191643320b59586b2d00b1a8d66d26cff462c701eded/pip_stripper-0.1.0.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fe483381415c301f32d63beed2f3e6eb",
                    "sha256": "8a4502a084c5b7160076f031f12e3eb8e42caa52f2e671f9288118d6ca0c38df"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "fe483381415c301f32d63beed2f3e6eb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20483,
                "upload_time": "2019-04-13T01:22:20",
                "url": "https://files.pythonhosted.org/packages/e0/ae/a7d150040f273b337f0059f0bfa871df056d9e95a60f55c4b05c3badc669/pip_stripper-0.1.1.tar.gz"
            }
        ],
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "eea5d487131b1176def1784c7005ddbd",
                    "sha256": "847c5ae48e37fc768693c3e7ca63621ded63709288c46330d07a92b96f69feff"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "eea5d487131b1176def1784c7005ddbd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25595,
                "upload_time": "2019-04-15T22:41:41",
                "url": "https://files.pythonhosted.org/packages/64/b0/cb466fafac886787804ea377343e5a45535a8004ccbea696a9d8c80c4bac/pip_stripper-0.1.5.tar.gz"
            }
        ],
        "0.1.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "736722bbe097b2a50573d11e90844095",
                    "sha256": "a8dc83901e26df16008960372c2aa122292af2f74f6cdcf04d2dbd8511e25cb0"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.1.6.tar.gz",
                "has_sig": false,
                "md5_digest": "736722bbe097b2a50573d11e90844095",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25629,
                "upload_time": "2019-04-15T22:41:42",
                "url": "https://files.pythonhosted.org/packages/3c/b0/65f1da6865131e28ccbdafcdefe41907dc21b7ed96b07c0b9300e672447e/pip_stripper-0.1.6.tar.gz"
            }
        ],
        "0.1.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "eb3d7bd175e5549f32c218427245491e",
                    "sha256": "3911a49b7920ad0128202f9686e6b1c1b3e4713a450e88097cfd8c228245bef5"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.1.7.tar.gz",
                "has_sig": false,
                "md5_digest": "eb3d7bd175e5549f32c218427245491e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 27786,
                "upload_time": "2019-04-16T01:04:48",
                "url": "https://files.pythonhosted.org/packages/0e/0b/4d82a3334d472a10033c728ad7adc3b24684b5545cc6f6b0ffc46bb1e707/pip_stripper-0.1.7.tar.gz"
            }
        ],
        "0.1.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ea3dc9a811389188df30a70883f989e6",
                    "sha256": "9f45adde3482823ea4944c9b8d8ff535937ea5a0bf885dff3d48140ee49dbfe9"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.1.8.tar.gz",
                "has_sig": false,
                "md5_digest": "ea3dc9a811389188df30a70883f989e6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 27813,
                "upload_time": "2019-04-16T01:04:49",
                "url": "https://files.pythonhosted.org/packages/1b/62/e6734ec47bdaae29f989008491aa390cf17a45487ecc88bbf3e9c67fec0d/pip_stripper-0.1.8.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9eb8983c63278e5d93f0eff83e2989c8",
                    "sha256": "6dd2d0653b006376d99162f74b5b3d30e20493397d3af2b051d0693ba285f147"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "9eb8983c63278e5d93f0eff83e2989c8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28452,
                "upload_time": "2019-04-16T22:41:46",
                "url": "https://files.pythonhosted.org/packages/53/72/22fef98671af2201f38927429e1a6fe42eb7e435bd6b481c6f2307a72f18/pip_stripper-0.2.0.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3729a0e5225312a85cf39425c8ecad13",
                    "sha256": "b1b7b0b824ac35c91c08bc50e02c46bc92eab1f0303ed57204d5420630c3fc42"
                },
                "downloads": -1,
                "filename": "pip_stripper-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3729a0e5225312a85cf39425c8ecad13",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29657,
                "upload_time": "2019-04-23T04:29:01",
                "url": "https://files.pythonhosted.org/packages/54/cf/37b45bae95c536070a48daeed5115dce0814444d916b6bbd3b52c7a7e263/pip_stripper-0.3.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "3729a0e5225312a85cf39425c8ecad13",
                "sha256": "b1b7b0b824ac35c91c08bc50e02c46bc92eab1f0303ed57204d5420630c3fc42"
            },
            "downloads": -1,
            "filename": "pip_stripper-0.3.0.tar.gz",
            "has_sig": false,
            "md5_digest": "3729a0e5225312a85cf39425c8ecad13",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 29657,
            "upload_time": "2019-04-23T04:29:01",
            "url": "https://files.pythonhosted.org/packages/54/cf/37b45bae95c536070a48daeed5115dce0814444d916b6bbd3b52c7a7e263/pip_stripper-0.3.0.tar.gz"
        }
    ]
}