{
    "info": {
        "author": "Software Heritage developers",
        "author_email": "swh-devel@inria.fr",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "swh-loader-npm\n==============\n\nSoftware Heritage loader to ingest [`npm`](https://www.npmjs.com/) packages into the archive.\n\n# What does the loader do?\n\nThe npm loader visits and loads a npm package [1].\n\nEach visit will result in:\n- 1 snapshot (which targets n revisions ; 1 per package release version)\n- 1 revision (which targets 1 directory ; the package release version uncompressed)\n\n[1] https://docs.npmjs.com/about-packages-and-modules\n\n## First visit\n\nGiven a npm package (origin), the loader, for the first visit:\n\n- retrieves information for the given package (notably released versions)\n- then for each associated released version:\n  - retrieves the associated tarball (with checks)\n  - uncompresses locally the archive\n  - computes the hashes of the uncompressed directory\n  - then creates a revision (using ``package.json`` metadata file) targeting such directory\n- finally, creates a snapshot targeting all seen revisions (uncompressed npm package released versions and metadata).\n\n## Next visit\n\nThe loader starts by checking if something changed since the last visit. If nothing changed, the visit's snapshot is left unchanged. The new visit targets the same snapshot.\n\nIf something changed, the already seen package release versions are skipped. Only the new ones are loaded. In the end, the loader creates a new snapshot based on the previous one. Thus, the new snapshot targets both the old and new package release versions.\n\n# Development\n\n## Configuration file\n\n### Location\n\nEither:\n- `/etc/softwareheritage/loader/npm.yml`\n- `~/.config/swh/loader/npm.yml`\n\n### Configuration sample\n\n```lang=yaml\nstorage:\n  cls: remote\n  args:\n    url: http://localhost:5002/\n\ndebug: false\n```\n\n## Local run\n\nThe built-in command-line will run the loader for a specified npm package.\n\nFor instance, to load `jquery`:\n```lang=bash\n$ python3 -m swh.loader.npm.loader jquery\n```\n\nIf you need more control, you can use the loader directly. It expects\nthree arguments:\n- `package_name` (required): a npm package name\n- `package_url` (optional): URL of the npm package description (human-readable html page) that will be used as the associated origin URL in the archive\n- `project_metadata_url` (optional): URL of the npm package metadata information (machine-parsable JSON document)\n\n```lang=python\nimport logging\n\nfrom urllib.parse import quote\n\nfrom swh.loader.npm.loader import NpmLoader\n\nlogging.basicConfig(level=logging.DEBUG)\n\npackage_name='webpack'\n\nNpmLoader().load(package_name,\n                 'https://www.npmjs.com/package/%s/' % package_name,\n                 'https://replicate.npmjs.com/%s/' % quote(package_name, safe=''))\n```\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://forge.softwareheritage.org/source/swh-loader-npm.git",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "swh.loader.npm",
        "package_url": "https://pypi.org/project/swh.loader.npm/",
        "platform": "",
        "project_url": "https://pypi.org/project/swh.loader.npm/",
        "project_urls": {
            "Bug Reports": "https://forge.softwareheritage.org/maniphest",
            "Funding": "https://www.softwareheritage.org/donate",
            "Homepage": "https://forge.softwareheritage.org/source/swh-loader-npm.git",
            "Source": "https://forge.softwareheritage.org/source/swh-loader-npm"
        },
        "release_url": "https://pypi.org/project/swh.loader.npm/0.0.7/",
        "requires_dist": [
            "python-dateutil",
            "requests",
            "setuptools",
            "vcversioner",
            "chardet",
            "swh.model (>=0.0.28)",
            "swh.storage (>=0.0.143)",
            "swh.scheduler",
            "swh.loader.core (>=0.0.40)",
            "pytest ; extra == 'testing'",
            "requests-mock ; extra == 'testing'",
            "swh.core[http] (>=0.0.61) ; extra == 'testing'"
        ],
        "requires_python": "",
        "summary": "Software Heritage loader for npm packages",
        "version": "0.0.7"
    },
    "last_serial": 5730565,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7ec4fded603faae21acfe5c3ad24fa43",
                    "sha256": "3035fe87a0e551e4d59d3162cb112016f938b7d6dfe63197e9ae4e632136d0e4"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7ec4fded603faae21acfe5c3ad24fa43",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 30732,
                "upload_time": "2019-04-09T14:12:21",
                "url": "https://files.pythonhosted.org/packages/f5/b2/8263b8a25ba051632b8cbcd79814f2bbd021fc91b4d00599c1708062bc85/swh.loader.npm-0.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3a7ee9c9c04afb62908ea474887ec726",
                    "sha256": "9001a2f4838a8d919c08055cffda0f5feb65117e9e99ff58617ad21f799bceb1"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "3a7ee9c9c04afb62908ea474887ec726",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14625,
                "upload_time": "2019-04-09T14:12:23",
                "url": "https://files.pythonhosted.org/packages/15/c3/0041bede7d70e3fe83b3f3d199d1f06ebd467cb05cca4439575d7f8525e6/swh.loader.npm-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "755adbb6a044b50106ae75eaa1de6499",
                    "sha256": "6d2f25341c40eed08e6bd31497247ff4914f45498289f69e840c03938926589a"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "755adbb6a044b50106ae75eaa1de6499",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 179666,
                "upload_time": "2019-04-09T14:27:10",
                "url": "https://files.pythonhosted.org/packages/ec/3e/c5d2e5bf2509ddd478544208c2caf329ded4f708f316211624e77a6060e7/swh.loader.npm-0.0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "d81d26cc7866ff5627bf82a4c95a2284",
                    "sha256": "0a68eba45b6040efb6ea0a779dc418f2cb5547de7602184cc80ccd906fa7cb2c"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "d81d26cc7866ff5627bf82a4c95a2284",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 160067,
                "upload_time": "2019-04-09T14:27:11",
                "url": "https://files.pythonhosted.org/packages/ba/78/19a814a01d3e2372b78955a95900bab5d715a5d59b08316f2634ad90557b/swh.loader.npm-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "552adcfa82d8ab547e8352e95abf8060",
                    "sha256": "5b70e2742c644ef9506966e279059898045a40594f02959c73816234aea13901"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "552adcfa82d8ab547e8352e95abf8060",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 179661,
                "upload_time": "2019-04-10T09:13:18",
                "url": "https://files.pythonhosted.org/packages/06/9b/5111843f839b50b95234ae17a8b1909aa56620d75dae87debee52e658abb/swh.loader.npm-0.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e872ef8308a1f66a9af0a28c45a02e4c",
                    "sha256": "c161af0bcd05bf298883c17b0af0ac24e4a945fe53d45e388106094e37f24e0f"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "e872ef8308a1f66a9af0a28c45a02e4c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 160080,
                "upload_time": "2019-04-10T09:13:20",
                "url": "https://files.pythonhosted.org/packages/8b/c8/dffcaac9378fb2ed88cb93bd3a00588f3b8d90a881357f8c8c4fa5b4c478/swh.loader.npm-0.0.3.tar.gz"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9e3cfe029aee14aadab37065e796c7a8",
                    "sha256": "39d80f687e3ff6c01f523eb5ff7d81a35d6eeb1f6281424950784747c78f9165"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "9e3cfe029aee14aadab37065e796c7a8",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 184280,
                "upload_time": "2019-04-12T08:55:56",
                "url": "https://files.pythonhosted.org/packages/fa/17/04303733cdf44015fd00753dbb6dc226196ac3c765c4c1926814f36febe8/swh.loader.npm-0.0.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "59931e4b3624df272311e73edbf8dd1b",
                    "sha256": "ccaf407454b4952f4c0a051c0d4b0bf140059f99b22432974b244832cc56956a"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "59931e4b3624df272311e73edbf8dd1b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 163469,
                "upload_time": "2019-04-12T08:55:58",
                "url": "https://files.pythonhosted.org/packages/1c/c7/f282302218e23dcb468fc19830f6bb58128160a170524f62657eed197fc4/swh.loader.npm-0.0.4.tar.gz"
            }
        ],
        "0.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "01000f1bd81d2df9ab5d6adaf4b13b95",
                    "sha256": "62a154de53c61febfcf227177b3f6a5e27d3ce43d00a3d9823e2edf99b70e18f"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "01000f1bd81d2df9ab5d6adaf4b13b95",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 184319,
                "upload_time": "2019-05-06T13:31:11",
                "url": "https://files.pythonhosted.org/packages/42/71/00fe3a11fe6cee2b227ff6cc1e9c41164fd23e5d9efef455a3be26dae982/swh.loader.npm-0.0.5-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "032d0e8a4f6aae710d6e85f45100cc57",
                    "sha256": "3dda0e5e2868e6169b724435f2a19b1e92cfea93ee406f29e8e557aae4998e85"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "032d0e8a4f6aae710d6e85f45100cc57",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 163497,
                "upload_time": "2019-05-06T13:31:13",
                "url": "https://files.pythonhosted.org/packages/c8/dc/4be2fc49baa9d5c43e919e52fcc77a753073498d71a5e199d55a3b46dad2/swh.loader.npm-0.0.5.tar.gz"
            }
        ],
        "0.0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "452f89d83d1db6db42a8de5e62f08592",
                    "sha256": "52167f3347abb5df5191089071e1d012aa3dd51327a41897ee262a095aa491cb"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.6-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "452f89d83d1db6db42a8de5e62f08592",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 184369,
                "upload_time": "2019-05-14T15:24:40",
                "url": "https://files.pythonhosted.org/packages/44/a5/acca3b872a617c82b00a3867883020eab8e7fbeea440cfeccd6c8f6e7930/swh.loader.npm-0.0.6-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "64ffa450181d3cf502e7ec684647912b",
                    "sha256": "a86b4f4a3da9526f7027a090faac1eca6771dd7ce55691ec8c7de6387a555e3e"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "64ffa450181d3cf502e7ec684647912b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 163538,
                "upload_time": "2019-05-14T15:24:42",
                "url": "https://files.pythonhosted.org/packages/b2/9b/91d5bb65842fd3404f250d5a6f07f64fc5a08d3a43296dca224ce191d623/swh.loader.npm-0.0.6.tar.gz"
            }
        ],
        "0.0.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b80cc2fcbb9397e096176bcdd28dfb65",
                    "sha256": "6e86879a8fc8fa6ba38bc618e485914495fe60cdd83696281b04bdaa1dcb3037"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.7-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b80cc2fcbb9397e096176bcdd28dfb65",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 185257,
                "upload_time": "2019-08-26T12:40:43",
                "url": "https://files.pythonhosted.org/packages/66/2f/e6aae974fbf5a2bcae27110b4dfb0e055c6280750135a5b949abba8c3679/swh.loader.npm-0.0.7-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8034e8128b5e80585a51680da3bcf84e",
                    "sha256": "1d9ab82a26d33be5cd2ac672e4b759c598c2d456af25ad6a98839ee74b333c4d"
                },
                "downloads": -1,
                "filename": "swh.loader.npm-0.0.7.tar.gz",
                "has_sig": false,
                "md5_digest": "8034e8128b5e80585a51680da3bcf84e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 164509,
                "upload_time": "2019-08-26T12:40:45",
                "url": "https://files.pythonhosted.org/packages/46/c9/bd1492a819f69476fef211c293c720511395db83071a9b77870ad82423d8/swh.loader.npm-0.0.7.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "b80cc2fcbb9397e096176bcdd28dfb65",
                "sha256": "6e86879a8fc8fa6ba38bc618e485914495fe60cdd83696281b04bdaa1dcb3037"
            },
            "downloads": -1,
            "filename": "swh.loader.npm-0.0.7-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "b80cc2fcbb9397e096176bcdd28dfb65",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 185257,
            "upload_time": "2019-08-26T12:40:43",
            "url": "https://files.pythonhosted.org/packages/66/2f/e6aae974fbf5a2bcae27110b4dfb0e055c6280750135a5b949abba8c3679/swh.loader.npm-0.0.7-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "8034e8128b5e80585a51680da3bcf84e",
                "sha256": "1d9ab82a26d33be5cd2ac672e4b759c598c2d456af25ad6a98839ee74b333c4d"
            },
            "downloads": -1,
            "filename": "swh.loader.npm-0.0.7.tar.gz",
            "has_sig": false,
            "md5_digest": "8034e8128b5e80585a51680da3bcf84e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 164509,
            "upload_time": "2019-08-26T12:40:45",
            "url": "https://files.pythonhosted.org/packages/46/c9/bd1492a819f69476fef211c293c720511395db83071a9b77870ad82423d8/swh.loader.npm-0.0.7.tar.gz"
        }
    ]
}