{
    "info": {
        "author": "OVH Group",
        "author_email": "opensource@ovh.net",
        "bugtrack_url": null,
        "classifiers": [
            "Programming Language :: Python"
        ],
        "description": "***********************************\nLogs Data Platform - Archive mirror\n***********************************\n\n.. image:: https://img.shields.io/pypi/v/ldp-archive-mirror.svg\n   :target: https://pypi.python.org/pypi/ldp-archive-mirror/\n   :alt: Latest Version\n\n.. image:: https://travis-ci.org/ovh/ldp-archive-mirror.svg?branch=master\n   :target: https://travis-ci.org/ovh/ldp-archive-mirror\n   :alt: Latest version\n\nThis tools allows you to get a local copy of all your cold stored archives on Logs Data Platform::\n\n    [2018-12-14 17:20:00,200 INFO LDP Mirror] Create local DB if not exists\n    [2018-12-14 17:20:00,204 INFO LDP Mirror] Looking for new archives of stream e9397556-31d1-4d4d-b2bd-e5367b522cc8\n    [2018-12-14 17:20:05,190 INFO LDP Mirror] Service ldp-jb-52560 found for stream e9397556-31d1-4d4d-b2bd-e5367b522cc8\n    [2018-12-14 17:20:05,579 INFO LDP Mirror] Archive 9fb75957-2cde-435e-bdd9-6dfd33663f2c added to cache\n    [2018-12-14 17:20:05,850 INFO LDP Mirror] Archive 11d8630a-7b38-4fa4-9d7c-dfd17b0b00f6 added to cache\n    ...\n    [2018-12-14 17:20:08,392 INFO LDP Mirror] Directory mirror/e9397556-31d1-4d4d-b2bd-e5367b522cc8 created\n    [2018-12-14 17:20:08,535 INFO LDP Mirror] Archive e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-10.zst missing, download scheduled\n    [2018-12-14 17:20:08,635 INFO LDP Mirror] Archive e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-11.zst missing, download scheduled\n    ...\n    [2018-12-14 17:20:09,535 INFO LDP Mirror] Will retry to download e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-10.zst after 596 seconds\n    [2018-12-14 17:20:09,745 INFO LDP Mirror] Will retry to download e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-11.zst after 598 seconds\n    ...\n    [2018-12-14 17:20:10,927 INFO LDP Mirror] Archive e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-10.zst downloaded\n    [2018-12-14 17:20:11,655 INFO LDP Mirror] Sha256 OK on e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-10.zst\n    [2018-12-14 17:20:10,927 INFO LDP Mirror] Archive e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-11.zst downloaded\n    [2018-12-14 17:20:11,655 INFO LDP Mirror] Sha256 OK on e9397556-31d1-4d4d-b2bd-e5367b522cc8/2018-12-11.zst\n\n\nQuickstart\n==========\n\nFirst, install **ldp-archive-mirror** using `pip <https://pip.pypa.io/en/stable/>`_::\n\n    pip install -U ldp-archive-mirror\n\nThen you can use the binary `ldp-mirror`::\n\n    usage: ldp-mirror [-h] [--app-key KEY] [--app-secret SECRET]\n                  [--consumer-key KEY] [--ovh-region REGION] [--db DIR]\n                  [--mirror DIR] [--ldp-host HOST] [--ldp-token TOKEN]\n                  STREAM_ID [STREAM_ID ...]\n\n    LDP archive Mirror CLI - 0.1.0\n\n    positional arguments:\n      STREAM_ID            LDP Stream UUIDs\n\n    optional arguments:\n      -h, --help           show this help message and exit\n      --app-key KEY        OVH application key (default: None)\n      --app-secret SECRET  OVH application secret (default: None)\n      --consumer-key KEY   OVH consumer key (default: None)\n      --ovh-region REGION  OVH region (default: ovh-eu)\n      --db DIR             Where to place the local sqlite database (default: db)\n      --mirror DIR         Where to place your archives (default: mirror)\n      --ldp-host HOST      If set, push logs of the current application to given\n                           LDP hostname\n      --ldp-token TOKEN    If set, push logs of the current application to\n                           associated LDP stream token\n\nSetup\n=====\n\n1. Create an application\n------------------------\n\nTo interact with the APIs, the application needs to identify itself using an\n`application_key` and an `application_secret`. To get them, you need\nto register your application. Depending the API you plan to use, visit:\n\n- `OVH Europe <https://eu.api.ovh.com/createApp/>`_\n- `OVH US <https://api.ovhcloud.com/createApp/>`_\n- `OVH North-America <https://ca.api.ovh.com/createApp/>`_\n- `So you Start Europe <https://eu.api.soyoustart.com/createApp/>`_\n- `So you Start North America <https://ca.api.soyoustart.com/createApp/>`_\n- `Kimsufi Europe <https://eu.api.kimsufi.com/createApp/>`_\n- `Kimsufi North America <https://ca.api.kimsufi.com/createApp/>`_\n\nOn the restriction step, we invite you to set the following access rules::\n\n    curl -XPOST -H\"X-Ovh-Application: OVH_API_AK\" -H \"Content-type: application/json\" \\\n    https://eu.api.ovh.com/1.0/auth/credential  -d '{\n        \"accessRules\": [\n            {\n                \"method\": \"GET\",\n                \"path\": \"/dbaas/logs\"\n            },\n            {\n                \"method\": \"GET\",\n                \"path\": \"/dbaas/logs/*/output/graylog/stream\"\n            },\n            {\n                \"method\": \"GET\",\n                \"path\": \"/dbaas/logs/*/output/graylog/stream/*/archive*\"\n            },\n            {\n                \"method\": \"POST\",\n                \"path\": \"/dbaas/logs/*/output/graylog/stream/*/archive/*/url\"\n            }\n        ],\n        \"redirection\":\"https://www.mywebsite.com/\"\n    }'\n\n\nOnce created, you will obtain an **application key (OVH_API_AK)** and an **application\nsecret (OVH_API_AS)**.\n\n2. Environment variables\n------------------------\n\nDefault cli values can be set using environment:\n\n============================  ====================  ============================================================================\nCli parameter                 Name                  About\n============================  ====================  ============================================================================\n--app-key                     OVH_API_AK            OVH application key\n--app-secret                  OVH_API_AS            OVH application secret\n--consumer-key                OVH_API_CK            OVH customer key\n--ovh-region                  OVH_API_REGION        OVH api location. Default: *ovh-eu*\n--db                          DB_DIRECTORY          Where to place the local sqlite database. Default: *db*\n--mirror                      MIRROR_DIRECTORY      Where to place your archives. Default: *mirror*\n--ldp-host                    LDP_HOST              If set, push logs of the current application to given LDP hostname\n--ldp-token                   LDP_TOKEN             If set, push logs of the current application to associated LDP stream token\n============================  ====================  ============================================================================\n\n\n3. Launch\n---------\n\nOnce all the mandatory environment variables set, launch the process like this::\n\n    $ ldp-mirror STREAM_ID [STREAM_ID ...]\n\nThis will:\n\n- looks for the LDP service associated with the given **STREAM_ID** (s)\n- populate a local cache with all the archives found on the API\n- request for each of them a temporary download url\n- download the files when unseal time is reached\n- ask every hour the api if a new archive is available\n\nCreate docker image from sources\n================================\n\nAs this application is supposed to be kept alive indefinitely, launching it from a Docker daemon looks obvious.\n\nTo build the image form the sources, uses the given `Makefile`::\n\n    $ git clone https://github.com/ovh/ldp-archive-mirror\n    $ cd ldp-archive-mirror\n    $ make build-docker\n\nAnd to run it::\n\n    $ docker run -v /my_backup/mirror/:/data/mirror -v /my_backup/db:/data/db \\\n    -e OVH_API_AK=MY_OVH_AK -e OVH_API_AS=MY_OVH_AS -e OVH_API_CK=MY_OVH_CK \\\n    -t MY_LDP_STREAM_ID_1 MY_LDP_STREAM_ID_2\n\nRequirements\n============\n\n- Python >= 3.3\n\nProject Links\n=============\n\n- PyPI: https://pypi.python.org/pypi/ldp-archive-mirror\n- Issues: https://github.com/ovh/ldp-archive-mirror/issues\n\nLicense\n=======\n\n`OVH SAS <https://github.com/ovh/ldp-archive-mirror/blob/master/LICENSE>`_",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/ovh/ldp-archive-mirror",
        "keywords": "LDP OVH",
        "license": "OVH SAS",
        "maintainer": "",
        "maintainer_email": "",
        "name": "ldp-archive-mirror",
        "package_url": "https://pypi.org/project/ldp-archive-mirror/",
        "platform": "",
        "project_url": "https://pypi.org/project/ldp-archive-mirror/",
        "project_urls": {
            "Homepage": "https://github.com/ovh/ldp-archive-mirror"
        },
        "release_url": "https://pypi.org/project/ldp-archive-mirror/0.1.7/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "This tools allows you to get a local copy of all your cold stored archives on Logs Data Platform.",
        "version": "0.1.7"
    },
    "last_serial": 4625900,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b0d2892b172e733184fb886da8249945",
                    "sha256": "5b3dd7e79d348a3a0e4434744ce70268fd5e5cf5b2c18ad03918325c706387c9"
                },
                "downloads": -1,
                "filename": "ldp-archive-mirror-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "b0d2892b172e733184fb886da8249945",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12308,
                "upload_time": "2018-12-18T16:07:56",
                "url": "https://files.pythonhosted.org/packages/66/fa/b1c21c11ea6b429515cc0ef00fb279d083fbfe544928869dd0d713ea9cd9/ldp-archive-mirror-0.1.0.tar.gz"
            }
        ],
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c17cef12174f933b5114bd5bc52bfd9a",
                    "sha256": "92bcd11a07f7c89bcbfe331eee8255e2aa283da3d6ee169f5589eb82f96a725d"
                },
                "downloads": -1,
                "filename": "ldp-archive-mirror-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "c17cef12174f933b5114bd5bc52bfd9a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12312,
                "upload_time": "2018-12-21T14:32:07",
                "url": "https://files.pythonhosted.org/packages/36/b7/3df2322f0dfb0f1635c799582ec8506196fe38c76b205f78acaf06b39db9/ldp-archive-mirror-0.1.5.tar.gz"
            }
        ],
        "0.1.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d5abd59ab4345860847afc041f96b68f",
                    "sha256": "fceed0c43c858eccfe81d4194e110efb952171970d79879103602bbb709eeee3"
                },
                "downloads": -1,
                "filename": "ldp-archive-mirror-0.1.6.tar.gz",
                "has_sig": false,
                "md5_digest": "d5abd59ab4345860847afc041f96b68f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12319,
                "upload_time": "2018-12-21T15:01:16",
                "url": "https://files.pythonhosted.org/packages/1c/6c/bffb0326e3d5762b4aa815c4fb982bf7f6f19ff0bb8959b90a0ea5885121/ldp-archive-mirror-0.1.6.tar.gz"
            }
        ],
        "0.1.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1390e273f45b86834644f337dea82c19",
                    "sha256": "8d1cc0a8bb0a972234294465c863be43a2681a520e4f138a7f005be9a70de0ff"
                },
                "downloads": -1,
                "filename": "ldp-archive-mirror-0.1.7.tar.gz",
                "has_sig": false,
                "md5_digest": "1390e273f45b86834644f337dea82c19",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12315,
                "upload_time": "2018-12-21T17:27:35",
                "url": "https://files.pythonhosted.org/packages/dd/a4/82a3055d12878f5aec3c0516936565c6db7637efc7188af264d06d9635b4/ldp-archive-mirror-0.1.7.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "1390e273f45b86834644f337dea82c19",
                "sha256": "8d1cc0a8bb0a972234294465c863be43a2681a520e4f138a7f005be9a70de0ff"
            },
            "downloads": -1,
            "filename": "ldp-archive-mirror-0.1.7.tar.gz",
            "has_sig": false,
            "md5_digest": "1390e273f45b86834644f337dea82c19",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 12315,
            "upload_time": "2018-12-21T17:27:35",
            "url": "https://files.pythonhosted.org/packages/dd/a4/82a3055d12878f5aec3c0516936565c6db7637efc7188af264d06d9635b4/ldp-archive-mirror-0.1.7.tar.gz"
        }
    ]
}