{
"info": {
"author": "Tim Santor",
"author_email": "tsantor@xstudios.agency",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 5 - Production/Stable",
"Environment :: Console",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3.4"
],
"description": "Manifest Ingest\n===============\n\nAuthor: Tim Santor tsantor@xstudios.agency\n\nOverview\n========\n\nDownload a project JSON manifest and its media files via S3, SFTP or\nHTTP. This tool was developed for internal use on all X Studios\ninteractive installations which need to have the ability to run\n\u201coffline\u201d when there is no active network connection.\n\nInstallation\n============\n\n::\n\n pip install manifest-ingest\n\nOr if you want to install a development version:\n\n::\n\n pip install git+https://bitbucket.org/xstudios/manifest-ingest.git@tag\n\nUsage\n=====\n\nCreate a Config file\n--------------------\n\nCreate a config file at ``~/manifest-ingest.cfg``:\n\n.. code:: ini\n\n [default]\n ; API endpoint which returns JSON\n api_url = http://192.168.1.69:8000/api/v1/music/playlist/\n ; Authorization header needed to access the API (optional)\n api_token = Token API_TOKEN\n ; Regex of key names to search for\n keys = (audio_url|artwork_url)\n ; Override the default manifest filename (optional)\n manifest_filename = manifest.json\n ; Local directory to place downloaded files\n local_dir = ~/target/dir/downloads\n ; Where to place the log file for the application\n log = ~/target/dir/manifest-ingest.log\n ; Max retries to make failed network request before giving up\n max_retries = 5\n ; Base URL for relative paths\n base_url = http://192.168.1.69:8000\n ; Concurrent downloads (number of cores * 2 is usually best)\n concurrent_downloads = 4\n\n [s3]\n profile_name = default\n bucket_name = BUCKET_NAME\n ; Compare MD5 hash of local file with remote file\n compare_md5 = true\n\n [sftp]\n server = 192.168.1.100\n username = USERNAME\n password = PASSWORD\n ; Remote directory on server to prepend to file URLs in manifest\n remote_dir = /path/to/project\n\n [post_download]\n ; Command to run when complete\n command = osascript -e 'tell application \"Safari\" to activate'\n\n..\n\n NOTE: The config file can be located anywhere that is readable by\n the user. The ``s3``, ``sftp`` and ``post_download`` sections are\n optional.\n\n.. usage-1:\n\nUsage\n-----\n\n.. code:: bash\n\n manifest-ingest --config=~/manifest-ingest.cfg\n\nRun at Startup (Mac)\n--------------------\n\nCreate a plist file as\n``/Library/LaunchAgents/com.user.manifestingest.plist``\n\n.. code:: xml\n\n \n \n \n \n Label\n com.user.manifestingest\n ProgramArguments\n \n /usr/local/bin/manifest-ingest\n --config\n ~/config/manifest-ingest.cfg\n \n RunAtLoad\n \n \n \n\nThen run:\n\n.. code:: bash\n\n launchctl load /Library/LaunchAgents/com.user.manifestingest.plist\n\nRun at Startup (PC)\n-------------------\n\nCreate a Scheduled Task.\n\n- **Triggers** - At log on of any user (Delay task for 30 secs)\n- **Actions** - Start a Program: manifest-ingest\n \u2013config=\u201c/Users/Admin/project/config/manifest-ingest.cfg\u201d\n\nOn Windows, your bat file to launch a Unity app will look something like\nthis:\n\n::\n\n START \"\" /D \"C:\\Users\\Admin\\Desktop\\AppName\" \"AppName.exe\" -screen-fullscreen 1 -screen-width 1920 -screen-height 1080\n\nVersion History\n===============\n\n- **0.0.1** - Initial release\n- **0.0.2** - Added saving and backup of manifest locally\n- **0.0.3** - Graceful error handling for when a remote file doesn\u2019t\n exist and we attempt to download it\n- **0.0.4** - Added config option to launch command line app when\n download finished\n- **0.0.5** - Added alternate media URL strip method\n- **0.0.6** - Major refactor to make more configurable\n- **0.0.7** - Added single package entry point (all script entry points\n us same config) and Removed ``manifest-quickstart``\n- **0.0.8** - Fixed bug where we expected a api_token in the config\n (not always needed)\n- **0.0.9** - Fixed bug with config setup importing from wrong package\n- **0.1.0** - Fixed bug with run_command call typo\n- **0.1.1** - Fixed url path removal from JSON and correct os paths\n- **0.1.2** - Fixed remote path bug on Windows\n- **0.1.3** - Added elapsed time to log\n- **0.2.0** - Key names are now a regex which is more flexible &\n powerful\n- **0.2.1** - Fixed bug where local filename would be striped of the\n word \u201cmedia\u201d\n- **0.2.2** - Additional logging\n- **0.2.3** - Removed some logging clutter\n- **0.2.4** - If manifest is empty (eg - {}), then we abort and do not\n launch post_download commands\n- **0.2.5** - Use Keychain (OSX) or Credential Manager (Windows) to get\n SFTP password\n- **0.2.6** - When executing ``manifest-sftp`` over SSH,\n Keychain/Credential Manager do not provide the password so we added\n the ability to manually provide a password via the ``-p`` or\n ``--password`` argument on the command line (use with caution)\n- **0.2.7** - Due to complications with running via command line, we\n removed the Keychain/Credential Manager support. You may now pass\n passwords via Base64 to at least thwart over the shoulder attacks.\n Also changed the way we execute the post download command to be\n non-blocking.\n- **0.2.8** - Improved connection and SSH exception handling and added\n connection retries up to a max number of times.\n- **0.2.9** - Lock in version of pysftp to 0.2.8 due to 0.2.9 HostKeys\n issue.\n- **0.3.0** - Revert to original manifest on SFTP login failure as we\n won\u2019t be able to download the files in the manifest.\n- **0.3.1** - Bug fix with creating remote path correctly on Windows.\n- **0.3.2** - After a failed SSH attempt, revert manifest to original\n and run post_download.\n- **0.3.3** - Added ``manifest_filename`` config so we can save to\n custom named manifest file.\n- **0.3.4** - Lock in version of pysftp to 0.2.8 due to 0.2.9 HostKeys\n issue\u2026again.\n- **0.3.5** - Added ``manifest-s3`` to handle manifests of S3 files.\n- **0.3.6** - Update the way we save the manifest with s3 manifests.\n- **0.4.0** - Complete refactor. Single entrypoint of\n ``manifest-ingest``. Via config can handle S3, SFTP and HTTP manifest\n URL downloads.\n- **0.4.1** - Added ability to compare local and remote MD5 hashes to\n determine if a file is newer on S3.\n- **0.4.2**\n\n - **Changed** Changed the way we look for keys with URLs (absolute\n or relative) to regex.\n - **Changed** Changed the way we strip URL prefixes to make paths\n relative (to to the download directory) to regex.\n - **Added** Added ability to specify AWS credenttial profile to use\n for S3 in config.\n\n- **0.4.3** - Added multiprocessing to speed up downloads\n- **0.4.4** - It\u2019s a mystery\u2026\n- **0.4.5** - Added Python 3.6+ support (still works with Python 2.7+)\n- **0.4.6** - No changes. Version bump so we can remove older versions\n from PyPi. Sorry.\n- **0.4.7** - No changes. Version bump so we can remove older versions\n from test PyPi. Sorry.\n- **0.4.8** - Support S3, SFTP and HTTP download URLs.\n- **0.4.9** - Bug fix and handle ports in IPs.\n- **0.5.0** - Bug fix with handling S3 downloads as the bucket name can\n be in the domain name or in the path (eg -\n bucket-name.s3.amazonaws.com or s3.amazonaws.com/bucket-name)\n- **0.5.1** - Bug fix with multiprocessing on Windows.\n\nIssues\n======\n\nIf you experience any issues, please create an\n`issue `__ on\nBitbucket.\n\n\n",
"description_content_type": null,
"docs_url": null,
"download_url": "https://bitbucket.org/xstudios/manifest-ingest/get/0.5.1.tar.gz",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://bitbucket.org/xstudios/manifest-ingest",
"keywords": "ftp manifest downloader",
"license": "MIT",
"maintainer": "",
"maintainer_email": "",
"name": "manifest-ingest",
"package_url": "https://pypi.org/project/manifest-ingest/",
"platform": "",
"project_url": "https://pypi.org/project/manifest-ingest/",
"project_urls": {
"Download": "https://bitbucket.org/xstudios/manifest-ingest/get/0.5.1.tar.gz",
"Homepage": "https://bitbucket.org/xstudios/manifest-ingest"
},
"release_url": "https://pypi.org/project/manifest-ingest/0.5.1/",
"requires_dist": [
"pysftp (==0.2.8)",
"paramiko (<3,>=2.2.1)",
"requests (<3,>=2.18.1)",
"six (<2,>=1.10.0)",
"boto3 (<2,>=1.4.4)"
],
"requires_python": "",
"summary": "Download a project JSON manifest and its media files via SFTP, HTTP or Amazon S3. This tool was developed for internal use on all X Studios interactive installations which need to have the ability to run \"offline\" when there is no active network connection.",
"version": "0.5.1"
},
"last_serial": 3429033,
"releases": {
"0.4.6": [
{
"comment_text": "",
"digests": {
"md5": "674536c32c8b519c60788bb4c8174bfc",
"sha256": "f94adda0e6beddd60b7ce1b73589f7f0fd5f1f960fcc9f83dc834196078636a3"
},
"downloads": -1,
"filename": "manifest_ingest-0.4.6-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "674536c32c8b519c60788bb4c8174bfc",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 16871,
"upload_time": "2017-11-16T00:18:17",
"url": "https://files.pythonhosted.org/packages/db/45/30aa5fe8815af1b938cf5c384b321b551928a4a95f346b5a439630dfc088/manifest_ingest-0.4.6-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "a555054be6e37beb2d51fc202b830944",
"sha256": "1077275a9a91473ae2503168a3c95129afd71175d398a85ecc6d1ca2984d0f1e"
},
"downloads": -1,
"filename": "manifest-ingest-0.4.6.tar.gz",
"has_sig": false,
"md5_digest": "a555054be6e37beb2d51fc202b830944",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11198,
"upload_time": "2017-11-16T00:18:19",
"url": "https://files.pythonhosted.org/packages/5a/ac/e4be14af65c9905bc2f195f4a50f76a4e6ad915022a277610eaa030eac0a/manifest-ingest-0.4.6.tar.gz"
}
],
"0.4.7": [
{
"comment_text": "",
"digests": {
"md5": "322c7e83b4fd4123d569ce15caae625c",
"sha256": "d5c35d40f83d0dfcd3003304902316f4ec8089f88625bc01d92aadb8dcdb7ac3"
},
"downloads": -1,
"filename": "manifest_ingest-0.4.7-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "322c7e83b4fd4123d569ce15caae625c",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 16907,
"upload_time": "2017-11-16T00:46:07",
"url": "https://files.pythonhosted.org/packages/1e/89/ce0c03e43883137ef519c9d268703cdbc38a8c3680866f4ed66bf76fd0c4/manifest_ingest-0.4.7-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "dbe801918b4ee7cb17373f810c821e57",
"sha256": "801260e3f5ca7aa99791f0ddc25f359a2979a2fd002a0d186508a2463c56f431"
},
"downloads": -1,
"filename": "manifest-ingest-0.4.7.tar.gz",
"has_sig": false,
"md5_digest": "dbe801918b4ee7cb17373f810c821e57",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11219,
"upload_time": "2017-11-16T00:46:08",
"url": "https://files.pythonhosted.org/packages/fc/88/90b20889141f2a8203ab035ca74eb9eae54be939daf531d853cb87b3f3b7/manifest-ingest-0.4.7.tar.gz"
}
],
"0.4.8": [
{
"comment_text": "",
"digests": {
"md5": "3f84e0159b07e018931001316614173c",
"sha256": "62e90e81c2361e47f1a8169f3f63b207a8e4f93f06b832ab57c7da7443b08476"
},
"downloads": -1,
"filename": "manifest_ingest-0.4.8-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "3f84e0159b07e018931001316614173c",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 16741,
"upload_time": "2017-11-29T22:46:48",
"url": "https://files.pythonhosted.org/packages/da/b3/277655e8a86d6253d09471994764e5a6a93f04cc757d0aea2d5fbf6dea40/manifest_ingest-0.4.8-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "093dc64fbd89da43c0dca94bbbb80a2a",
"sha256": "9b2b29f59995ee915846224aab3c4cd04586d51efb4758f35f0e5dbdc66c1c42"
},
"downloads": -1,
"filename": "manifest-ingest-0.4.8.tar.gz",
"has_sig": false,
"md5_digest": "093dc64fbd89da43c0dca94bbbb80a2a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11088,
"upload_time": "2017-11-29T22:46:50",
"url": "https://files.pythonhosted.org/packages/84/4e/ed5fc23749d5c226fa08ef3830837c9e0cdd227fc93407d5517e584c6c99/manifest-ingest-0.4.8.tar.gz"
}
],
"0.4.9": [
{
"comment_text": "",
"digests": {
"md5": "a0b99a63a384a6e53c1774f21472fb2d",
"sha256": "612eaa4e46072e0a65cb65d9fe14078bd1936b020152d10880110a2d7f6c4f7a"
},
"downloads": -1,
"filename": "manifest_ingest-0.4.9-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "a0b99a63a384a6e53c1774f21472fb2d",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 16827,
"upload_time": "2017-11-29T23:58:31",
"url": "https://files.pythonhosted.org/packages/9f/bd/497ae6a2ac0b01933a7ac60d026253460f31109e30128f0087ea95ee24cf/manifest_ingest-0.4.9-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "4c36242b4dc1bb77540b0784c88343cb",
"sha256": "7f4a295ec64ebaaed5af46b499647167f8cce5b59dea00d677379dc879aa8368"
},
"downloads": -1,
"filename": "manifest-ingest-0.4.9.tar.gz",
"has_sig": false,
"md5_digest": "4c36242b4dc1bb77540b0784c88343cb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11156,
"upload_time": "2017-11-29T23:58:34",
"url": "https://files.pythonhosted.org/packages/10/50/606e7366b19c7a9d627668ac5dd2e1c1c6fed1f4b2ebee19556a124a90b7/manifest-ingest-0.4.9.tar.gz"
}
],
"0.5.0": [
{
"comment_text": "",
"digests": {
"md5": "fc47d7b5a73970b25602e77aa696b7ae",
"sha256": "0edc618ca58f54cee81c042ffbe1b149a8cf977283352eb6f2248b4fc283578e"
},
"downloads": -1,
"filename": "manifest_ingest-0.5.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "fc47d7b5a73970b25602e77aa696b7ae",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 17402,
"upload_time": "2017-12-05T22:20:56",
"url": "https://files.pythonhosted.org/packages/84/77/d2640e4edd5fbb86338172e0fbca7113ea0819f1449ea64b093c5b1d6150/manifest_ingest-0.5.0-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "2275cefb20f9f136286e386e8081289a",
"sha256": "e17d96e2800ae879a4634c35df13606472ce1a96d64592fba502e0cb7f88a44c"
},
"downloads": -1,
"filename": "manifest-ingest-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "2275cefb20f9f136286e386e8081289a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11495,
"upload_time": "2017-12-05T22:20:58",
"url": "https://files.pythonhosted.org/packages/30/39/db051b8ba138173e827a36a2f7715d1e5d25d6597a3c3767805dec4eb883/manifest-ingest-0.5.0.tar.gz"
}
],
"0.5.1": [
{
"comment_text": "",
"digests": {
"md5": "1fff87d4b3e58ead2c12f480fb6f2a33",
"sha256": "661030ad3174da4530cae0fb16528480587de8b7059806615b575bbaf038f02b"
},
"downloads": -1,
"filename": "manifest_ingest-0.5.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "1fff87d4b3e58ead2c12f480fb6f2a33",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 17439,
"upload_time": "2017-12-19T18:06:46",
"url": "https://files.pythonhosted.org/packages/34/39/b33132b43d9196448ad0ccd01a48d55e94f64a306314861e1c4c8adf46f4/manifest_ingest-0.5.1-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "1d48179f7e8e24d70abec9a2c263f009",
"sha256": "43e00c0f00cc042d6997733a73eb29eeb35f05f00f60acb8e306a005adebc2f2"
},
"downloads": -1,
"filename": "manifest-ingest-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "1d48179f7e8e24d70abec9a2c263f009",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11506,
"upload_time": "2017-12-19T18:06:49",
"url": "https://files.pythonhosted.org/packages/b6/76/7a54b944bd8ceef3211dec116b5098e8118a54f5285762009556cedb0d7d/manifest-ingest-0.5.1.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "1fff87d4b3e58ead2c12f480fb6f2a33",
"sha256": "661030ad3174da4530cae0fb16528480587de8b7059806615b575bbaf038f02b"
},
"downloads": -1,
"filename": "manifest_ingest-0.5.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "1fff87d4b3e58ead2c12f480fb6f2a33",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 17439,
"upload_time": "2017-12-19T18:06:46",
"url": "https://files.pythonhosted.org/packages/34/39/b33132b43d9196448ad0ccd01a48d55e94f64a306314861e1c4c8adf46f4/manifest_ingest-0.5.1-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "1d48179f7e8e24d70abec9a2c263f009",
"sha256": "43e00c0f00cc042d6997733a73eb29eeb35f05f00f60acb8e306a005adebc2f2"
},
"downloads": -1,
"filename": "manifest-ingest-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "1d48179f7e8e24d70abec9a2c263f009",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11506,
"upload_time": "2017-12-19T18:06:49",
"url": "https://files.pythonhosted.org/packages/b6/76/7a54b944bd8ceef3211dec116b5098e8118a54f5285762009556cedb0d7d/manifest-ingest-0.5.1.tar.gz"
}
]
}