{ "info": { "author": "Samapriya Roy", "author_email": "samapriya.roy@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering :: GIS" ], "description": "# pobatch: porder wrapper for Ordersv2 Batch Client\r\n [](https://doi.org/10.5281/zenodo.3263015)\r\n[](https://badge.fury.io/py/pobatch)\r\n[](https://opensource.org/licenses/Apache-2.0)\r\n\r\n\r\n**This tool is an add-on to the porder tool, so [read about the project here](https://github.com/samapriya/porder) and make sure you take additional installation steps before starting with this**. This tool fits the need to order large orders in terms of spatial and temporal grids, splitting them into smaller manageable orders while keeping in mind the concurrency limits(number of orders you can place simultaneously). In this case, the user sets a concurrency limit, and the tool automatically checks and waits before placing the next order. The tool can also estimate order size in terms of bytes per order download. The last step is to perform the same, download using porder's downloaders and an order list as created by multiorder tools. The design of this tool is kept simple, meaning you are relying on mixed use or **porder** and **pobatch** to perform these operations. [Ordersv2 is the next iteration of Planet's API](https://developers.planet.com/docs/orders/) in getting Analysis Ready Data (ARD) delivered to you. Orders v2 allows you to improved functionality in this domain, including the capability to submit an number of images in a batch order, and perform operations such as top of atmospheric reflectance, compression, coregistration and also enhanced notifications such as email and webhooks.\r\n\r\n**Please note: This tool is in no way an official tool or Planet offering, but is a personal project created and maintained by Samapriya Roy**\r\n\r\nIf you find this tool useful, star and cite it as below\r\n\r\n```\r\nSamapriya Roy. (2019, June 30). samapriya/pobatch: pobatch: porder wrapper for Ordersv2 Batch Client (Version 0.0.5). Zenodo.\r\nhttp://doi.org/10.5281/zenodo.3263015\r\n```\r\n\r\n## Table of contents\r\n* [Installation](#installation)\r\n* [Getting started](#getting-started)\r\n* [porder wrapper for Ordersv2 Batch Client](#porder wrapper for Ordersv2 Batch Client)\r\n * [quota](#quota)\r\n * [idlist](#idlist)\r\n * [idsplit](#idsplit)\r\n * [multiorder](#multiorder)\r\n * [ordsize](#ordsize)\r\n * [downloader](#downloader)\r\n * [Example Setup](#example-setup)\r\n\r\n## Installation\r\nThis assumes that you have native python & pip installed in your system, you can test this by going to the terminal (or windows command prompt) and trying\r\n\r\n```python``` and then ```pip list```\r\n\r\nIf you get no errors and you have python 2.7.14 or higher you should be good to go. Please note that I have tested this only on python 2.7.15 but it should run on python 3.\r\n\r\nShapely is notoriously tricky as a library to install on windows machines so follow the steps mentioned from [Shapely\u2019s PyPI package page](https://pypi.org/project/Shapely/). You can download and install from the [Unofficial Wheel files from here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely) download depending on the python version you have. You will get a wheel file or a file ending with .whl. You can now simply browse to the folder or migrate to it in your command prompt, for example in my case I was running Python 2.7.15 and win32 version so the command was\r\n\r\n```pip install Shapely-1.6.4.post1-cp27-cp27m-win32.whl```\r\n\r\nOr you can use [anaconda to install](https://conda-forge.github.io/). Again, both of these options are mentioned on [Shapely\u2019s Official PyPI page](https://pypi.org/project/Shapely/). **Fiona** is a recommended install used by the simplify tool, but it is not necessary. You can find installation instructions [here](https://pypi.org/project/Fiona/1.8.6/#description)\r\n\r\nOnce you have shapely configured. To install **pobatch: porder wrapper for Ordersv2 Batch Client** you can install using two methods\r\n\r\n```pip install pobatch```\r\n\r\non Ubuntu I found it helps to specify the pip type and use sudo\r\n\r\n```sudo pip2 install pobatch or sudo pip3 install pobatch```\r\n\r\nor you can also try\r\n\r\n```\r\ngit clone https://github.com/samapriya/pobatch.git\r\ncd pobatch\r\npython setup.py install\r\n```\r\nFor linux use sudo or --user.\r\n\r\nInstallation is an optional step; the application can also be run directly by executing pobatch.py script. The advantage of having it installed is being able to execute pobatch as any command line tool. I recommend installation within a virtual environment. If you don't want to install, browse into the pobatch folder and try ```python pobatch.py``` to get to the same result.\r\n\r\n\r\n## Getting started\r\n\r\nMake sure you initialized planet client by typing ```planet init``` or ```export``` or ```set PL_API_KEY=Your API Key``` As usual, to print help:\r\n\r\n```\r\npobatch -h\r\nusage: pobatch [-h] {idsplit,multiorder,ordsize,downloader} ...\r\n\r\nporder wrapper for Ordersv2 Batch Client\r\n\r\npositional arguments:\r\n {quota,idsplit,multiorder,ordsize,downloader}\r\n idlist Get idlist using geometry & filters\r\n quota Prints your Planet Quota Details\r\n idsplit Splits ID list incase you want to run them in small\r\n batches\r\n multiorder Place multiple orders based on idlists in folder\r\n ordsize Estimates total download size for each completed\r\n order(Takes times)\r\n downloader Download using order url list\r\n\r\noptional arguments:\r\n -h, --help show this help message and exit\r\n```\r\n\r\nTo obtain help for a specific functionality, simply call it with _help_ switch, e.g.: `pobatch idsplit -h`. If you didn't install pobatch, then you can run it just by going to *pobatch* directory and running `python pobatch.py [arguments go here]`\r\n\r\n## porder wrapper for Ordersv2 Batch Client\r\nThe tool is built as a wrapper around the [porder tool](https://github.com/samapriya/porder). The **porder tool** contains additionally useful tools such as convert shapefile to geojson, base64 encode your gcs credentials, simplify your geometry to fit the 500 vertices requirements and so on. So the idea is to use both of those tools in conjunction and make desired pipelines as needed. This tools is created to give the user some control over long and tedious order queue and implement push and pull of data in a batch manner.\r\n\r\n### quota\r\nJust a simple tool to print your planet subscription quota quickly.\r\n\r\n```\r\nporder quota\r\n```\r\n\r\n### idlist\r\nCreate an idlist for your geometry based on some basic filters, including geometry, start and end date and cloud cover. If no cloud cover is specified, everything from 0 to 100% cloud cover is included. For now the tool can handle geojson,json and kml files. The output is a csv file with ids. The tool also allows you to make sure you get percentage overlap, when selecting image, for clip operations adjust it accordingly (usally --ovp 1 for orders not to fail during clip). The tool now also prints estimated area in Square kilometers for the download and estimated area if you clipped your area with the geometry you are searching (just estimates).\r\n\r\n**I have changed the setup to now do the following two things**\r\n\r\n* The number option is optional, so it can look for all images in the time range, but be careful if the area is too large, _use at own risk_. A better option is to supply the number.\r\n\r\n* It is possible to often forget about the different asset types, so you can now not pass an item and the script will return every possible type of asset for each item type depending on the bundle.\r\n\r\n```\r\npobatch idlist -h\r\nusage: pobatch idlist [-h] --input INPUT --start START --end END --item ITEM\r\n [--asset ASSET] --outfile OUTFILE [--cmin CMIN]\r\n [--cmax CMAX] [--number NUMBER] [--overlap OVERLAP]\r\n [--filters FILTERS [FILTERS ...]]\r\n\r\noptional arguments:\r\n -h, --help show this help message and exit\r\n\r\nRequired named arguments.:\r\n --input INPUT Input geometry file for now geojson/json/kml\r\n --start START Start date in format YYYY-MM-DD\r\n --end END End date in format YYYY-MM-DD\r\n --item ITEM Item Type PSScene4Band|PSOrthoTile|REOrthoTile etc\r\n --asset ASSET Asset Type analytic, analytic_sr,visual etc\r\n --outfile OUTFILE Output csv file\r\n\r\nOptional named arguments:\r\n --cmin CMIN Minimum cloud cover\r\n --cmax CMAX Maximum cloud cover\r\n --number NUMBER Total number of assets, give a large number if you are\r\n not sure\r\n --overlap OVERLAP Percentage overlap of image with search area range\r\n between 0 to 100\r\n --filters FILTERS [FILTERS ...]\r\n Add an additional string or range filter\r\n\r\n```\r\n\r\nA simple setup would be the following for 800 max item ids and an overlap of 5% with the geometry we pass to the filter\r\n```\r\npobatch idlist --input \"path to geometry.geojson\" --start \"YYYY-MM-DD\" --end \"YYYY-MM-DD\" --item \"PSScene4Band\" --asset \"analytic_sr\" --number 800 --outfile \"path to idlist.csv file\" --overlap 5\r\n```\r\n\r\nTo run an experiment to add additional filter, you can now pass an additional string or range filter or both flag for string and range filters, a setup would be. The additional filters are optional\r\n\r\n```\r\npobatch idlist --input \"Path to geojson file\" --start \"YYYY-MM-DD\" --end \"YYYY-MM-DD\" --item \"PSScene4Band\" --asset \"analytic\" --outfile \"Path to idlist.csv\" --filters range:clear_percent:55:100 --number 20\r\n\r\npobatch idlist --input \"Path to geojson file\" --start \"YYYY-MM-DD\" --end \"YYYY-MM-DD\" --item \"PSScene4Band\" --asset \"analytic\" --outfile \"Path to idlist.csv\" --filters string:satellite_id:\"1003,1006,1012,1020,1038\" --number 20\r\n\r\npobatch idlist --input \"Path to geojson file\" --start \"YYYY-MM-DD\" --end \"YYYY-MM-DD\" --item \"PSScene4Band\" --asset \"analytic\" --outfile \"Path to idlist.csv\" --filters string:satellite_id:\"1003,1006,1012,1020,1038\" range:clear_percent:55:100 --number 20\r\n```\r\n\r\nThe idlist tool can now use a multipolygon and iteratively look for scenes.\r\n\r\n### idsplit\r\nThis allows you to split your idlist into small csv files to created batches of orders.\r\n\r\n```\r\nusage: pobatch idsplit [-h] [--idlist IDLIST] [--lines LINES] [--local LOCAL]\r\n\r\noptional arguments:\r\n -h, --help show this help message and exit\r\n --idlist IDLIST Idlist txt file to split\r\n --lines LINES Maximum number of lines in each split files\r\n --local LOCAL Output folder where split files will be exported\r\n```\r\n\r\nA simple setup would be\r\n```\r\npobatch idsplit --idlist \"path to idlist.csv\" --lines \"number of lines in each idlist\" --local \"folder path to export split id lists\"\r\n```\r\n\r\n### multiorder\r\nThis tool allows you to place the order using the idlist that you created earlier. the ```--op``` argument allows you to take operations, delivery and notifications in a sequence for example ```--op toar clip email``` performs Top of Atmospheric reflectance, followed by clipping to your geometry and send you an email notification once the order has completed, failed or had any change of status. The list of operations is below and **order is important**.\r\n\r\n