{
    "info": {
        "author": "Evgeny Medvedev",
        "author_email": "evge.medvedev@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "# Ethereum ETL\n\n[![Join the chat at https://gitter.im/ethereum-eth](https://badges.gitter.im/ethereum-etl.svg)](https://gitter.im/ethereum-etl/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)\n[![Build Status](https://travis-ci.org/blockchain-etl/ethereum-etl.png)](https://travis-ci.org/blockchain-etl/ethereum-etl)\n[Join Telegram Group](https://t.me/joinchat/GsMpbA3mv1OJ6YMp3T5ORQ)\n\nInstall Ethereum ETL:\n\n```bash\npip3 install ethereum-etl\n```\n\nExport blocks and transactions ([Schema](#blockscsv), [Reference](#export_blocks_and_transactions)):\n\n```bash\n> ethereumetl export_blocks_and_transactions --start-block 0 --end-block 500000 \\\n--provider-uri https://mainnet.infura.io --blocks-output blocks.csv --transactions-output transactions.csv\n```\n\nExport ERC20 and ERC721 transfers ([Schema](#token_transferscsv), [Reference](#export_token_transfers)):\n\n```bash\n> ethereumetl export_token_transfers --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output token_transfers.csv\n```\n\nExport traces ([Schema](#tracescsv), [Reference](#export_traces)):\n\n```bash\n> ethereumetl export_traces --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/parity.ipc --output traces.csv\n```\n\n---\n\nStream blocks, transactions, logs, token_transfers continually to console ([Reference](#stream)):\n\n```bash\n> pip3 install ethereum-etl[streaming]\n> ethereumetl stream --start-block 500000 -e block,transaction,log,token_transfer --log-file log.txt\n```\n\nFind other commands [here](#command-reference).\n\nFor the latest version, check out the repo and call \n```bash\n> pip3 install -e . \n> python3 ethereumetl.py\n```\n\n[LIMITATIONS](#limitations)\n\n## Table of Contents\n\n- [Schema](#schema)\n  - [blocks.csv](#blockscsv)\n  - [transactions.csv](#transactionscsv)\n  - [token_transfers.csv](#token_transferscsv)\n  - [receipts.csv](#receiptscsv)\n  - [logs.csv](#logscsv)\n  - [contracts.csv](#contractscsv)\n  - [tokens.csv](#tokenscsv)\n  - [traces.csv](#tracescsv)\n- [Exporting the Blockchain](#exporting-the-blockchain)\n  - [Export in 2 Hours](#export-in-2-hours)\n  - [Command Reference](#command-reference)\n- [Ethereum Classic Support](#ethereum-classic-support)\n- [Querying in Amazon Athena](#querying-in-amazon-athena)\n- [Querying in Google BigQuery](#querying-in-google-bigquery)\n  - [Public Dataset](#public-dataset)\n  - [How to Query Balances for all Ethereum Addresses](#how-to-query-balances-for-all-ethereum-addresses)\n  - [Building Token Recommender in Google Cloud Platform](#building-token-recommender-in-google-cloud-platform)\n- [Blockchain ETL in Media](#blockchain-etl-in-media)\n\n\n## Schema\n\n### blocks.csv\n\nColumn            | Type               |\n------------------|--------------------|\nnumber            | bigint             |\nhash              | hex_string         |\nparent_hash       | hex_string         |\nnonce             | hex_string         |\nsha3_uncles       | hex_string         |\nlogs_bloom        | hex_string         |\ntransactions_root | hex_string         |\nstate_root        | hex_string         |\nreceipts_root     | hex_string         |\nminer             | address            |\ndifficulty        | numeric            |\ntotal_difficulty  | numeric            |\nsize              | bigint             |\nextra_data        | hex_string         |\ngas_limit         | bigint             |\ngas_used          | bigint             |\ntimestamp         | bigint             |\ntransaction_count | bigint             |\n\n### transactions.csv\n\nColumn           |    Type     |\n-----------------|-------------|\nhash             | hex_string  |\nnonce            | bigint      |\nblock_hash       | hex_string  |\nblock_number     | bigint      |\ntransaction_index| bigint      |\nfrom_address     | address     |\nto_address       | address     |\nvalue            | numeric     |\ngas              | bigint      |\ngas_price        | bigint      |\ninput            | hex_string  |\nblock_timestamp  | bigint      |\n\n### token_transfers.csv\n\nColumn              |    Type     |\n--------------------|-------------|\ntoken_address       | address     |\nfrom_address        | address     |\nto_address          | address     |\nvalue               | numeric     |\ntransaction_hash    | hex_string  |\nlog_index           | bigint      |\nblock_number        | bigint      |\n\n### receipts.csv\n\nColumn                       |    Type     |\n-----------------------------|-------------|\ntransaction_hash             | hex_string  |\ntransaction_index            | bigint      |\nblock_hash                   | hex_string  |\nblock_number                 | bigint      |\ncumulative_gas_used          | bigint      |\ngas_used                     | bigint      |\ncontract_address             | address     |\nroot                         | hex_string  |\nstatus                       | bigint      |\n\n### logs.csv\n\nColumn                   |    Type     |\n-------------------------|-------------|\nlog_index                | bigint      |\ntransaction_hash         | hex_string  |\ntransaction_index        | bigint      |\nblock_hash               | hex_string  |\nblock_number             | bigint      |\naddress                  | address     |\ndata                     | hex_string  |\ntopics                   | string      |\n\n### contracts.csv\n\nColumn                       |    Type     |\n-----------------------------|-------------|\naddress                      | address     |\nbytecode                     | hex_string  |\nfunction_sighashes           | string      |\nis_erc20                     | boolean     |\nis_erc721                    | boolean     |\nblock_number                 | bigint      |\n\n### tokens.csv\n\nColumn                       |    Type     |\n-----------------------------|-------------|\naddress                      | address     |\nsymbol                       | string      |\nname                         | string      |\ndecimals                     | bigint      |\ntotal_supply                 | numeric     |\n\n### traces.csv\n\nColumn                       |    Type     |\n-----------------------------|-------------|\nblock_number                 | bigint      |\ntransaction_hash             | hex_string  |\ntransaction_index            | bigint      |\nfrom_address                 | address     |\nto_address                   | address     |\nvalue                        | numeric     |\ninput                        | hex_string  |\noutput                       | hex_string  |\ntrace_type                   | string      |\ncall_type                    | string      |\nreward_type                  | string      |\ngas                          | bigint      |\ngas_used                     | bigint      |\nsubtraces                    | bigint      |\ntrace_address                | string      |\nerror                        | string      |\nstatus                       | bigint      |\n\nYou can find column descriptions in [https://github.com/medvedev1088/ethereum-etl-airflow](https://github.com/medvedev1088/ethereum-etl-airflow/tree/master/dags/resources/stages/raw/schemas)\n\nNote: for the `address` type all hex characters are lower-cased.\n`boolean` type can have 2 values: `True` or `False`.\n\n## LIMITATIONS\n\n- In case the contract is a proxy, which forwards all calls to a delegate, interface detection doesn\u2019t work,\nwhich means `is_erc20` and `is_erc721` will always be false for proxy contracts and they will be missing in the `tokens`\ntable.\n- The metadata methods (`symbol`, `name`, `decimals`, `total_supply`) for ERC20 are optional, so around 10% of the\ncontracts are missing this data. Also some contracts (EOS) implement these methods but with wrong return type,\nso the metadata columns are missing in this case as well.\n- `token_transfers.value`, `tokens.decimals` and `tokens.total_supply` have type `STRING` in BigQuery tables,\nbecause numeric types there can't handle 32-byte integers. You should use\n`cast(value as FLOAT64)` (possible loss of precision) or\n`safe_cast(value as NUMERIC)` (possible overflow) to convert to numbers.\n- The contracts that don't implement `decimals()` function but have the\n[fallback function](https://solidity.readthedocs.io/en/v0.4.21/contracts.html#fallback-function) that returns a `boolean`\nwill have `0` or `1` in the `decimals` column in the CSVs.\n\n## Exporting the Blockchain\n\nIf you'd like to have the blockchain data platform \nset up and hosted for you in AWS or GCP, get in touch with us \n[here](https://d5ai.typeform.com/to/cmOoLe).\n\n1. Install python 3.5.3+ https://www.python.org/downloads/\n\n1. You can use Infura if you don't need ERC20 transfers (Infura doesn't support eth_getFilterLogs JSON RPC method).\nFor that use `-p https://mainnet.infura.io` option for the commands below. If you need ERC20 transfers or want to\nexport the data ~40 times faster, you will need to set up a local Ethereum node:\n\n1. Install geth https://github.com/ethereum/go-ethereum/wiki/Installing-Geth\n\n1. Start geth.\nMake sure it downloaded the blocks that you need by executing `eth.syncing` in the JS console.\nYou can export blocks below `currentBlock`,\nthere is no need to wait until the full sync as the state is not needed (unless you also need contracts bytecode\nand token details; for those you need to wait until the full sync).\n\n1. Install Ethereum ETL:\n\n    ```bash\n    > pip3 install ethereum-etl\n    ```\n\n1. Export all:\n\n    ```bash\n    > ethereumetl export_all --help\n    > ethereumetl export_all -s 0 -e 5999999 -b 100000 -p file://$HOME/Library/Ethereum/geth.ipc -o output\n    ```\n    \n    In case `ethereumetl` command is not available in PATH, use `python3 -m ethereumetl` instead.\n\n    The result will be in the `output` subdirectory, partitioned in Hive style:\n\n    ```bash\n    output/blocks/start_block=00000000/end_block=00099999/blocks_00000000_00099999.csv\n    output/blocks/start_block=00100000/end_block=00199999/blocks_00100000_00199999.csv\n    ...\n    output/transactions/start_block=00000000/end_block=00099999/transactions_00000000_00099999.csv\n    ...\n    output/token_transfers/start_block=00000000/end_block=00099999/token_transfers_00000000_00099999.csv\n    ...\n    ```\n\nShould work with geth and parity, on Linux, Mac, Windows.\nIf you use Parity you should disable warp mode with `--no-warp` option because warp mode\ndoes not place all of the block or receipt data into the database https://wiki.parity.io/Getting-Synced\n\nIf you see weird behavior, e.g. wrong number of rows in the CSV files or corrupted files,\ncheck out this issue: https://github.com/medvedev1088/ethereum-etl/issues/28\n\n### Export in 2 Hours\n\nYou can use AWS Auto Scaling and Data Pipeline to reduce the exporting time to a few hours.\nRead this article for details https://medium.com/@medvedev1088/how-to-export-the-entire-ethereum-blockchain-to-csv-in-2-hours-for-10-69fef511e9a2\n\n### Running in Docker\n\n1. Install Docker https://docs.docker.com/install/\n\n1. Build a docker image\n    ```bash\n    > docker build -t ethereum-etl:latest .\n    > docker image ls\n    ```\n\n1. Run a container out of the image\n    ```bash\n    > docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 0 -e 5499999 -b 100000 -p https://mainnet.infura.io\n    > docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 2018-01-01 -e 2018-01-01 -p https://mainnet.infura.io\n    ```\n\n1. Run streaming to console or Pub/Sub\n    ```bash\n    > docker build -t ethereum-etl:latest-streaming -f Dockerfile_with_streaming .\n    > echo \"Stream to console\"\n    > docker run ethereum-etl:latest-streaming stream --start-block 500000 --log-file log.txt\n    > echo \"Stream to Pub/Sub\"\n    > docker run -v /path_to_credentials_file/:/ethereum-etl/ --env GOOGLE_APPLICATION_CREDENTIALS=/ethereum-etl/credentials_file.json ethereum-etl:latest-streaming stream --start-block 500000 --output projects/<your-project>/topics/crypto_ethereum\n    ```\n\n### Command Reference\n\n- [export_blocks_and_transactions](#export_blocks_and_transactions)\n- [export_token_transfers](#export_token_transfers)\n- [extract_token_transfers](#extract_token_transfers)\n- [export_receipts_and_logs](#export_receipts_and_logs)\n- [export_contracts](#export_contracts)\n- [export_tokens](#export_tokens)\n- [export_traces](#export_traces)\n- [export_geth_traces](#export_geth_traces)\n- [extract_geth_traces](#extract_geth_traces)\n- [get_block_range_for_date](#get_block_range_for_date)\n- [get_keccak_hash](#get_keccak_hash)\n- [stream](#stream)\n\nAll the commands accept `-h` parameter for help, e.g.:\n\n```bash\n> ethereumetl export_blocks_and_transactions -h\n\nUsage: ethereumetl export_blocks_and_transactions [OPTIONS]\n\n  Export blocks and transactions.\n\nOptions:\n  -s, --start-block INTEGER   Start block\n  -e, --end-block INTEGER     End block  [required]\n  -b, --batch-size INTEGER    The number of blocks to export at a time.\n  -p, --provider-uri TEXT     The URI of the web3 provider e.g.\n                              file://$HOME/Library/Ethereum/geth.ipc or\n                              https://mainnet.infura.io\n  -w, --max-workers INTEGER   The maximum number of workers.\n  --blocks-output TEXT        The output file for blocks. If not provided\n                              blocks will not be exported. Use \"-\" for stdout\n  --transactions-output TEXT  The output file for transactions. If not\n                              provided transactions will not be exported. Use\n                              \"-\" for stdout\n  -h, --help                  Show this message and exit.\n```\n\nFor the `--output` parameters the supported types are csv and json. The format type is inferred from the output file name.\n\n#### export_blocks_and_transactions\n\n```bash\n> ethereumetl export_blocks_and_transactions --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc \\\n--blocks-output blocks.csv --transactions-output transactions.csv\n```\n\nOmit `--blocks-output` or `--transactions-output` options if you want to export only transactions/blocks.\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\n[Blocks and transactions schema](#blockscsv).\n\n#### export_token_transfers\n\nThe API used in this command is not supported by Infura, so you will need a local node.\nIf you want to use Infura for exporting ERC20 transfers refer to [extract_token_transfers](#extract_token_transfers)\n\n```bash\n> ethereumetl export_token_transfers --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --batch-size 100 --output token_transfers.csv\n```\n\nInclude `--tokens <token1> --tokens <token2>` to filter only certain tokens, e.g.\n\n```bash\n> ethereumetl export_token_transfers --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output token_transfers.csv \\\n--tokens 0x86fa049857e0209aa7d9e616f7eb3b3b78ecfdb0 --tokens 0x06012c8cf97bead5deae237070f9587f8e7a266d\n```\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\n[Token transfers schema](#token_transferscsv).\n\n#### export_receipts_and_logs\n\nFirst extract transaction hashes from `transactions.csv`\n(Exported with [export_blocks_and_transactions](#export_blocks_and_transactions)):\n\n```bash\n> ethereumetl extract_csv_column --input transactions.csv --column hash --output transaction_hashes.txt\n```\n\nThen export receipts and logs:\n\n```bash\n> ethereumetl export_receipts_and_logs --transaction-hashes transaction_hashes.txt \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --receipts-output receipts.csv --logs-output logs.csv\n```\n\nOmit `--receipts-output` or `--logs-output` options if you want to export only logs/receipts.\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\nUpvote this feature request https://github.com/paritytech/parity/issues/9075,\nit will make receipts and logs export much faster.\n\n[Receipts and logs schema](#receiptscsv).\n\n#### extract_token_transfers\n\nFirst export receipt logs with [export_receipts_and_logs](#export_receipts_and_logs).\n\nThen extract transfers from the logs.csv file:\n\n```bash\n> ethereumetl extract_token_transfers --logs logs.csv --output token_transfers.csv\n```\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\n[Token transfers schema](#token_transferscsv).\n\n#### export_contracts\n\nFirst extract contract addresses from `receipts.csv`\n(Exported with [export_receipts_and_logs](#export_receipts_and_logs)):\n\n```bash\n> ethereumetl extract_csv_column --input receipts.csv --column contract_address --output contract_addresses.txt\n```\n\nThen export contracts:\n\n```bash\n> ethereumetl export_contracts --contract-addresses contract_addresses.txt \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output contracts.csv\n```\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\n[Contracts schema](#contractscsv).\n\n#### export_tokens\n\nFirst extract token addresses from `contracts.json`\n(Exported with [export_contracts](#export_contracts)):\n\n```bash\n> ethereumetl filter_items -i contracts.json -p \"item['is_erc20'] or item['is_erc721']\" | \\\nethereumetl extract_field -f address -o token_addresses.txt\n```\n\nThen export ERC20 / ERC721 tokens:\n\n```bash\n> ethereumetl export_tokens --token-addresses token_addresses.txt \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output tokens.csv\n```\n\nYou can tune `--max-workers` for performance.\n\n[Tokens schema](#tokenscsv).\n\n#### export_traces\n\nAlso called internal transactions.\nThe API used in this command is not supported by Infura, \nso you will need a local Parity archive node (`parity --tracing on`). \nMake sure your node has at least 8GB of memory, or else you will face timeout errors. \nSee [this issue](https://github.com/blockchain-etl/ethereum-etl/issues/137) \n\n```bash\n> ethereumetl export_traces --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/parity.ipc --batch-size 100 --output traces.csv\n```\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\n[Traces schema](#tracescsv).\n\n#### export_geth_traces\n\nRead [Differences between geth and parity traces.csv](#differences-between-geth-and-parity-tracescsv)\n\nThe API used in this command is not supported by Infura, \nso you will need a local Geth archive node (`geth --gcmode archive --syncmode full --ipcapi debug`).\nWhen using rpc, add `--rpc --rpcapi debug` options.\n\n```bash\n> ethereumetl export_geth_traces --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --batch-size 100 --output geth_traces.json\n```\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\n#### extract_geth_traces\n\n```bash\n> ethereumetl extract_geth_traces --input geth_traces.json --output traces.csv\n```\n\nYou can tune `--batch-size`, `--max-workers` for performance.\n\n#### get_block_range_for_date\n\n```bash\n> ethereumetl get_block_range_for_date --provider-uri=https://mainnet.infura.io --date 2018-01-01\n4832686,4838611\n```\n\n#### get_keccak_hash\n\n```bash\n> ethereumetl get_keccak_hash -i \"transfer(address,uint256)\"\n0xa9059cbb2ab09eb219583f4a59a5d0623ade346d962bcd4e46b11da047c9049b\n```\n\n#### stream\n\n```bash\n> pip3 install ethereum-etl[streaming]\n> ethereumetl stream --provider-uri https://mainnet.infura.io --start-block 500000\n```\n\n- This command outputs blocks, transactions, logs, token_transfers to the console by default.\n- Entity types can be specified with the `-e` option, \ne.g. `-e block,transaction,log,token_transfer,trace,contract,token`.\n- Use `--output` option to specify the Google Pub/Sub topic where to publish blockchain data, \ne.g. `projects/<your-project>/topics/bitcoin_blockchain`. Data will be pushed to \n`projects/<your-project>/topics/bitcoin_blockchain.blocks`, `projects/<your-project>/topics/bitcoin_blockchain.transactions` \netc. topics.\n- The command saves its state to `last_synced_block.txt` file where the last synced block number is saved periodically.\n- Specify either `--start-block` or `--last-synced-block-file` option. `--last-synced-block-file` should point to the \nfile where the block number, from which to start streaming the blockchain data, is saved.\n- Use the `--lag` option to specify how many blocks to lag behind the head of the blockchain. It's the simplest way to \nhandle chain reorganizations - they are less likely the further a block from the head.\n- You can tune `--period-seconds`, `--batch-size`, `--block-batch-size`, `--max-workers` for performance.\n- Refer to [blockchain-etl-streaming](https://github.com/blockchain-etl/blockchain-etl-streaming) for\ninstructions on deploying it to Kubernetes. \n\nStream blockchain data continually to Google Pub/Sub:\n\n```bash\n> export GOOGLE_APPLICATION_CREDENTIALS=/path_to_credentials_file.json\n> ethereumetl stream --start-block 500000 --output projects/<your-project>/topics/crypto_ethereum\n```\n\n### Running Tests\n\n```bash\n> pip3 install -e .[dev,streaming]\n> export ETHEREUM_ETL_RUN_SLOW_TESTS=True\n> pytest -vv\n```\n\n### Running Tox Tests\n\n```bash\n> pip3 install tox\n> tox\n```\n\n### Ethereum Classic Support\n\nFor getting ETC csv files, make sure you pass in the `--chain classic` param where it's required for the scripts you want to export. \nETC won't run if your `--provider-uri` is Infura. It will provide a warning and change the provider-uri to `https://ethereumclassic.network` instead. For faster performance, run a client instead locally for classic such as `parity chain=classic` and Geth-classic.\n\n### Differences between geth and parity traces.csv\n\n- `to_address` field differs for `callcode` trace (geth seems to return correct value, as parity value of `to_address` is same as `to_address` of parent call);\n- geth output doesn't have `reward` traces;\n- geth output doesn't have `to_address`, `from_address`, `value` for `suicide` traces;\n- `error` field contains human readable error message, which might differ in geth/parity output;\n- geth output doesn't have `transaction_hash`;\n- `gas_used` is 0 on traces with error in geth, empty in parity;\n- zero output of subcalls is `0x000...` in geth, `0x` in parity;\n\n## Querying in Amazon Athena\n\n- Upload the files to S3:\n\n```bash\n> cd output\n> aws s3 sync . s3://<your_bucket>/ethereumetl/export --region ap-southeast-1\n```\n\n- Sign in to Athena https://console.aws.amazon.com/athena/home\n\n- Create a database:\n\n```sql\nCREATE DATABASE ethereumetl;\n```\n\n- Create the tables:\n  - blocks: [schemas/aws/blocks.sql](schemas/aws/blocks.sql)\n  - transactions: [schemas/aws/transactions.sql](schemas/aws/transactions.sql)\n  - token_transfers: [schemas/aws/token_transfers.sql](schemas/aws/token_transfers.sql)\n  - contracts: [schemas/aws/contracts.sql](schemas/aws/contracts.sql)\n  - receipts: [schemas/aws/receipts.sql](schemas/aws/receipts.sql)\n  - logs: [schemas/aws/logs.sql](schemas/aws/logs.sql)\n  - tokens: [schemas/aws/tokens.sql](schemas/aws/tokens.sql)\n\n### Airflow DAGs\n\nRefer to https://github.com/medvedev1088/ethereum-etl-airflow for the instructions.\n\n### Tables for Parquet Files\n\nRead this article on how to convert CSVs to Parquet https://medium.com/@medvedev1088/converting-ethereum-etl-files-to-parquet-399e048ddd30\n\n- Create the tables:\n  - parquet_blocks: [schemas/aws/parquet/parquet_blocks.sql](schemas/aws/parquet/parquet_blocks.sql)\n  - parquet_transactions: [schemas/aws/parquet/parquet_transactions.sql](schemas/aws/parquet/parquet_transactions.sql)\n  - parquet_token_transfers: [schemas/aws/parquet/parquet_token_transfers.sql](schemas/aws/parquet/parquet_token_transfers.sql)\n\nNote that DECIMAL type is limited to 38 digits in Hive https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-decimal\nso values greater than 38 decimals will be null.\n\n## Querying in Google BigQuery\n\n### Public Dataset\n\nYou can query the data that's updated daily in the public BigQuery dataset\nhttps://medium.com/@medvedev1088/ethereum-blockchain-on-google-bigquery-283fb300f579\n\n### How to Query Balances for all Ethereum Addresses\n\nRead this article \nhttps://medium.com/google-cloud/how-to-query-balances-for-all-ethereum-addresses-in-bigquery-fb594e4034a7\n\n### Building Token Recommender in Google Cloud Platform\n\nRead this article \nhttps://medium.com/google-cloud/building-token-recommender-in-google-cloud-platform-1be5a54698eb\n\n## Blockchain ETL in Media\n\n- A Technical Breakdown Of Google's New Blockchain Search Tools: https://www.forbes.com/sites/michaeldelcastillo/2019/02/05/google-launches-search-for-bitcoin-ethereum-bitcoin-cash-dash-dogecoin-ethereum-classic-litecoin-and-zcash/#394fc868c789\n- Navigating Bitcoin, Ethereum, XRP: How Google Is Quietly Making Blockchains Searchable: https://www.forbes.com/sites/michaeldelcastillo/2019/02/04/navigating-bitcoin-ethereum-xrp-how-google-is-quietly-making-blockchains-searchable/?ss=crypto-blockchain#49e111da4248",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/blockchain-etl/ethereum-etl",
        "keywords": "ethereum",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "ethereum-etl",
        "package_url": "https://pypi.org/project/ethereum-etl/",
        "platform": "",
        "project_url": "https://pypi.org/project/ethereum-etl/",
        "project_urls": {
            "Bug Reports": "https://github.com/blockchain-etl/ethereum-etl/issues",
            "Chat": "https://gitter.im/ethereum-etl/Lobby",
            "Homepage": "https://github.com/blockchain-etl/ethereum-etl",
            "Source": "https://github.com/blockchain-etl/ethereum-etl"
        },
        "release_url": "https://pypi.org/project/ethereum-etl/1.3.1/",
        "requires_dist": null,
        "requires_python": ">=3.5.3,<3.8.0",
        "summary": "Tools for exporting Ethereum blockchain data to CSV or JSON",
        "version": "1.3.1"
    },
    "last_serial": 5242830,
    "releases": {
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "80a6e211b184c66451befe63626193af",
                    "sha256": "136e5f31c9bcc97cb3bd5b090077e75f2a6fee5357ff5ebdcca906171b452af5"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "80a6e211b184c66451befe63626193af",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6.0,<3.7.0",
                "size": 67074,
                "upload_time": "2018-11-07T13:54:36",
                "url": "https://files.pythonhosted.org/packages/a4/8a/bece629ce70b1f3574fba55a91949a55eee0a4ad000e8e4461ea0b0e9230/ethereum-etl-1.0.0.tar.gz"
            }
        ],
        "1.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b04c2994eed06c73d8f2b188345ce5aa",
                    "sha256": "a01e0799cb9f1f56abc6a7e10165d02e449c026015dd1d71952156cd3095fde9"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "b04c2994eed06c73d8f2b188345ce5aa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.3,<3.8.0",
                "size": 322820,
                "upload_time": "2018-11-15T18:37:15",
                "url": "https://files.pythonhosted.org/packages/f8/37/9ba562480d1c870399e71accf83ac4f6142ed65abed9276ed5f64d331797/ethereum-etl-1.2.0.tar.gz"
            }
        ],
        "1.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d90a6f1d4ae42a4f43081371b248d2d0",
                    "sha256": "28bea2377a606885064548941e692c52fa0d773dbef9a0e22d142b89e113c1aa"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "d90a6f1d4ae42a4f43081371b248d2d0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.3,<3.8.0",
                "size": 324264,
                "upload_time": "2019-01-17T07:08:40",
                "url": "https://files.pythonhosted.org/packages/9b/20/fae19833097a9c26c1114acf38793ddc185b446847ae862afba47ea465ec/ethereum-etl-1.2.1.tar.gz"
            }
        ],
        "1.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "47c315b89ddd98ae1cbb83f57671e073",
                    "sha256": "1e2bffe3070c2ab325d0b6c6fdaaa9581c924ca5d4aa79c28146ebafbaed9973"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "47c315b89ddd98ae1cbb83f57671e073",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.3,<3.8.0",
                "size": 324341,
                "upload_time": "2019-01-17T07:34:28",
                "url": "https://files.pythonhosted.org/packages/e5/08/ce53609361f4966e440dcbc7ed51d015d16f66abf341aa9987d7b017260d/ethereum-etl-1.2.2.tar.gz"
            }
        ],
        "1.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7a590b628aee05cd281106f9e68395f7",
                    "sha256": "c63889e70aa7c7bc55196ad810c937a1ad029377158c800e82e82abfdf5ef17f"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "7a590b628aee05cd281106f9e68395f7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.3,<3.8.0",
                "size": 324657,
                "upload_time": "2019-01-30T09:11:28",
                "url": "https://files.pythonhosted.org/packages/cc/29/3ffae2ea8d88aa4fd92bcb25663a19ed45987c4fc3c18ff4d4add940bce9/ethereum-etl-1.2.3.tar.gz"
            }
        ],
        "1.2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "29c980c2d862b95bf23eecc7a000249d",
                    "sha256": "d8675c83b71bac508ee2855d70c9096b5d80fc5ba9531638c90dd476cf9f25e8"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.2.4.tar.gz",
                "has_sig": false,
                "md5_digest": "29c980c2d862b95bf23eecc7a000249d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.3,<3.8.0",
                "size": 325086,
                "upload_time": "2019-03-03T07:48:53",
                "url": "https://files.pythonhosted.org/packages/c7/9c/d1e41dd5e7a049327b9e785ae9c676cb8d9787c168e8797317c096b3d464/ethereum-etl-1.2.4.tar.gz"
            }
        ],
        "1.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f8d0c811bed0c7dd10d48b73545346a9",
                    "sha256": "bd779ffb40954da2b98d8dfdfba30dd1828eaa1170d89032d3bf96bc271c2b4e"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "f8d0c811bed0c7dd10d48b73545346a9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.3,<3.8.0",
                "size": 337897,
                "upload_time": "2019-04-15T13:08:55",
                "url": "https://files.pythonhosted.org/packages/79/fc/1159c489f8675f755de1b677ac45246d430846c366c6c28461d609377b94/ethereum-etl-1.3.0.tar.gz"
            }
        ],
        "1.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3faa78fda975614b527e5ce3faa6527e",
                    "sha256": "9c5119fde063d264873c72af43964dee5b83d72943625aa7c435efdc642504bd"
                },
                "downloads": -1,
                "filename": "ethereum-etl-1.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "3faa78fda975614b527e5ce3faa6527e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5.3,<3.8.0",
                "size": 340546,
                "upload_time": "2019-05-08T13:14:33",
                "url": "https://files.pythonhosted.org/packages/40/b3/ba3f2d053fc403e55fcc17740858579ff5e3a630b864f6b153bc4f763ef2/ethereum-etl-1.3.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "3faa78fda975614b527e5ce3faa6527e",
                "sha256": "9c5119fde063d264873c72af43964dee5b83d72943625aa7c435efdc642504bd"
            },
            "downloads": -1,
            "filename": "ethereum-etl-1.3.1.tar.gz",
            "has_sig": false,
            "md5_digest": "3faa78fda975614b527e5ce3faa6527e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.5.3,<3.8.0",
            "size": 340546,
            "upload_time": "2019-05-08T13:14:33",
            "url": "https://files.pythonhosted.org/packages/40/b3/ba3f2d053fc403e55fcc17740858579ff5e3a630b864f6b153bc4f763ef2/ethereum-etl-1.3.1.tar.gz"
        }
    ]
}