{ "info": { "author": "Cash Costello", "author_email": "cash.costello@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Topic :: Internet :: WWW/HTTP", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "tweetvac\n================================================================\n.. image:: https://secure.travis-ci.org/cash/tweetvac.png?branch=master\n\t:target: https://travis-ci.org/cash/tweetvac\n\nPython package for sucking down tweets from Twitter. It implements\nTwitter's `guidelines for working with timelines\n`_ so\nthat you don't have to.\n\ntweetvac supports retrospective pulling of tweets from Twitter. For\nexample, it can pull down a large number of tweets by a specific user or\nall the tweets from a geographic area that mentions a search term. It\nautomatically generates the requests to work backward along the\ntimeline.\n\nInstallation\n============\nInstall tweetvac using pip:\n\n.. code-block:: bash\n\n $ pip install tweetvac\n\nIf cloning this repository, you need to install\n`twython `_ and its dependencies.\n\nAuthentication\n==============\n\nTwitter requires OAuth. tweetvac can store a user's authentication\ninformation in a configuration file for reuse.\n\n1. Log into Twitter and open\n `https://dev.twitter.com/apps `_.\n2. Create a new application. The name needs to be unique across all\n Twitter apps. A callback is not needed.\n3. Create an OAuth access token on your application web page.\n4. Create a file called tweetvac.cfg and format it as follows:\n\n::\n\n [Auth]\n consumer_key = Gx33LSA3IICoqqPoJOp9Q\n consumer_secret = 1qkKAljfpQMH9EqDZ8t50hK1HbahYXAUEi2p505umY0\n oauth_token = 14574199-4iHhtyGRAeCvVzGpPNz0GLwfYC54ba3sK5uBl4hPe\n oauth_token_secret = K80YytdT9FRXEoADlVzJ64HDQEaUMwb37N9NBykCNw5gw\n\nAlternatively, you can pass those four parameters as a tuple in the\nabove order into the ``Tweetvac`` constructor rather than storing them\nin a configuration file.\n\nThe Basics\n==========\n\nImport tweetvac\n--------------------\n\n.. code-block:: python\n\n from tweetvac import TweetVac\n\nCreate a TweetVac instance\n----------------------------\n\nYou can pass the OAuth parameters as a tuple:\n\n.. code-block:: python\n\n vac = TweetVac((consumer_key, consumer_secret, oauth_token, oauth_token_secret))\n\nor use the configuration object:\n\n.. code-block:: python\n\n config = TweetVacAuthConfig()\n vac = TweetVac(config)\n\nSuck down tweets\n-------------------\n\ntweetvac expects a Twitter endpoint and a dictionary of parameters for\nthat endpoint. Read the `Twitter\ndocumentation `_ for a list of\nendpoints and their parameters. It is recommended to set the count\noption in the params dict to the largest value supported by that\nendpoint.\n\n.. code-block:: python\n\n params = {'screen_name': 'struckDC', 'count': 200}\n data = vac.suck('statuses/user_timeline', params)\n\nWork with the data\n------------------\n\nThe data returned is a list of dicts. The fields in the dict are listed in the Twitter\nAPI `documentation on the Tweet object `_.\n\nThe data can be converted back to json and stored to a file like this:\n\n.. code-block:: python\n\n with open('data.json', 'w') as outfile:\n json.dump(data, outfile)\n\nAdvanced\n========\n\nFiltering the tweets\n--------------------\n\nTwitter provides several parameters on each endpoint for selecting what\ntweets you want to retrieve. Additional culling is available by passing\na list of filter functions.\n\n.. code-block:: python\n\n def remove_mention_tweets(tweet):\n return not '@' in tweet['text']\n\n data = vac.suck('statuses/user_timeline', params, filters=[remove_mention_tweets])\n\nReturn false from your function to remove the tweet from the list.\n\nTurning off the vacuum\n----------------------\n\ntweetvac will suck down tweets until you reach your rate limit or you\nconsume all the available tweets. To stop sooner, you can pass a cutoff\nfunction that returns true when tweetvac should stop.\n\n.. code-block:: python\n\n def stop(tweet):\n cutoff_date = time.strptime(\"Wed Jan 01 00:00:00 +0000 2014\", '%a %b %d %H:%M:%S +0000 %Y')\n tweet_date = time.strptime(tweet['created_at'], '%a %b %d %H:%M:%S +0000 %Y')\n return tweet_date < cutoff_date\n\n data = vac.suck('statuses/user_timeline', params, cutoff=stop)\n\nYou can also pass a hard limit to the number of requests to stop\ntweetvac early:\n\n.. code-block:: python\n\n data = vac.suck('statuses/user_timeline', params, max_requests=10)\n\nTwitter API\n===========\n\nSupported Endpoints\n-------------------\n\n- `statuses/user\\_timeline `_\n - tweets by the specified user.\n- `statuses/home\\_timeline `_\n - tweets by those followed by the authenticating user.\n- `statuses/mentions\\_timeline `_\n - tweets mentioning the authenticating user.\n- `statuses/retweets\\_of\\_me `_\n - tweets that are retweets of the authenticating user.\n- `search/tweets `_\n - search over tweets\n- `lists/statuses `_\n - tweets from a list of users\n\nThe endpoints have different request rate limits, count limits per\nrequest, and total tweet count limits.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/cash/tweetvac", "keywords": "twitter tweet tweetvac cursor client", "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "tweetvac", "package_url": "https://pypi.org/project/tweetvac/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/tweetvac/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/cash/tweetvac" }, "release_url": "https://pypi.org/project/tweetvac/0.3/", "requires_dist": null, "requires_python": null, "summary": "Package that makes sucking down tweets from Twitter easy.", "version": "0.3" }, "last_serial": 992878, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "1092e0a9181bccecc807a0228dcd943e", "sha256": "04871a0939c67d8bd6bd20def5ed87b4a6c34f8c725883f833c7e7bcf0729d60" }, "downloads": -1, "filename": "tweetvac-0.1.tar.gz", "has_sig": false, "md5_digest": "1092e0a9181bccecc807a0228dcd943e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6423, "upload_time": "2014-02-06T01:03:06", "url": "https://files.pythonhosted.org/packages/ef/b5/26297fbe3f89b7a41f3d0cfd3f61402c3f4eec14f71b569c12d6b921d183/tweetvac-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "069168f86b9798862b737edfddc0d2e6", "sha256": "5e480fa9d83389047b3eec6486b94ca5586d7844845f8b4a6f4f4e11ea47f200" }, "downloads": -1, "filename": "tweetvac-0.2.tar.gz", "has_sig": false, "md5_digest": "069168f86b9798862b737edfddc0d2e6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5970, "upload_time": "2014-02-06T20:11:38", "url": "https://files.pythonhosted.org/packages/34/dc/5b01f81c6da4d0eb4561e426bee982be07652ea07e9276887d6800dc685f/tweetvac-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "e1f2e55d33cb893d8adea6e67cb2c842", "sha256": "ecf1cda4061cf00c35eb6a7f16f4faf6c6b4306de8d594e9225a57183c3980fc" }, "downloads": -1, "filename": "tweetvac-0.3.tar.gz", "has_sig": false, "md5_digest": "e1f2e55d33cb893d8adea6e67cb2c842", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6513, "upload_time": "2014-02-06T20:16:06", "url": "https://files.pythonhosted.org/packages/dd/6e/344054cc08b70b1f65f5ba3c2b70818d939ca06a7af321fd2a007837b5ed/tweetvac-0.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e1f2e55d33cb893d8adea6e67cb2c842", "sha256": "ecf1cda4061cf00c35eb6a7f16f4faf6c6b4306de8d594e9225a57183c3980fc" }, "downloads": -1, "filename": "tweetvac-0.3.tar.gz", "has_sig": false, "md5_digest": "e1f2e55d33cb893d8adea6e67cb2c842", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6513, "upload_time": "2014-02-06T20:16:06", "url": "https://files.pythonhosted.org/packages/dd/6e/344054cc08b70b1f65f5ba3c2b70818d939ca06a7af321fd2a007837b5ed/tweetvac-0.3.tar.gz" } ] }