{ "info": { "author": "Dennis Ren", "author_email": "code@dennisren.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3 :: Only", "Topic :: Internet :: WWW/HTTP :: Indexing/Search" ], "description": "# A Simple Scraper for Southwest Airlines\n\nSouthwest doesn't provide a direct API to inquire their fares. Other flight fare APIs exist but most are paywalled. This simple scraper simply generate a URL for southwest's web UI and scraps its contents for information. Of course, doing this is rather slow(5-10s per scrape depending on your machine), but for most people who are only interested to automate some simple price checks, a few times day is more than enough. I have not noticed any anti-scraping or CAPTCHA on their site, so we can scrape forever in theory.\n\nHave a great time scraping!\n\n-dr\n## Requirements\nPython3\n\nChrome Web Driver - get it here http://chromedriver.chromium.org/downloads\n\nSee requirements.txt for python packages used, you don't need to worry about these if you use pip to install \n## Installation\n```\npip install scrapeswa\n```\n## Example Usage\n### Import\n```\nfrom datetime import datetime\nimport scrapeswa as sswa\n```\n\n### Get Round Trip (object, recommended)\n```\n>>> outbound, return = sswa.getRoundTrip('LAX','SFO',datetime(2019,5,5),datetime(2019,5,10),returnObject=True)\n```\n\nReturns two list of outbound and return of flight object\n\n```\n>>> outbound\n([, , ... ])\n```\n\nAccessing information within flight objects\n\n```\n>>> outbound[0].__dict__\n{'flight': '2994', 'src': 'LAX', 'dst': 'SFO', 'economy': , 'anytime': , 'business': , 'leave': datetime.datetime(2019, 5, 5, 6, 45), 'arrive': datetime.datetime(2019, 5, 5, 8, 10)}\n\noutbound[0].economy\n\n\noutbound[0].economy.__dict__\n{'flightClass': 'Economy', 'fare': 59, 'earn': 250, 'pts': 3242, 'ppd': 54.94915254237288, 'epd': 4.237288135593221}\n```\n\n### Print human readable strings\n\n```\n>>> print(outbound[0])\nFlight:2994 Leaving LAX Sun 06:45 AM Arriving SFO Sun 08:10 AM\nEconomy costs $59 (3242pts) earning 250 pts\n\n```\n### Fare arithmitic\n```\n>>> sumFare = outbound[0] + outbound[1]\n>>> sumFare\n\n>>> print(sumFare)\nEconomy costs $134 (7646pts) earning 589 pts\n```\n## Get Round Trip (dictionary)\n```\n>>> outbound, return = sswa.getRoundTrip('LAX','SFO',datetime(2019,5,5),datetime(2019,5,10))\n```\nReturns two list of outbound and return flight info dictionaries\n```\n([{'Flight': '2994', 'Leave': datetime.datetime(2019, 5, 5, 6, 45), 'Arrive': datetime.datetime(2019, 5, 5, 8, 10), 'src': 'LAX', 'dst': 'SFO', 'bestAval': 'Economy', 'Business': {'fare': 253, 'earn': 2665, 'pts': 17319, 'ppd': 68.45, 'epd': 10.53}, 'Anytime': {'fare': 225, 'earn': 1960, 'pts': 15287, 'ppd': 67.94, 'epd': 8.71}, 'Economy': {'fare': 59, 'earn': 250, 'pts': 3242, 'ppd': 54.94, 'epd': 4.23}},\n\n...\n\n{'Flight': '1077', 'Leave': datetime.datetime(2019, 5, 10, 21, 25), 'Arrive': datetime.datetime(2019, 5, 10, 22, 55), 'src': 'SFO', 'dst': 'LAX', 'bestAval': 'Economy', 'Business': {'fare': 253, 'earn': 2665, 'pts': 17319, 'ppd': 68.45, 'epd': 10.53}, 'Anytime': {'fare': 225, 'earn': 1960, 'pts': 15287, 'ppd': 67.94, 'epd': 8.71}, 'Economy': {'fare': 75, 'earn': 339, 'pts': 4404, 'ppd': 58.72, 'epd': 4.52}}])\n```\n## Non-pip utilities\nIf you check out this repository, you will be able to use a few more utilities built based on this tool.\n\n#### checkSW.py\nThis is a tool I built to figure out who shouhld visit who bwtween my girlfriend and I. If you're in a long distance relationship, this tool could really help you. I'm still working on perfecting the batch scrape ability of this script. Next step is to set it up in crontab to run daily and populate everyday's results in a database, then build a data visualizer to get a good idea of ticket pricing trends and best course of action to see my beloved.\n\n#### models.py\nA sqlalchemy ORM built to help orginize the scraped data\n\n#### card.html\nA snippet of a single flight card from a fully loaded page. Helps you understand how the scraper is built\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/rmr1012/scrapeswa", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "scrapeswa", "package_url": "https://pypi.org/project/scrapeswa/", "platform": "", "project_url": "https://pypi.org/project/scrapeswa/", "project_urls": { "Homepage": "https://github.com/rmr1012/scrapeswa" }, "release_url": "https://pypi.org/project/scrapeswa/0.0.4/", "requires_dist": [ "selenium", "beautifulsoup4", "requests", "ansicolors", "sqlalchemy", "lxml" ], "requires_python": "", "summary": "A Simple Scraper for Southwest Airlines", "version": "0.0.4" }, "last_serial": 4790017, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "12a2d423165fad8132a869bcdf8a3263", "sha256": "502b596b032f59c4cd141ce4ac4f44dee1e2ac147d907453a2a975db3f35e7a2" }, "downloads": -1, "filename": "scrapeswa-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "12a2d423165fad8132a869bcdf8a3263", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 5769, "upload_time": "2019-02-07T02:51:33", "url": "https://files.pythonhosted.org/packages/6e/97/cd1af7c662ac29a7b08d4dd08751c13580d165998f07d3da9c92c5fb294e/scrapeswa-0.0.1-py3-none-any.whl" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "2f4fb0e682ccb19b1326c11bb172c5da", "sha256": "6f1768a3a2d8e06c2b6722f5334eeb9b413fc2ecdef39b984200038da61c64a5" }, "downloads": -1, "filename": "scrapeswa-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "2f4fb0e682ccb19b1326c11bb172c5da", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7399, "upload_time": "2019-02-07T03:36:26", "url": "https://files.pythonhosted.org/packages/c9/f1/2975e317ab31a1cadd620e30c44a48215afe23904447420d51cc41c3f207/scrapeswa-0.0.2-py3-none-any.whl" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "140238b7c35fbb75fccd4ae192967a46", "sha256": "c84e311460d72ec3090f615f8afc16096b078bff2e677478dbcf50148b3c1c4f" }, "downloads": -1, "filename": "scrapeswa-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "140238b7c35fbb75fccd4ae192967a46", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7400, "upload_time": "2019-02-07T03:37:40", "url": "https://files.pythonhosted.org/packages/ec/9e/8d4635ac7d12aade9eed075e301e5ca9f4f52261f61978b81a7840ca1197/scrapeswa-0.0.3-py3-none-any.whl" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "1249a26595f84d9d4da8c66ded2cf388", "sha256": "a72e3132edc57c7f2507d07e30dc9724051351a05974d2394e6b27eea2916d47" }, "downloads": -1, "filename": "scrapeswa-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "1249a26595f84d9d4da8c66ded2cf388", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7392, "upload_time": "2019-02-07T08:16:02", "url": "https://files.pythonhosted.org/packages/37/94/66000a9291502613b2d524274d899ca1fe2ee98cf7c182f27297709c4907/scrapeswa-0.0.4-py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1249a26595f84d9d4da8c66ded2cf388", "sha256": "a72e3132edc57c7f2507d07e30dc9724051351a05974d2394e6b27eea2916d47" }, "downloads": -1, "filename": "scrapeswa-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "1249a26595f84d9d4da8c66ded2cf388", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7392, "upload_time": "2019-02-07T08:16:02", "url": "https://files.pythonhosted.org/packages/37/94/66000a9291502613b2d524274d899ca1fe2ee98cf7c182f27297709c4907/scrapeswa-0.0.4-py3-none-any.whl" } ] }