{ "info": { "author": "Ian Gow", "author_email": "ian.gow@unimelb.edu.au", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "## WRDS to PG Migration\nThis software has two functions:\n- Download tables from wrds and uploads to PG. \n- Upload sas file (`*.sas7dbat`) to PG.\n\n#### 1. Python\nThe software uses Python 3 and depends on Pandas, SQLAlchemy and Paramiko.\n\n#### 2. WRDS Settings\nSet `WRDS_ID` with `wrds_id=your_wrds_id`, otherwise the software will grep from OS environment variables.\n\nTo use public-key authentication to access WRDS, follow hints [here](https://debian-administration.org/article/152/Password-less_logins_with_OpenSSH). First set up a public key, then copied that key to the WRDS server from terminal. \n\n(Note that this code assumes you have a directory `.ssh` in your home directory. If not, log into WRDS via SSH, then type `mkdir ~/.ssh` to create this.) \n\nHere's code to create the key and send it to WRDS.\n\n```\nssh-keygen -t rsa\ncat ~/.ssh/id_rsa.pub | ssh your_wrds_id@wrds-cloud.wharton.upenn.edu \"cat >> ~/.ssh/authorized_keys\"\n```\nUse an empty passphrase in setting up my key so that the scripts can run without user intervention.\n\n#### 3. PG Settings\nIf you have set `PGHOST`, `PGDATABASE`, `PGUSER` as environment variables, the software can grep them. Otherwise, users are expected to specify them when using `wrds_udpate()`. Default `PGPORT` is`5432`.\n\nTwo variables `table` and `schema` are required.\n\n#### 4. Table Settings\nTo tailor tables, specify the following variables:\n\n`fix_missing`: set to `True` to fix missing values. Default value is `False`. \n\n`fix_cr`: set to `True` to fix characters. Default value is `False`.\n\n`drop`: add column names to be dropped.eg.`drop=\"id name\"` will drop column `id` and `name`.\n\n`obs`: add maxium number of observations. eg.`obs=10` will export the top 10 rows from the table.\n\n`rename`: rename columns. eg.`rename=\"fee=mngt_fee\"` rename `fee` to `mngt_fee`.\n\n`force`: set to `True` to force update. Default value is `False`.\n\n#### 5. Upload SAS File\nThe software can also upload SAS file directly to PG. You need to have local SAS in order to use this function.\n\nUse `fpath` to specify file path.\n\n#### 6. Examples\nHere are some examples.\n\n```py\nfrom wrdstopg import wrdstopg\n\n# 1. Download crsp.mcti from wrds and upload to pg as crps.mcti\n# Simplest version\nwrdstopg.wrds_update(table=\"mcti\", schema=\"crsp\")\n# Tailor table to your needs\nwrdstopg.wrds_update(table=\"mcti\", schema=\"crsp\", host=your_pghost, dbname=your_pg_database, fix_missing=True, \n\tfix_cr=True, drop=\"b30ret b30ind\", obs=10, rename=\"caldt=calendar_date\", force=True)\n\n# 2. Upload test.sas7dbat to pg as crsp.mcti\nwrdstopg.wrds_update(table=\"mcti\", schema=\"crsp\", fpath=\"your_path/test.sas7dbat\")\n```\n\n#### 7. Report Bugs\nAuthor: Ian Gow, \n\nContributor: Jingyu Zhang, \n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/iangow-public/wrds_pg/", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "wrdstopg", "package_url": "https://pypi.org/project/wrdstopg/", "platform": "", "project_url": "https://pypi.org/project/wrdstopg/", "project_urls": { "Homepage": "https://github.com/iangow-public/wrds_pg/" }, "release_url": "https://pypi.org/project/wrdstopg/0.0.9/", "requires_dist": [ "pandas", "sqlalchemy", "paramiko" ], "requires_python": ">=3", "summary": "Download wrds tables and upload to PostgreSQL, upload SAS file to PG", "version": "0.0.9" }, "last_serial": 4403467, "releases": { "0.0.9": [ { "comment_text": "", "digests": { "md5": "2822b152a6d9f8eb9bec25e7e5d9d7fc", "sha256": "b24bcdffcdb770aa7e80602ce616b816b1ea2759bbfe012830ba42650047dd0c" }, "downloads": -1, "filename": "wrdstopg-0.0.9-py3-none-any.whl", "has_sig": false, "md5_digest": "2822b152a6d9f8eb9bec25e7e5d9d7fc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 6697, "upload_time": "2018-10-22T19:35:25", "url": "https://files.pythonhosted.org/packages/4b/8b/1f8fd05b4057db7fd5e4ebe49e46b1fd9da8a53b0c9fdda2bd01cacf12cf/wrdstopg-0.0.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b6c81394011d239d738c1649db7b5e80", "sha256": "6486d90bc21a43d1b2ef165feafe1ea49140311c44b1cb61e45c435bafa92f22" }, "downloads": -1, "filename": "wrdstopg-0.0.9.tar.gz", "has_sig": false, "md5_digest": "b6c81394011d239d738c1649db7b5e80", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 6391, "upload_time": "2018-10-22T19:35:27", "url": "https://files.pythonhosted.org/packages/e1/95/c722440ad083dc0e162c59afec9b0a207dc35ea9ec82f15be2e1ef041ecb/wrdstopg-0.0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2822b152a6d9f8eb9bec25e7e5d9d7fc", "sha256": "b24bcdffcdb770aa7e80602ce616b816b1ea2759bbfe012830ba42650047dd0c" }, "downloads": -1, "filename": "wrdstopg-0.0.9-py3-none-any.whl", "has_sig": false, "md5_digest": "2822b152a6d9f8eb9bec25e7e5d9d7fc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3", "size": 6697, "upload_time": "2018-10-22T19:35:25", "url": "https://files.pythonhosted.org/packages/4b/8b/1f8fd05b4057db7fd5e4ebe49e46b1fd9da8a53b0c9fdda2bd01cacf12cf/wrdstopg-0.0.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b6c81394011d239d738c1649db7b5e80", "sha256": "6486d90bc21a43d1b2ef165feafe1ea49140311c44b1cb61e45c435bafa92f22" }, "downloads": -1, "filename": "wrdstopg-0.0.9.tar.gz", "has_sig": false, "md5_digest": "b6c81394011d239d738c1649db7b5e80", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3", "size": 6391, "upload_time": "2018-10-22T19:35:27", "url": "https://files.pythonhosted.org/packages/e1/95/c722440ad083dc0e162c59afec9b0a207dc35ea9ec82f15be2e1ef041ecb/wrdstopg-0.0.9.tar.gz" } ] }