{ "info": { "author": "Joshua Nelson, Brady O'Leary, Charles Daniels, and James Coman", "author_email": "jyn514@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Education", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Operating System :: OS Independent", "Programming Language :: Python :: 3 :: Only", "Topic :: Database" ], "description": "# USC-scraping\n[![Build Status](https://travis-ci.org/jyn514/GradeForge.svg?branch=dev)](https://travis-ci.org/jyn514/GradeForge)\n\n[my.sc.edu](https://ssb.onecarolina.sc.edu/BANP/bwskfcls.P_GetCrse) has an\nabsolutely terrible web interface. This is a repository to download courses for\nviewing offline. Support is available for an SQL database. Work on a web\ninterface is [ongoing](https://github.com/jyn514/GradeForge/tree/frontend).\n\n## Requirements\n- GNU `make` (`gmake` for BSD users)\n- `python3`\n- `pip` and modules from `requirements.txt`. if not using a packaged version of `lxml`, you will need\n\t- a working C compiler\n\t- the `python-dev` library\n\t- `libxml2-dev`\n\t- `libxslt-dev`\n- [`pdftotext`](https://poppler.freedesktop.org/) (part of `poppler-utils`)\n- [`chromedriver`](http://chromedriver.chromium.org/)\n- [`tidy`](http://www.html-tidy.org/)\n\n## Goals\n### Long-Term\n- have all the information needed or useful to register on one page. this includes\n\t- RateMyProfessor\n\t- past grade distributions\n\t- schedule planner or an equivalent\n\t- degreeworks\n\t- required textbooks\n\n### Short-Term\n- parse_bookstore has yet to be implemented either in the makefile or in `parse sections`\n- all the `parse` functions should take a boolean `create`\n\t- if true, assert the output file does not exist\n\t- if false, don't write headers\n- the submit button for `index.html` is broken\n- add rules in the makefile for courses in past years.\n\n### Non-Goals\n- registering automatically. this would require storing the *university*\nusernames and passwords of anyone who used the service. this is acceptable\nfor personal use (and feel free to do this, `login.py` is file you're looking\nfor), but absolutely unacceptable for other users.\n\n\n## Usage\n- SQL database: `make`\n- Web server: `make web` or `make server`\n- Dump of everything: `make dump`\n- Unit tests (the few we have): `make test`\n\n## Development\n### Setting up\n1. `git clone https://github.com/jyn514/gradeforge` && cd gradeforge\n2. ln -s ../../scripts/pre-commit .git/hooks\n3. (optional) run `make data` to pre-populate the HTML\n\n### Bugs\n- `parse_section` does not parse days met properly if the times are different\non different days. run `make` on branch `broken` for an example.\n- course['attributes'] is a tuple on `broken`; this crashes `create_sql.py`\n\n### Notes\n- please do not try to use gradeforge directly for parsing,\nthe dependencies will drive you mad. use the beautiful makefile instead.\n- data for grades is available back until 2008, but data for sections is only available until 2013.\n- columns in grades ending in `_GF` stand for 'Grade Forgiveness'\n- png_for won't work for this semester (because the grades haven't been published).\n this sounds stupid but I was wracking my brains trying to figure out why it was broken.\n\n### Types\n- semester: a date in YYYYMM format, where MM is one of 01, 05, 08 and YYYY >= 2008; for example, 201608\n- department: a department name which matches the regex [A-Z]{4}; for example, CSCE\n- code: a class name which matches the regex [A-Z]?[0-9]{3}; for example, 145\n- section: a section identifier which identifies an instance of a class; for example 001\n- uid: a section identifier which is unique within a semester; matches the regex [0-9]{5}. for example, 84495\n\n## Relevant Links\n### Search Pages\n- [Bookstore](http://sc.bncollege.com/webapp/wcs/stores/servlet/TBWizardView?catalogId=10001&langId=-1&storeId=10052)\n- [Sections](https://ssb.onecarolina.sc.edu/BANP/bwckschd.p_disp_dyn_sched)\n- [Sign up for sections](https://ssb.onecarolina.sc.edu/BANP/bwskfreg.P_AltPin)\n\n### Result Examples\n- [Catalog](https://ssb.onecarolina.sc.edu/BANP/bwckctlg.p_disp_course_detail?cat_term_in=201808&subj_code_in=BADM&crse_numb_in=B210)\n- [Bulletin](http://bulletin.sc.edu/preview_course.php?catoid=70&coid=85439)\n- [Section](https://ssb.onecarolina.sc.edu/BANP/bwckschd.p_disp_detail_sched?term_in=201808&crn_in=12566)\n- [Exams](https://www.sc.edu/about/offices_and_divisions/registrar/final_exams/final-exams-spring-2018.php)\n\n### External Links\n- [Login](https://cas.auth.sc.edu/cas/login)\n- [Semester starts and ends](https://my.sc.edu/codes/partofterms/list)\n- [RateMyProfessor](https://www.ratemyprofessors.com/search.jsp?queryBy=schoolId&schoolID=1309)\n- [Schedule Planner](https://sc.collegescheduler.com/entry)\n- [Grade Spreads](https://www.sc.edu/about/offices_and_divisions/registrar/toolbox/grade_processing/grade_spreads/index.php)\n- [Grade Abbreviations](https://www.sc.edu/about/offices_and_divisions/registrar/transcripts_and_records/grade_point_scale/index.php)\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/jyn514/GradeForge", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "gradeforge", "package_url": "https://pypi.org/project/gradeforge/", "platform": "", "project_url": "https://pypi.org/project/gradeforge/", "project_urls": { "Homepage": "https://github.com/jyn514/GradeForge" }, "release_url": "https://pypi.org/project/gradeforge/0.0.1/", "requires_dist": [ "lxml (<=4.2.1)", "requests", "flask", "selenium", "pandas (<=0.23.0)", "matplotlib", "xlsx2csv", "pylint" ], "requires_python": "", "summary": "view available classes for the University of South Carolina", "version": "0.0.1" }, "last_serial": 4178630, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "293575aee7e99af39222679f990ec154", "sha256": "a76aff90a371111e257266d982bd342bcec6826a70fe0aaaec66844f5a5a76d1" }, "downloads": -1, "filename": "gradeforge-0.0.1-py3.6.egg", "has_sig": false, "md5_digest": "293575aee7e99af39222679f990ec154", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": null, "size": 7136228, "upload_time": "2018-08-17T01:27:07", "url": "https://files.pythonhosted.org/packages/54/d8/0d4fc6c4a3c263e53cba0368faa9d0df9b75673b283724b365565c05a639/gradeforge-0.0.1-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "4152a8470990d775639c6b6fad9a3d46", "sha256": "3b9d65acb4ad6eadf7d5f5c8d4c310edca29694b7495d12269d30eac858a4e1a" }, "downloads": -1, "filename": "gradeforge-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "4152a8470990d775639c6b6fad9a3d46", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7137061, "upload_time": "2018-08-17T01:22:28", "url": "https://files.pythonhosted.org/packages/22/f4/d80d015c6d8434fd215c4acb249be7794dc5b905c757032117317e3b4ceb/gradeforge-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a994325191c17c7c2410532ee72b1e0b", "sha256": "1799c45373b68c51e18ebc815ec045615bf4f037786309a5cce438432b54333b" }, "downloads": -1, "filename": "gradeforge-0.0.1.tar.gz", "has_sig": false, "md5_digest": "a994325191c17c7c2410532ee72b1e0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7040069, "upload_time": "2018-08-17T01:27:12", "url": "https://files.pythonhosted.org/packages/ff/92/04e0fac6009094bd6829cdeea3eedc9ccbf2820d900655d68084a0a6727b/gradeforge-0.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "293575aee7e99af39222679f990ec154", "sha256": "a76aff90a371111e257266d982bd342bcec6826a70fe0aaaec66844f5a5a76d1" }, "downloads": -1, "filename": "gradeforge-0.0.1-py3.6.egg", "has_sig": false, "md5_digest": "293575aee7e99af39222679f990ec154", "packagetype": "bdist_egg", "python_version": "3.6", "requires_python": null, "size": 7136228, "upload_time": "2018-08-17T01:27:07", "url": "https://files.pythonhosted.org/packages/54/d8/0d4fc6c4a3c263e53cba0368faa9d0df9b75673b283724b365565c05a639/gradeforge-0.0.1-py3.6.egg" }, { "comment_text": "", "digests": { "md5": "4152a8470990d775639c6b6fad9a3d46", "sha256": "3b9d65acb4ad6eadf7d5f5c8d4c310edca29694b7495d12269d30eac858a4e1a" }, "downloads": -1, "filename": "gradeforge-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "4152a8470990d775639c6b6fad9a3d46", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7137061, "upload_time": "2018-08-17T01:22:28", "url": "https://files.pythonhosted.org/packages/22/f4/d80d015c6d8434fd215c4acb249be7794dc5b905c757032117317e3b4ceb/gradeforge-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a994325191c17c7c2410532ee72b1e0b", "sha256": "1799c45373b68c51e18ebc815ec045615bf4f037786309a5cce438432b54333b" }, "downloads": -1, "filename": "gradeforge-0.0.1.tar.gz", "has_sig": false, "md5_digest": "a994325191c17c7c2410532ee72b1e0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7040069, "upload_time": "2018-08-17T01:27:12", "url": "https://files.pythonhosted.org/packages/ff/92/04e0fac6009094bd6829cdeea3eedc9ccbf2820d900655d68084a0a6727b/gradeforge-0.0.1.tar.gz" } ] }