{
    "info": {
        "author": "Yelp Security",
        "author_email": "opensource@yelp.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "![osxcollector](https://raw.githubusercontent.com/Yelp/osxcollector/master/osx-github.png)\n\n[![Stories in Ready](https://badge.waffle.io/Yelp/osxcollector.png?label=ready&title=Ready)](https://waffle.io/Yelp/osxcollector)\n[![Stories in In Progress](https://badge.waffle.io/Yelp/osxcollector.png?label=in%20progress&title=In%20Progress)](https://waffle.io/Yelp/osxcollector)\n[![Build Status](https://travis-ci.org/Yelp/osxcollector.svg)](https://travis-ci.org/Yelp/osxcollector)\n[![PyPI](https://img.shields.io/pypi/v/osxcollector.svg)](https://pypi.python.org/pypi/osxcollector)\n\n# OSXCollector Manual\nOSXCollector is a forensic evidence collection & analysis toolkit for OSX.\n\n#### Forensic Collection\nThe collection script runs on a potentially infected machine and outputs a JSON file that describes the target machine. OSXCollector gathers information from plists, SQLite databases and the local file system.\n\n#### Forensic Analysis\nArmed with the forensic collection, an analyst can answer the question like:\n* _Is this machine infected?_\n* _How'd that malware get there?_\n* _How can I prevent and detect further infection?_\n\nYelp automates the analysis of most OSXCollector runs converting its output into an easily readable and actionable summary of _just the suspicious stuff_. Check out [OSXCollector Output Filters project](https://github.com/Yelp/osxcollector_output_filters) to learn how to make the most of the automated OSXCollector output analysis.\n\n## Performing Collection\n[`osxcollector.py`](https://raw.githubusercontent.com/Yelp/osxcollector/master/osxcollector/osxcollector.py) is a single Python file that runs without any dependencies on a standard OSX machine. This makes it really easy to run collection on any machine - no fussing with brew, pip, config files, or environment variables. Just copy the single file onto the machine and run it:\n\n`sudo osxcollector.py` is all it takes.\n\n```shell\n$ sudo osxcollector.py\nWrote 35394 lines.\nOutput in osxcollect-2014_12_21-08_49_39.tar.gz\n```\n\nIf you have just cloned the GitHub repository, `osxcollector.py` is inside `osxcollector/` directory, so you need to run it as:\n\n```shell\n$ sudo osxcollector/osxcollector.py\n```\n\n**IMPORTANT:** please make sure that `python` command on your Mac OS X machine uses the default Python interpreter shipped with the system and is not overridden, e.g. by the Python version installed through brew. OSXCollector relies on a couple of native Python bindings for OS X libraries, which might be not available in other Python versions than the one originally installed on your system.\nAlternatively, you can run `osxcollector.py` explicitly specifying the Python version you would like to use:\n\n```shell\n$ sudo /usr/bin/python2.7 osxcollector/osxcollector.py\n```\n\nThe JSON output of the collector, along with some helpful files like system logs, has been bundled into a .tar.gz for hand-off to an analyst.\n\n`osxcollector.py` also has a lot of useful options to change how collection works:\n* `-i INCIDENT_PREFIX`/`--id=INCIDENT_PREFIX`:\n  Sets an identifier which is used as the prefix of the output file. The default value is `osxcollect`.\n  ```shell\n  $ sudo osxcollector.py -i IncontinentSealord\n  Wrote 35394 lines.\n  Output in IncontinentSealord-2014_12_21-08_49_39.tar.gz\n  ```\n  Get creative with incident names, it makes it easier to laugh through the pain.\n\n* `-p ROOTPATH`/`--path=ROOTPATH`:\n  Sets the path to the root of the filesystem to run collection on. The default value is `/`. This is great for running collection on the image of a disk.\n  ```shell\n  $ sudo osxcollector.py -p '/mnt/powned'\n  ```\n\n* `-s SECTION`/`--section=SECTION`:\n  Runs only a portion of the full collection. Can be specified more than once. The full list of sections and subsections is:\n  * `version`\n  * `system_info`\n  * `kext`\n  * `startup`\n    * `launch_agents`\n    * `scripting_additions`\n    * `startup_items`\n    * `login_items`\n  * `applications`\n    * `applications`\n    * `install_history`\n  * `quarantines`\n  * `downloads`\n    * `downloads`\n    * `email_downloads`\n    * `old_email_downloads`\n  * `chrome`\n    * `history`\n    * `archived_history`\n    * `cookies`\n    * `login_data`\n    * `top_sites`\n    * `web_data`\n    * `databases`\n    * `local_storage`\n    * `preferences`\n  * `firefox`\n    * `cookies`\n    * `downloads`\n    * `formhistory`\n    * `history`\n    * `signons`\n    * `permissions`\n    * `addons`\n    * `extension`\n    * `content_prefs`\n    * `health_report`\n    * `webapps_store`\n    * `json_files`\n  * `safari`\n    * `downloads`\n    * `history`\n    * `extensions`\n    * `databases`\n    * `localstorage`\n    * `extension_files`\n  * `accounts`\n    * `system_admins`\n    * `system_users`\n    * `social_accounts`\n    * `recent_items`\n  * `mail`\n  * `full_hash`\n\n  ```shell\n  $ sudo osxcollector.py -s 'startup' -s 'downloads'\n  ```\n\n* `-c`/`--collect-cookies`:\n  Collect cookies' value.\n  By default OSXCollector does not dump the value of a cookie, as it may contain sensitive information (e.g. session id).\n\n* `-l`/`--collect-local-storage`:\n  Collect the values stored in web browsers' local storage.\n  By default OSXCollector does not dump the values as they may contain sensitive information.\n\n* `-d`/`--debug`:\n  Enables verbose output and python breakpoints. If something is wrong with OSXCollector, try this.\n\n  ```shell\n  $ sudo osxcollector.py -d\n  ```\n\n## Details of Collection\nThe collector outputs a `.tar.gz` containing all the collected artifacts. The archive contains a JSON file with the majority of information.  Additionally, a set of useful logs from the target system logs are included.\n\n#### Common Keys\n\n##### Every Record\nEach line of the JSON file records 1 _piece of information_.  There are some common keys that appear in every JSON record:\n* `osxcollector_incident_id`: A unique ID shared by every record.\n* `osxcollector_section`: The _section_ or type of data this record holds.\n* `osxcollector_subsection`: The _subsection_ or more detailed descriptor of the type of data this record holds.\n\n##### File Records\nFor records representing files there are a bunch of useful keys:\n* `atime`: The file accessed time.\n* `ctime`: The file creation time.\n* `mtime`: The file modified time.\n* `file_path`: The absolute path to the file.\n* `md5`: MD5 hash of the file contents.\n* `sha1`: SHA1 hash of the file contents.\n* `sha2`: SHA2 hash of the file contents.\n\nFor records representing downloaded files:\n* `xattr-wherefrom`: A list containing the source and referrer URLs for the downloaded file.\n* `xattr-quarantines`: A string describing which application downloaded the file.\n\n##### SQLite Records\nFor records representing a row of a SQLite database:\n* `osxcollector_table_name`: The table name the row comes from.\n* `osxcollector_db_path`: The absolute path to the SQLite file.\n\nFor records that represent data associated with a specific user:\n* `osxcollector_username`: The name of the user\n\n#### Timestamps\nOSXCollector attempts to convert timestamps to human readable date/time strings in the format `YYYY-mm-dd hh:MM:ss`. It uses heuristics to automatically identify various timestamps:\n* seconds since epoch\n* milliseconds since epoch\n* seconds since 2001-01-01\n* seconds since 1601-01-01\n\n#### Sections\n##### `version` section\n\nThe current version of OSXCollector.\n\n##### `system_info` section\n\nCollects basic information about the system:\n\n - system name\n - node name\n - release\n - version\n - machine\n\n##### `kext` section\n\nCollects the Kernel extensions from:\n- `/System/Library/Extensions`\n- `/Library/Extensions`\n\n##### `startup` section\n\nCollects information about the\n[LaunchAgents](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man5/launchd.plist.5.html),\nLaunchDaemons, ScriptingAdditions,\n[StartupItems](https://developer.apple.com/library/mac/documentation/macosx/conceptual/bpsystemstartup/chapters/StartupItems.html)\nand other login items from:\n\n - `/System/Library/LaunchAgents`\n - `/System/Library/LaunchDaemons`\n - `/Library/LaunchAgents`\n - `~/Library/LaunchAgents`\n - `/Library/LaunchDaemons`\n - `/System/Library/ScriptingAdditions`\n - `/Library/ScriptingAdditions`\n - `/System/Library/StartupItems`\n - `/Library/StartupItems`\n - `~/Library/Preferences/com.apple.loginitems.plist`\n\nMore information about the Max OS X startup can be found here:\nhttp://www.malicious-streams.com/article/Mac_OSX_Startup.pdf\n\n##### `applications` section\n\nHashes installed applications and gathers install history from:\n\n - `/Applications`\n - `~/Applications`\n - `/Library/Receipts/InstallHistory.plist`\n\n##### `quarantines` section\n\nQuarantines are basically the info necessary to show the 'Are you sure you wanna run this?' when a user is trying to open a file downloaded from the Internet.\nFor some more details, checkout the Apple Support explanation of Quarantines:\nhttp://support.apple.com/kb/HT3662\n\nThis section collects also information from XProtect hash-based malware check\nfor quarantines files. The plist is at:\n`/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/XProtect.plist`\n\nXProtect also add minimum versions for Internet Plugins. That plist is at:\n`/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/XProtect.meta.plist`\n\n##### `downloads` section\n\nHashes all users' downloaded files from:\n\n - `~/Downloads`\n - `~/Library/Mail Downloads`\n - `~/Library/Containers/com.apple.mail/Data/Library/Mail Downloads`\n\n##### `chrome` section\n\nCollects following information from Google Chrome web browser:\n\n - History\n - Archived History\n - Cookies\n - Extensions\n - Login Data\n - Top Sites\n - Web Data\n\nThis data is extracted from `~/Library/Application Support/Google/Chrome/Default`\n\n##### `firefox` section\n\nCollects information from the different SQLite databases in a Firefox profile:\n\n - Cookies\n - Downloads\n - Form History\n - History\n - Signons\n - Permissions\n - Addons\n - Extensions\n - Content Preferences\n - Health Report\n - Webapps Store\n\nThis information is extracted from `~/Library/Application Support/Firefox/Profiles`\n\nFor more details about Firefox profile folder see\nhttp://kb.mozillazine.org/Profile_folder_-_Firefox\n\n##### `safari` section\n\nCollects information from the different plists and SQLite databases in a Safari\nprofile:\n\n - Downloads\n - History\n - Extensions\n - Databases\n - Local Storage\n\n##### `accounts` section\nCollects information about users' accounts:\n\n - system admins: `/private/var/db/dslocal/nodes/Default/groups/admin.plist`\n - system users: `/private/var/db/dslocal/nodes/Default/users`\n - social accounts: `~/Library/Accounts/Accounts3.sqlite`\n - users' recent items: `~/Library/Preferences/com.apple.recentitems.plist`\n\n##### `mail` section\nHashes files in the mail app directories:\n\n - `~/Library/Mail`\n - `~/Library/Mail Downloads`\n\n##### `full_hash` section\nHashes all the files on disk. All of 'em. This does not run by default. It must be triggered with:\n```shell\n$ sudo osxcollector.py -s full_hash\n```\n\n## Basic Manual Analysis\nForensic analysis is a bit of art and a bit of science. Every analyst will see a bit of a different story when reading the output from OSXCollector. That's part of what makes analysis fun.\n\nGenerally, collection is performed on a target machine because something is hinky: anti-virus found a file it doesn't like, deep packet inspect observed a callout, endpoint monitoring noticed a new startup item. The details of this initial alert - a file path, a timestamp, a hash, a domain, an IP, etc. - that's enough to get going.\n\n#### Timestamps\nSimply greping a few minutes before and after a timestamp works great:\n\n```shell\n$ cat INCIDENT32.json | grep '2014-01-01 11:3[2-8]'\n```\n\n#### Browser History\nIt's in there. A tool like [jq](http://stedolan.github.io/jq/) can be very helpful to do some fancy output:\n\n```shell\n$ cat INCIDENT32.json | grep '2014-01-01 11:3[2-8]' | jq 'select(has(\"url\"))|.url'\n```\n\n#### A Single User\n```shell\n$ cat INCIDENT32.json | jq 'select(.osxcollector_username==\"ivanlei\")|.'\n```\n\n## Automated Analysis\nThe [OSXCollector Output Filters project](https://github.com/Yelp/osxcollector_output_filters) contains filters that process and transform the output of OSXCollector. The goal of filters is to make it easy to analyze OSXCollector output.\n\n#### Development Tips\nThe functionality of OSXCollector is stored in a single file: `osxcollector.py`. The collector should run on a naked install of OS X without any additional packages or dependencies.\n\nEnsure that all of the OSXCollector tests pass before editing the source code. You can run the tests using: `make test`\n\nAfter making changes to the source code, run `make test` again to verify that your changes did not break any of the tests.\n\n## License\nThis work is licensed under the GNU General Public License and a derivation of [https://github.com/jipegit/OSXAuditor](https://github.com/jipegit/OSXAuditor)\n\n## Blog post\n\n* [OSXCollector: Forensic Collection and Automated Analysis for OS X](http://engineeringblog.yelp.com/2015/01/osxcollector-forensic-collection-and-automated-analysis-for-os-x.html) by Ivan Leichtling\n\n## Presentations\n\n* [OSXCollector: Automated forensic evidence collection & analysis for OS X](https://www.youtube.com/watch?v=l-lhyPcSd6I) by Kuba Sendor @ BruCON 0x07\n* [Squashing Rotten Apples: Automated forensics & analysis for Mac OS X with OSXCollector](https://www.youtube.com/watch?v=XeeCO8moyeE) by Kuba Sendor @ BSides Manchester 2015\n* [OSXCollector](http://macbrained.org/recap-august15-yelp/) by Ivan Leichtling @ Macbrained's August Meet-Up\n* [OSXCollector - Automated Forensic Evidence Collection & Analysis for OS X](https://www.youtube.com/watch?v=Yqny1rMTfyY) by Ivan Leichtling @ OpenNSM\n* [OSXCollector - Automated Forensic Evidence Collection & Analysis for OS X](https://www.youtube.com/watch?v=DfANq2ncaKU) by Ivan Leichtling @ Duo Tech Talk ([blog post](https://www.duosecurity.com/blog/duo-tech-talk-osxcollector-automated-forensic-evidence-collection-and-analysis-for-os-x))\n\n## External Presentations\n\n* [OSX Archaeology: Becoming Indiana Jones with OSXCollector and Strata](https://www.youtube.com/watch?v=9wvhOoXl2Os) by Chris Henderson & Justin Larson @ SAINTCon 2015\n\n## Resources\nWant to learn more about OS X forensics?\n* [Sarah Edward's mac4n6.com](http://www.mac4n6.com/) - The best presentations on Mac forensics.\n\nA couple of other interesting tools:\n* [KnockKnock](https://github.com/synack/knockknock) - KnockKnock is a command line python script that displays persistent OS X binaries that are set to execute automatically at each boot.\n* [Grr](https://github.com/google/grr) - Google Rapid Response: remote live forensics for incident response\n* [osquery](https://github.com/facebook/osquery) - SQL powered operating system instrumentation, monitoring, and analytics\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/Yelp/osxcollector",
        "keywords": "",
        "license": "GNU General Public License",
        "maintainer": "",
        "maintainer_email": "",
        "name": "osxcollector",
        "package_url": "https://pypi.org/project/osxcollector/",
        "platform": "",
        "project_url": "https://pypi.org/project/osxcollector/",
        "project_urls": {
            "Homepage": "https://github.com/Yelp/osxcollector"
        },
        "release_url": "https://pypi.org/project/osxcollector/1.12/",
        "requires_dist": [
            "macholib (>=1.7)",
            "pyobjc (>=3.0.4)",
            "xattr (>=0.8.0)"
        ],
        "requires_python": "",
        "summary": "A tool for answering \"How'd that malware get there?\"",
        "version": "1.12"
    },
    "last_serial": 5123477,
    "releases": {
        "1.11": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d4c17d8754ceb151e382493becb3aa49",
                    "sha256": "60ba68f7e707e6a9ee5ea9588273c0a8c71891c133862fe4ef2cd927802076df"
                },
                "downloads": -1,
                "filename": "osxcollector-1.11-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "d4c17d8754ceb151e382493becb3aa49",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 22705,
                "upload_time": "2019-04-05T15:00:11",
                "url": "https://files.pythonhosted.org/packages/1a/ba/b4e37ac787712194e81970750a963546cd216e9c7e828afd50f619b13ea2/osxcollector-1.11-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e4824960cf5c8a30ccbe4fd145a23f7d",
                    "sha256": "95a707c080c312493825133ea7a6a6a267b1bb2bb13ec55885466b3031d03752"
                },
                "downloads": -1,
                "filename": "osxcollector-1.11.tar.gz",
                "has_sig": false,
                "md5_digest": "e4824960cf5c8a30ccbe4fd145a23f7d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 22458,
                "upload_time": "2019-04-05T15:00:12",
                "url": "https://files.pythonhosted.org/packages/d3/f3/ec5c71c1e5a5e04217935be3e870bcb08166090a788930f683a5489a874a/osxcollector-1.11.tar.gz"
            }
        ],
        "1.12": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "959fea0021ba4cf2697bae8722d1c073",
                    "sha256": "ccb7e7e064aa1365de37dc1fea7b112f31881ca53f51ca85f8f2741c1177d5c5"
                },
                "downloads": -1,
                "filename": "osxcollector-1.12-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "959fea0021ba4cf2697bae8722d1c073",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 22734,
                "upload_time": "2019-04-10T11:12:24",
                "url": "https://files.pythonhosted.org/packages/81/14/6eef941f882dc1f3b8746e1f76d385c0ee614976d1cc4769c558cc45821a/osxcollector-1.12-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e2343b9aa9d2498956711ef61d79e254",
                    "sha256": "e66ef8f6c7b8e7fe4abbfd23b23477c4690e5e6eae4929adf836d78413d793d8"
                },
                "downloads": -1,
                "filename": "osxcollector-1.12.tar.gz",
                "has_sig": false,
                "md5_digest": "e2343b9aa9d2498956711ef61d79e254",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 23080,
                "upload_time": "2019-04-10T11:12:25",
                "url": "https://files.pythonhosted.org/packages/03/62/e980de55c4b75def24ea2e5f9d4a41f9bc6b4e2c7ca2916de5b6c83f5e73/osxcollector-1.12.tar.gz"
            }
        ],
        "1.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c4c7af07446d9700fcbacf5c259756cd",
                    "sha256": "1160112990df2a2540cd83be86067338ebd3854ac420fd7f6a89c2d1caf36465"
                },
                "downloads": -1,
                "filename": "osxcollector-1.7-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "c4c7af07446d9700fcbacf5c259756cd",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 17021,
                "upload_time": "2015-12-22T17:21:02",
                "url": "https://files.pythonhosted.org/packages/71/a7/49d73ea6a6a7a4891691bd2d20903a992d823cd90ec35fc71a5c04764f78/osxcollector-1.7-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "22184e28b8ac08889fc282de472f6156",
                    "sha256": "ea275d0760afc29709605978d74854a67f589803354e52dea97abe1ab2c9c251"
                },
                "downloads": -1,
                "filename": "osxcollector-1.7.tar.gz",
                "has_sig": false,
                "md5_digest": "22184e28b8ac08889fc282de472f6156",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15551,
                "upload_time": "2015-12-22T17:21:08",
                "url": "https://files.pythonhosted.org/packages/5b/f7/093081faf6371561969d65aec05eb2ee45f9f9c83b774cfb8ae886005642/osxcollector-1.7.tar.gz"
            }
        ],
        "1.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "038a512b33ac719b3135c1ff30639e71",
                    "sha256": "def96c099ef04c560277527d6443414fe25134d3cf46192c5f2a382fccde19a6"
                },
                "downloads": -1,
                "filename": "osxcollector-1.8.tar.gz",
                "has_sig": false,
                "md5_digest": "038a512b33ac719b3135c1ff30639e71",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17640,
                "upload_time": "2016-04-28T21:00:19",
                "url": "https://files.pythonhosted.org/packages/e6/1a/e7c9852dab54c5d9d92c53adf95283a5c78f0088305652f85fa8eb7fcd65/osxcollector-1.8.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "959fea0021ba4cf2697bae8722d1c073",
                "sha256": "ccb7e7e064aa1365de37dc1fea7b112f31881ca53f51ca85f8f2741c1177d5c5"
            },
            "downloads": -1,
            "filename": "osxcollector-1.12-py2-none-any.whl",
            "has_sig": false,
            "md5_digest": "959fea0021ba4cf2697bae8722d1c073",
            "packagetype": "bdist_wheel",
            "python_version": "py2",
            "requires_python": null,
            "size": 22734,
            "upload_time": "2019-04-10T11:12:24",
            "url": "https://files.pythonhosted.org/packages/81/14/6eef941f882dc1f3b8746e1f76d385c0ee614976d1cc4769c558cc45821a/osxcollector-1.12-py2-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "e2343b9aa9d2498956711ef61d79e254",
                "sha256": "e66ef8f6c7b8e7fe4abbfd23b23477c4690e5e6eae4929adf836d78413d793d8"
            },
            "downloads": -1,
            "filename": "osxcollector-1.12.tar.gz",
            "has_sig": false,
            "md5_digest": "e2343b9aa9d2498956711ef61d79e254",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 23080,
            "upload_time": "2019-04-10T11:12:25",
            "url": "https://files.pythonhosted.org/packages/03/62/e980de55c4b75def24ea2e5f9d4a41f9bc6b4e2c7ca2916de5b6c83f5e73/osxcollector-1.12.tar.gz"
        }
    ]
}