{
    "info": {
        "author": "Mike Malinowski",
        "author_email": "mike@twisted.space",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python"
        ],
        "description": "# Overview\n\n\nFracture is a lightweight and flexible data management system. It allows\nyou to interact with data through a trait compositing mechanism, whilst\nalso exposing the ability to quickly query and access information\nabout the data you're exploring.\n\n\n# How it works\n\n\nYou start by creating a fracture project. The project file is where all\nthe metadata and look up tables are stored - allowing you to easily search\nfor data assets as well as find changes.\n\nFracture comes built-in with a file searching mechanism, but you can extend\nthis with your own search mechanisms too. For instance, if you have data\non an FTP, or within Source Control and you want to add that data to the\nproject without having to have it physically on disk you're able to do so\nby implementing a fracture.ScanProcess plugin.\n\nFinally, and probably the most important is the DataElement. This is a class\nwhich you can use to express the functionality of data. Rather than having a\n1:1 relationship between a DataElement class and a data type the DataElement\nclass supports class compositing. This allows for a piece of data to be\nrepresented by more than one class simultaneously.\n\n\n# Examples\n\n\nThis example uses the ```dino_example``` data which you can download from\nhttps://github.com/mikemalinowski/fracture. \n\nTo start with, we create a fracture project. To do this we must specify\ntwo pieces of information, the first being where we want to save our \nproject file and the second being the locations where we want fracture\nto look for Scan and Data Plugins.\n\n```python\nimport os\nimport fracture\n\nproject = fracture.create(\n    project_path=os.path.join(current_path, '_dino.fracture'),\n    plugin_locations=[os.path.join(current_path, 'plugins')]\n)\n```\n\nThis returns a fracture.Project instance which we can then start interacting\nwith, for instance we can define locations where the project should start\nlooking for data:\n\n```python\n# -- Tell the project where to look for data\nproject.add_scan_location('/usr/my_data'))\n```\n\nFinally, with the project made, and at least one search location added we\ncan initiate a search...\n\n```python\n# -- Now we initiate a scan. This will cycle over all the\n# -- scan locations and scrape them for data\nproject.scan()\n```\n\nScanning is the process of running over all the scan plugins - of which there\nis always at least one (the file scraper), and populating the project with\ninformation about each piece of data which is found. The process is pretty\nquick and the amount of data stored is minimal - primarily just the identifier\nsuch as the path along with any tags as defined by any DataElement composites\nwhich can represent that data.\n\nWith the project populated we can now start querying the project for\ndata\n\n```python\n# -- Now we have scanned we can start to run queries over data\n# -- very quickly. We can search by tags, or use the special\n# -- * wildcard\nfor item in project.find('*'):\n\n    # -- By default we get string identifiers back from a find, as\n    # -- this is incredibly quick. However, we can ask for the data\n    # -- composite representation of the item. A data composite is\n    # -- a composition of all the data plugins which can represent\n    # -- this identifier.\n    item = project.get(item)\n\n    # -- Print the identifier, and the item (which also shows the\n    # -- class composition)\n    print(item.identifier())\n    print('\\t%s' % item)\n\n    # -- We can start interacting with this data, calling\n    # -- functionality will return a dictionary of all the\n    # -- functionality exposed by all the data plugins representing\n    # -- this item\n    for k, v in item.functionality().items():\n        print('\\t\\t%s = %s' % (k, v))\n```\n\nThe process of querying is very quick, even for reasonably large data sets. In\nthe example above we're then asking the project to 'get' the item. This process\ntake the identifier and binds all the relevent DataElements together which\ncan possibly represent the data.\n\nBinding is particularly useful when there is no obvious hierarchy between\ntwo elements. For instance, in the ```dino_example``` data set we have a\ntrait which is ```carnivore``` and a trait which is ```herbivore```. There\nis no hierarchical relationship between the two, but an omnivore would need\nboth. By using class compositing we avoid complex multi-inheritence situations.\n\nUsing this same mechanism, if we know the locator of a piece of information,\nsuch as a file path, we can get the composited class directly without having\nto run a query, as shown here:\n\n```python\n# -- We do not have to utilise the find method to get access to data,\n# -- and in fact we can get a Composite representation of data even\n# -- if that data is not within our scan location.\ndata = project.get('/usr/my_data/my_file.txt')\n```\n\nFor a full demo, download the ```dino_example``` and run main.py\n\n\n# Data Composition\n\n\nAs mentioned in the examples, we use class composition to bind traits together\nto represent data. This means we can have small, self contained traits which\ndo not need rigid hierarchical structures designed for them.\n\nThere are three main composited methods in the DataElement class, specifically:\n\n* label : The first call that returns a positive result is taken\n* mandatory_tags : All the lists are collated from all compositions and made unique\n* functionality : All dictionaries are combined into a single dictionary\n* icon : The first call that returns a positive result is taken\n\nGiven the ```dino_example``` files, the velociraptor.png file, when passed\nto ```project.get('/usr/my_data/.../velociraptor.png')``` is expressed\nas a class formed of the following traits: [Carnivore; File; Image;] where each\ntrait can expose its own information.\n\nAn implementation of a DataElement plugin looks like this:\n\n```python\nimport re\nimport fracture\n\n# -- All plugins must inherit from the fracture.DataElement class in order\n# -- to be dynamically picked up.\nclass CarnivoreTrait(fracture.DataElement):\n\n    # -- The data type is mandatory, and is your way of\n    # -- denoting the name of this plugin\n    data_type = 'carnivore'\n\n    # -- These two lines are not at all required and are here\n    # -- just to make performance better\n    _split = re.compile('/|\\.|,|-|:|_', re.I)\n    _has_trait = re.compile('(carnivore|omnivore).*\\.', re.I)\n\n    # --------------------------------------------------------------------------\n    # -- This method must be re-implemented, and its your oppotunity to\n    # -- decide whether this plugin can viably represent the given data\n    # -- identifier.\n    # -- In this example we use a regex check, but it could be anything\n    # -- you want. The key thing to remember is that this is called a lot,\n    # -- so performance is worth keeping in mind.\n    @classmethod\n    def can_represent(cls, identifier):\n        if CarnivoreTrait._has_trait.search(identifier):\n            return True\n        return False\n\n    # --------------------------------------------------------------------------\n    # -- This is your way of exposing functionality in a common and consistent\n    # -- way. If you know the data types you can of course call things directly\n    # -- but this is a good catch up consistent way of exposing functionality\n    # -- and is typically harnessed by user interfaces.\n    def functionality(self):\n        return dict(\n            feed_meat=self.feed_meat,\n            ),\n        )\n\n    # --------------------------------------------------------------------------\n    # -- This should return a 'nice name' for the identifier\n    def label(self):\n        return os.path.basename(self.identifier())\n\n    # --------------------------------------------------------------------------\n    # -- As fracture heavily utilises tags, this is your way of defining a\n    # -- set of tags which are mandatory for anything with this trait\n    def mandatory_tags(self):\n        return ['carnivore', 'meat', 'hunter']\n\n    # --------------------------------------------------------------------------\n    # -- This is here just as a demonstration of a callable function which\n    # -- which can be accessed on the trait\n    def feed_meat(self):\n        print('Would feed this creature some meat...')\n```\n\nBy placing a trait plugin anywhere within the plugin locations you define\nfor your project will immediately make it accessible.\n\n\n## ScanProcess\n\n\nBy default fracture comes with one built-in scan plugin which handles file\nscanning, so that is a good example when wanting to write your own - if you \nhave need to do so.\n\nThis plugin type defines how to find data. If your data is files on a disk\nsuch as those in the example above then your scan plugin may do little more\nthan cycle directories and yield file paths.\n\nAlternatively if you're caching data from a REST API you might be utilising\nrequests within the scan process and feeding back URL's.\n\n\n# Origin\n\n\nThis library is a variation on the tools demonstrated during\nGDC2018 (A Practical Approach to Developing Forward-Facing Rigs, Tools and\nPipelines), which can be explored in more detail here:\nhttps://www.gdcvault.com/play/1025427/A-Practical-Approach-to-Developing\n\nSlide 55 onward explores this concept. It is also explored in detail\non this webpage:\nhttps://www.twisted.space/blog/insight-localised-asset-management\n\n\n# Collaboration\n\nI am always open to collaboration, so if you spot bugs lets me know, or if\nyou would like to contribute or get involved just shout!\n\n\n# Compatibility\n\nLaunchpad has been tested under Python 2.7 and Python 3.7 on Windows and Ubuntu.",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/mikemalinowski/fracture",
        "keywords": "fracture data composite",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "fracture",
        "package_url": "https://pypi.org/project/fracture/",
        "platform": "",
        "project_url": "https://pypi.org/project/fracture/",
        "project_urls": {
            "Homepage": "https://github.com/mikemalinowski/fracture"
        },
        "release_url": "https://pypi.org/project/fracture/0.9.2/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Fracture is a lightweight and flexible data management system",
        "version": "0.9.2"
    },
    "last_serial": 5010298,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d80e91b83ebc3e18f5140190609c8bb3",
                    "sha256": "75583297549e7e2b85bc3bca652e69ab83a95814a9d33130ce7d6bd7c6eafab7"
                },
                "downloads": -1,
                "filename": "fracture-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "d80e91b83ebc3e18f5140190609c8bb3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1703,
                "upload_time": "2019-03-27T13:50:33",
                "url": "https://files.pythonhosted.org/packages/8f/28/c6f3ba3066de9483290470166f149cce265e171d3dc7ca7075f338046ec7/fracture-0.0.1.tar.gz"
            }
        ],
        "0.9.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "54a0af35f26166b1f754858452b4bc7a",
                    "sha256": "aac7c8822d05bdaef822b025c2c0062aee9d83eb86a6b6e9ec08a7a080e814c8"
                },
                "downloads": -1,
                "filename": "fracture-0.9.1.tar.gz",
                "has_sig": false,
                "md5_digest": "54a0af35f26166b1f754858452b4bc7a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 116943,
                "upload_time": "2019-03-30T21:50:35",
                "url": "https://files.pythonhosted.org/packages/af/27/da20030ca2b3d6fed772fabc2368b5c5c8d7a73a98a0713d6ad6f27f1b7a/fracture-0.9.1.tar.gz"
            }
        ],
        "0.9.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a4f41c2b98963de51871a55bdc3684d7",
                    "sha256": "ab30e20d329d503272293024ae11991dd7a53108a27e4f20068dcc27f6db3ebf"
                },
                "downloads": -1,
                "filename": "fracture-0.9.2.tar.gz",
                "has_sig": false,
                "md5_digest": "a4f41c2b98963de51871a55bdc3684d7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 117532,
                "upload_time": "2019-03-31T20:16:26",
                "url": "https://files.pythonhosted.org/packages/72/2f/beeff94ed93e2ed03d6932261e5d64b97186f02b821cbdcb1f158420d53e/fracture-0.9.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "a4f41c2b98963de51871a55bdc3684d7",
                "sha256": "ab30e20d329d503272293024ae11991dd7a53108a27e4f20068dcc27f6db3ebf"
            },
            "downloads": -1,
            "filename": "fracture-0.9.2.tar.gz",
            "has_sig": false,
            "md5_digest": "a4f41c2b98963de51871a55bdc3684d7",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 117532,
            "upload_time": "2019-03-31T20:16:26",
            "url": "https://files.pythonhosted.org/packages/72/2f/beeff94ed93e2ed03d6932261e5d64b97186f02b821cbdcb1f158420d53e/fracture-0.9.2.tar.gz"
        }
    ]
}