{ "info": { "author": "RegioHelden GmbH", "author_email": "entwicklung@regiohelden.de", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Framework :: Django", "Framework :: Django :: 1.11", "Framework :: Django :: 2.0", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Natural Language :: English", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Security", "Topic :: Software Development" ], "description": "# Django Scrubber\n\n[![Build Status](https://travis-ci.org/RegioHelden/django-scrubber.svg?branch=master)](https://travis-ci.org/RegioHelden/django-scrubber)\n[![PyPI](https://img.shields.io/pypi/v/django-scrubber.svg)](https://pypi.org/project/django-scrubber/)\n\n`django_scrubber` is a django app meant to help you anonymize your project's database data. It destructively alters data directly on the DB and therefore **should not be used on production**.\n\nThe main use case is providing developers with realistic data to use during development, without having to distribute your customers' or users' potentially sensitive information.\nTo accomplish this, `django_scrubber` should be plugged in a step during the creation of your database dumps.\n\nSimply mark the fields you want to anonymize and call the `scrub_data` management command. Data will be replaced based on different *scrubbers* (see below), which define how the anonymous content will be generated.\n\n## Installation\n\nSimply run:\n```\npip install django-scrubber\n```\n\nAnd add `django_scrubber` to your django `INSTALLED_APPS`. I.e.: in `settings.py` add:\n```\nINSTALLED_APPS = [\n ...\n 'django_scrubber',\n ...\n]\n```\n\n## Scrubbing data\n\nIn order to scrub data, i.e.: to replace DB data with anonymized versions, `django-scrubber` must know which models and fields it should act on, and how the data should be replaced.\n\nThere are a few different ways to select which data should be scrubbed, namely: explicitly per model field; or globally per name or field type.\n\nAdding scrubbers directly to model, matching scrubbers to fields by name:\n```python\nclass MyModel(Model):\n somefield = CharField()\n\n class Scrubbers:\n somefield = scrubbers.Hash('somefield')\n```\n\nAdding scrubbers globally, either by field name or field type:\n\n```python\n# (in settings.py)\n\nSCRUBBER_GLOBAL_SCRUBBERS = {\n 'name': scrubbers.Hash,\n EmailField: scrubbers.Hash,\n}\n```\n\nModel scrubbers override field-name scrubbers, which in turn override field-type scrubbers.\n\nTo disable global scrubbing in some specific model, simply set the respective field scrubber to `None`.\n\nWhich mechanism will be used to scrub the selected data is determined by using one of the provided scrubbers in `django_scrubber.scrubbers`. See below for a list.\nAlternatively, values may be anything that can be used as a value in a `QuerySet.update()` call (like `Func` instances, string literals, etc), or any `callable` that returns such an object when called with a `Field` object as argument.\n\nBy default, `django_scrubber` will affect all models from all registered apps. This may lead to issues with third-party apps if the global scrubbers are too general. This can be avoided with the `SCRUBBER_APPS_LIST` setting. Using this, you might for instance split your `INSTALLED_APPS` into multiple `SYSTEM_APPS` and `LOCAL_APPS`, then set `SCRUBBER_APPS_LIST = LOCAL_APPS`, to scrub only your own apps.\n\nFinally just run `./manage.py scrub_data` to **destructively** scrub the registered fields.\n\n## Built-In scrubbers\n\n### Hash\n\nSimple hashing of content:\n```python\nclass Scrubbers:\n somefield = scrubbers.Hash # will use the field itself as source\n someotherfield = scrubbers.Hash('somefield') # can optionally pass a different field name as hashing source\n```\n\nCurrently this uses the MD5 hash which is supported in a wide variety of DB engines. Additionally, since security is not the main objective, a shorter hash length has a lower risk of being longer than whatever field it is supposed to replace.\n\n### Lorem\n\nSimple scrubber meant to replace `TextField` with a static block of text. Has no options.\n```python\nclass Scrubbers:\n somefield = scrubbers.Lorem\n```\n\n### Concat\n\nWrapper around `django.db.functions.Concat` to enable simple concatenation of scrubbers. This is useful if you want to ensure a fields uniqueness through composition of, for instance, the `Hash` and `Faker` (see below) scrubbers. \n\nThe following will generate random email addresses by hashing the user-part and using `faker` for the domain part:\n```python\nclass Scrubbers:\n email = scrubbers.Concat(scrubbers.Hash('email'), models.Value('@'), scrubbers.Faker('domain_name'))\n```\n\n### Faker\n\nReplaces content with the help of [faker](https://pypi.python.org/pypi/Faker).\n\n```python\nclass Scrubbers:\n first_name = scrubbers.Faker('first_name')\n last_name = scrubbers.Faker('last_name')\n```\n\nThe replacements are done on the database-level and should therefore be able to cope with large amounts of data with reasonable performance.\n\nThe `Faker` scrubber accepts a single required argument: the faker provider used to generate random data. All [faker providers](https://faker.readthedocs.io/en/latest/providers.html) are supported and you can also register your own custom providers.\n\n#### Locales\n\nFaker will be initialized with the current django `LANGUAGE_CODE` and will populate the DB with localized data. If you want localized scrubbing, simply set it to some other value.\n\n#### Idempotency\n\nBy default, the faker instance used to populate the DB uses a fixed random seed, in order to ensure different scrubbings of the same data generate the same output. This is particularly useful if the scrubbed data is imported as a dump by developers, since changing data during troubleshooting would otherwise be confusing.\n\nThis behaviour can be changed by setting `SCRUBBER_RANDOM_SEED=None`, which ensures every scrubbing will generate random source data.\n\n#### Limitations\n\nScrubbing unique fields may lead to `IntegrityError`s, since there is no guarantee that the random content will not be repeated. Playing with different settings for `SCRUBBER_RANDOM_SEED` and `SCRUBBER_ENTRIES_PER_PROVIDER` may alleviate the problem.\nUnfortunately, for performance reasons, the source data for scrubbing with faker is added to the database, and arbitrarily increasing `SCRUBBER_ENTRIES_PER_PROVIDER` will significantly slow down scrubbing (besides still not guaranteeing uniqueness).\n\n## Settings\n\n### `SCRUBBER_GLOBAL_SCRUBBERS`:\nDictionary of global scrubbers. Keys should be either field names as strings or field type classes. Values should be one of the scrubbers provided in `django_scrubber.scrubbers`. \n\nExample:\n```python\nSCRUBBER_GLOBAL_SCRUBBERS = {\n 'name': scrubbers.Hash,\n EmailField: scrubbers.Hash,\n}\n```\n\n### `SCRUBBER_RANDOM_SEED`:\nThe seed used when generating random content by the Faker scrubber. Setting this to `None` means each scrubbing will generate different data.\n\n(default: 42)\n\n### `SCRUBBER_ENTRIES_PER_PROVIDER`:\nNumber of entries to use as source for Faker scrubber. Increasing this value will increase the randomness of generated data, but decrease performance. \n\n(default: 1000)\n\n### `SCRUBBER_SKIP_UNMANAGED`:\nDo not attempt to scrub models which are not managed by the ORM.\n\n(default: True)\n\n### `SCRUBBER_APPS_LIST`:\nOnly scrub models belonging to these specific django apps. If unset, will scrub all installed apps.\n\n(default: None)\n\n### `SCRUBBER_ADDITIONAL_FAKER_PROVIDERS`:\nAdd additional fake providers to be used by Faker. Must be noted as full dotted path to the provider class.\n\n(default: empty list) \n\n## Making a new release\n\n[bumpversion](https://github.com/peritus/bumpversion) is used to manage releases.\n\nAdd your changes to the [CHANGELOG](./CHANGELOG.md) and run `bumpversion `, then push (including tags)\n\n\n# Changelog\nAll notable changes to this project will be documented in this file.\n\nThe format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)\nand this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).\n\n\n\n## [0.3.1] - 2018-09-10\n### Fixed\n- [#9](https://github.com/RegioHelden/django-scrubber/pull/9) `Hash` scrubber choking on fields with `max_length=None` - Thanks to [Charlie Denton](https://github.com/meshy)\n\n## [0.3.0] - 2018-09-06\n### Added\n- Finally added some basic tests (thanks [Marco De Felice](https://github.com/md-f))\n- `Hash` scrubber can now also be used on sqlite\n\n### Changed\n- **BREAKING**: scrubbers that are lazily initialized now receive `Field` instances as parameters, instead of field\n names. If you have custom scrubbers depending on the previous behavior, these should be updated. Accessing the\n field's name from the object instance is trivial: `field_instance.name`. E.g.: if you have `some_field = MyCustomScrubber`\n in any of your models' `Scrubbers`, this class must accept a `Field` instance as first parameter.\n Note that explicitly intializing any of the built-in scrubbers with field names is still supported, so if you were\n just using built-in scrubbers, you should not be affected by this change.\n- related to the above, `FuncField` derived classes can now do connection-based setup by implementing the\n `connection_setup` method. This is mostly useful for doing different things based on the DB vendor, and is used to\n implement `MD5()` on sqlite (see added feature above)\n- Ignore proxy models when scrubbing (thanks [Marco De Felice](https://github.com/md-f))\n- Expand tests to include python 3.7 and django 2.1\n\n## [0.2.1] - 2018-08-14\n### Added\n- Option to scrub only one model from the management command\n- Support loading additional faker providers by config setting SCRUBBER\\_ADDITIONAL\\_FAKER\\_PROVIDERS\n\n### Changed\n- Switched changelog format to the one proposed on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)\n\n## [0.2.0] - 2018-08-13\n### Added\n- scrubbers.Concat to make simple concatenation of scrubbers possible\n\n## [0.1.4] - 2018-08-13\n### Changed\n- Make our README look beautiful on PyPI\n\n## [0.1.3] - 2018-08-13\n### Fixed\n- [#1](https://github.com/RegioHelden/django-scrubber/pull/1) badly timed import - Thanks to [Charlie Denton](https://github.com/meshy)\n\n## [0.1.2] - 2018-06-22\n### Changed\n- Use bumpversion and travis to make new releases\n- rename project: django\\_scrubber \u2192 django-scrubber\n\n## [0.1.0] - 2018-06-22\n### Added\n- Initial release", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/regiohelden/django-scrubber", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "django-scrubber", "package_url": "https://pypi.org/project/django-scrubber/", "platform": "", "project_url": "https://pypi.org/project/django-scrubber/", "project_urls": { "Homepage": "https://github.com/regiohelden/django-scrubber" }, "release_url": "https://pypi.org/project/django-scrubber/0.3.1/", "requires_dist": null, "requires_python": "", "summary": "Data Anonymizer for Django", "version": "0.3.1" }, "last_serial": 4257433, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "7127a9c8f433c86326796aed8a1a51a6", "sha256": "2f8d20c8c26b0263b7b3324557b4b1f1cac9b02b3f51900203004668f4c6f40f" }, "downloads": -1, "filename": "django-scrubber-0.1.0.tar.gz", "has_sig": false, "md5_digest": "7127a9c8f433c86326796aed8a1a51a6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12803, "upload_time": "2018-06-22T14:59:29", "url": "https://files.pythonhosted.org/packages/54/64/e174a9339244b7fbc23e2e3a48738c413cc8a8a5ef4cf9d57614c5c52379/django-scrubber-0.1.0.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "1d021f622a1f17a27d45e22411d420e1", "sha256": "7c086fdbdf33d702da7a64e847d9eee89750a915b669d1c986a62411d201f40f" }, "downloads": -1, "filename": "django-scrubber-0.1.2.tar.gz", "has_sig": false, "md5_digest": "1d021f622a1f17a27d45e22411d420e1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8935, "upload_time": "2018-06-22T15:30:23", "url": "https://files.pythonhosted.org/packages/87/f4/30ce6c16065bcb55f400c1ccdc8192e01f8b0a4192bb4da7773be6dd82cf/django-scrubber-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "6a960e4505dbff0a021f91df9e4ef695", "sha256": "88c31ad60f5ca6acf204211672eae360604ab245fd081ff5613f9bd1716e96e1" }, "downloads": -1, "filename": "django-scrubber-0.1.3.tar.gz", "has_sig": false, "md5_digest": "6a960e4505dbff0a021f91df9e4ef695", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9167, "upload_time": "2018-08-12T20:14:26", "url": "https://files.pythonhosted.org/packages/a4/71/8f82b3558167af82f1411370d401b6a8840225ec6b02a34455706dc3093c/django-scrubber-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "c48f8c420c01e32824b73ff3b9008c9b", "sha256": "3da4b401a4b31a5ab3ed90a208e8d7d5073136139ae86908ce26c03e00fb6961" }, "downloads": -1, "filename": "django-scrubber-0.1.4.tar.gz", "has_sig": false, "md5_digest": "c48f8c420c01e32824b73ff3b9008c9b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9248, "upload_time": "2018-08-12T20:24:20", "url": "https://files.pythonhosted.org/packages/b4/ec/787bfa419492aeeea2f16df10a0377bfc8640b65ae5f2bc26a59638b43ef/django-scrubber-0.1.4.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "0deab5798d31e5f93281ef8f21d28e62", "sha256": "b1a80d9c071ac8ef2b7f135cd84cb9506e9dc43b53573e86a9a9d7f6c98a205b" }, "downloads": -1, "filename": "django-scrubber-0.2.0.tar.gz", "has_sig": false, "md5_digest": "0deab5798d31e5f93281ef8f21d28e62", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10394, "upload_time": "2018-08-13T16:07:23", "url": "https://files.pythonhosted.org/packages/68/fb/7557204800febff0f3dad8009701a494bbc910060cda66f0be4e9252875a/django-scrubber-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "e1396c39fa17856866834c83a732327b", "sha256": "73f36f3559dca72f5809a95ecd83704e1d0c7e0c8380efcebfa352eed185af3d" }, "downloads": -1, "filename": "django-scrubber-0.2.1.tar.gz", "has_sig": false, "md5_digest": "e1396c39fa17856866834c83a732327b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11171, "upload_time": "2018-08-14T17:00:39", "url": "https://files.pythonhosted.org/packages/13/6b/f79836585af87ca5f3ae8a04331c246813abe92748631841cb0f8eeea97a/django-scrubber-0.2.1.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "d7130135305c8a16902507ffe158a940", "sha256": "04063973b02e0d75e2c7d6be80f026b5fe5ce4a833f4588acbe8dd79cbb6111f" }, "downloads": -1, "filename": "django-scrubber-0.3.0.tar.gz", "has_sig": false, "md5_digest": "d7130135305c8a16902507ffe158a940", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12917, "upload_time": "2018-09-07T10:44:20", "url": "https://files.pythonhosted.org/packages/5b/e6/aee22659a898827a7845bdee767c965d322eb3c15c42f037efa1821b9e5d/django-scrubber-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "cf3b6312a8d5a4441ef0debdcc3f4c70", "sha256": "fe5c760f22b61edddb0f2515cdcbabaca5ac0d92747b31583bfb1d55df2de197" }, "downloads": -1, "filename": "django-scrubber-0.3.1.tar.gz", "has_sig": false, "md5_digest": "cf3b6312a8d5a4441ef0debdcc3f4c70", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12980, "upload_time": "2018-09-10T12:14:30", "url": "https://files.pythonhosted.org/packages/48/a1/2a5d03094c42414ff83f8592c64f3acf5673e259ad05af4508656d87b8fe/django-scrubber-0.3.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "cf3b6312a8d5a4441ef0debdcc3f4c70", "sha256": "fe5c760f22b61edddb0f2515cdcbabaca5ac0d92747b31583bfb1d55df2de197" }, "downloads": -1, "filename": "django-scrubber-0.3.1.tar.gz", "has_sig": false, "md5_digest": "cf3b6312a8d5a4441ef0debdcc3f4c70", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12980, "upload_time": "2018-09-10T12:14:30", "url": "https://files.pythonhosted.org/packages/48/a1/2a5d03094c42414ff83f8592c64f3acf5673e259ad05af4508656d87b8fe/django-scrubber-0.3.1.tar.gz" } ] }