{ "info": { "author": "Agustin Barto", "author_email": "abarto@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Software Development :: Libraries" ], "description": "pandas\\_drf\\_tools\n==================\n\nIntroduction\n------------\n\npandas-drf-tools is a set of viewsets, serializers and mixins to allow\nusing `Pandas `__ DataFrames with `Django\nREST Framework `__ sites.\n\nInstallation\n------------\n\nThe package can be installed using ``pip`` from\n`PyPI `__:\n\n::\n\n $ pip install pandas-drf-tools\n\nAn you can also install it from source cloning the project's GitHub\nrepository:\n\n::\n\n $ git clone git://github.com/abarto/pandas-drf-tools.git\n $ cd pandas-drf-tools\n $ python setup.py install\n\nUsage\n-----\n\nHow you use pandas-drf-tools depends on the level of integration you\nneed. The simplest use-case are regular DRF views that expose a\nDataFrame. pandas-drf-tools provides several Serializers that turn a\nDataFrame into its JSON representation using ``to_*`` methods in the\nDataFrame API and a little bit of data processing. You can also parse\n(and validate) data sent to the view into a DataFrame using the provided\nSerializers. For example:\n\n.. code:: python\n\n class DataFrameIndexSerializerTestView(views.APIView):\n def get_serializer_class(self):\n return DataFrameIndexSerializer\n\n def get(self, request, *args, **kwargs):\n sample = get_some_dataframe().sample(20)\n serializer = self.get_serializer_class()(sample)\n return response.Response(serializer.data)\n\n def post(self, request, *args, **kwargs):\n serializer = self.get_serializer_class()(data=request.data)\n serializer.is_valid(raise_exception=True)\n data_frame = serializer.validated_data\n data = {\n 'columns': list(data_frame.columns),\n 'len': len(data_frame)\n }\n return response.Response(data)\n\nThe ``APIView`` above uses ``DataFrameIndexSerializer`` to serialize the\nDataFrame sample on the ``get`` method, and to de-serialize the request\npayload on the ``post`` method. It also provide basic validation. Here's\nthe code for ``DataFrameIndexSerializer``:\n\n.. code:: python\n\n class DataFrameIndexSerializer(Serializer):\n def to_internal_value(self, data):\n try:\n data_frame = pd.DataFrame.from_dict(data, orient='index').rename(index=int)\n return data_frame\n except ValueError as e:\n raise ValidationError({api_settings.NON_FIELD_ERRORS_KEY: [str(e)]})\n\n def to_representation(self, instance):\n instance = instance.rename(index=str)\n return instance.to_dict(orient='index')\n\nAs you can see, the brunt of the work is done by ``DataFrame.to_dict``.\nThese are all the Serializers available:\n\n- DataFrameReadOnlyToDictRecordsSerializer: A read-only (it doesn't\n implement ``to_internal_value``) serializer that uses\n ``DataFrame.to_dict`` with ``records`` orientation.\n- DataFrameListSerializer: A serializer that uses ``DataFrame.to_dict``\n with ``list`` orientation for serialization and ``columns`` for\n de-serialization.\n- DataFrameIndexSerializer: A serializer that uses\n ``DataFrame.to_dict`` with ``index`` orientation for serialization\n and de-serialization. Due to the restrictions imposed on keys by the\n JSON format, the index is converted to ``str`` on serialization and\n to ``int`` on deseralization.\n- DataFrameRecordsSerializer: A serializer that uses\n ``DataFrame.to_records`` for serialization and\n ``DataFrame.from_records`` de-serialization.\n\nBesides serializers, pandas-drf-tools also provides a\n``GenericDataFrameAPIView`` to expose a DataFrame using a view, the same\nway DRF's ``GenericAPIView`` does it with Django's querysets. This class\nwill rarely be used directly. Same as with DRF, pandas-drf-tools also\nprovides a ``GenericDataFrameViewSet`` class that, combined with custom\nlist, retrieve, create, and update mixins turn into ``DataFrameViewSet``\n(and ``ReadOnlyDataFrameViewSet``) which mimics the behaviour of\n``ModelViewSet``.\n\nInstead of setting a ``queryset`` field of overriding ``get_queryset``,\nusers of ``DataFrameViewSet`` need to set a ``dataframe`` field or\noverride the ``get_dataframe`` method. Another difference is that, by\ndefault, write operations do not change the original dataframe. The\n``create``, ``update``, and ``destroy`` methods defined in the mixins\nreturn a **new** DataFrame based on the one set by ``get_dataframe``. In\norder to give the users the chance of doing something with the new\nDataFrame, we provide an ``update_dataframe`` callback that is called\nwhenever a write operation is called. Take a look at the\n``CreateDataFrameMixin`` class:\n\n.. code:: python\n\n class CreateDataFrameMixin(object):\n \"\"\"\n Adds a row to the dataframe.\n \"\"\"\n def create(self, request, *args, **kwargs):\n serializer = self.get_serializer(data=request.data)\n serializer.is_valid(raise_exception=True)\n self.perform_create(serializer)\n headers = self.get_success_headers(serializer.data)\n return Response(serializer.data, status=status.HTTP_201_CREATED, headers=headers)\n\n def perform_create(self, serializer):\n dataframe = self.get_dataframe()\n return self.update_dataframe(dataframe.append(serializer.validated_data))\n\n def get_success_headers(self, data):\n try:\n return {'Location': data[api_settings.URL_FIELD_NAME]}\n except (TypeError, KeyError):\n return {}\n\nWe call ``append`` on the original dataframe and we pass the result onto\n``update_dataframe``. The default behaviour of ``update_dataframe`` is\njust returning whatever was passed onto it, so all operations are\nbasically read-only. Here's an example of how to integrate all the\ncomponents:\n\n.. code:: python\n\n import pandas as pd\n\n class TestDataFrameViewSet(DataFrameViewSet):\n serializer_class = DataFrameRecordsSerializer\n\n def get_dataframe(self):\n return pd.read_pickle('test.pkl')\n\n def update_dataframe(self, dataframe):\n dataframe.read_pickle('test.pkl')\n return dataframe\n\nThis viewset can then be used the same way as regular DRF viewset. For\ninstance, we could use a router:\n\n.. code:: python\n\n from rest_framework.routers import DefaultRouter\n\n router = DefaultRouter()\n router.register(r'test', TestDataFrameViewSet, base_name='test')\n\nThe only caveat is that, since there's no queryset (nor model)\nassociated with the viewset, DRF cannot guess the base name, so it has\nto be set explicitly.\n\nThat's everything you need. Now you API is ready to receive regular REST\ncalls (POST for create, PUT for update, etc.) that will read or change\nthe DataFrame.\n\nWhenever possible, I followed DRF's existing architecture so most things\nshould feel natural if you already have experience with the framework.\n\nExample\n-------\n\nA complete example that uses the US Census Data is available on\n`GitHub `__.\n\nWhat's missing?\n---------------\n\n- No unit tests. Although the package is fully functional, I wouldn't\n use it in any production environment yet as I haven't had time to\n fully test it just.\n- No validation. The serializers just use pandas' methods without\n checking payload thoroughly. I'm still looking for ways on improving\n this, probably using the columns dtypes to validate each serialized\n cell.\n- No filtering backends. If you need filtering, you can override the\n ``filter_dataframe`` method, which is does the same as the\n ``filter_queryset`` method. I'm planning on implementing some filters\n (like the ``SearchFilter``) to provide guidance if you want to build\n your own.\n- No page pagination. Only ``LimitOffsetPagination`` is provided.\n- Proper documentation.\n\nFeedback\n--------\n\nComments, tickets and pull requests are welcomed. You can also reach me\nat `abarto@machinalis.com `__ if you\nhave specific questions.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/abarto/pandas-drf-tools", "keywords": "pandas djangorestframework django", "license": "BSD", "maintainer": null, "maintainer_email": null, "name": "pandas_drf_tools", "package_url": "https://pypi.org/project/pandas_drf_tools/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/pandas_drf_tools/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/abarto/pandas-drf-tools" }, "release_url": "https://pypi.org/project/pandas_drf_tools/0.1.1/", "requires_dist": null, "requires_python": null, "summary": "A set of tools to make Pandas easy to use with Django REST Framework projects", "version": "0.1.1" }, "last_serial": 2408872, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "7ef0e511db23c0c5e906323cbb1c9455", "sha256": "e3ef19c2bb232a80b35feadf49f84a4400ababf7d0b5284bdc9de8b6279a6065" }, "downloads": -1, "filename": "pandas_drf_tools-0.1.0.tar.gz", "has_sig": false, "md5_digest": "7ef0e511db23c0c5e906323cbb1c9455", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9459, "upload_time": "2016-10-05T14:47:18", "url": "https://files.pythonhosted.org/packages/5c/07/3cd9b9168c5e46b960c3dc1cc4618f1d0e54cc361121e81ffdf50423836e/pandas_drf_tools-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "73c9409311fd2dce6d38a4f2bdd63c42", "sha256": "9236f1317b0343e3c8731b7c0bc14c6ff02b064b030acd51bd6c7763157308c6" }, "downloads": -1, "filename": "pandas_drf_tools-0.1.1.tar.gz", "has_sig": false, "md5_digest": "73c9409311fd2dce6d38a4f2bdd63c42", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9468, "upload_time": "2016-10-18T21:58:29", "url": "https://files.pythonhosted.org/packages/c6/9f/86e3d3493846f779778eb8993ebde9322d0f59632afa109ed6352884717c/pandas_drf_tools-0.1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "73c9409311fd2dce6d38a4f2bdd63c42", "sha256": "9236f1317b0343e3c8731b7c0bc14c6ff02b064b030acd51bd6c7763157308c6" }, "downloads": -1, "filename": "pandas_drf_tools-0.1.1.tar.gz", "has_sig": false, "md5_digest": "73c9409311fd2dce6d38a4f2bdd63c42", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9468, "upload_time": "2016-10-18T21:58:29", "url": "https://files.pythonhosted.org/packages/c6/9f/86e3d3493846f779778eb8993ebde9322d0f59632afa109ed6352884717c/pandas_drf_tools-0.1.1.tar.gz" } ] }