{ "info": { "author": "Nick Stern, Vincent Viego, Summer Yuan, Zach Wehrwein", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# Dotua Documentation\n\n### Nick Stern, Vincent Viego, Summer Yuan, Zach Wehrwein\n\n## Introduction\nCalculus, according to the American mathematician Michael Spivak in his noted textbook, is fundamentally the study of \"infinitesimal change.\" An infinitesimal change, according to Johann Bernoulli as Spivak quotes, is so tiny that \"if a quantity is increased or decreased by an infinitesimal, then that quantity is neither increased nor decreased.\" The study of these infinitesimal changes is the study of relationships of change, not the computation of change itself. The derivative is canonically found as function of a limit of a point as it approaches 0 -- we care about knowing the relationship of change, not the computation of change itself.\n\nOne incredibly important application of the derivative is varieties of optimization problems. Machines are able to traverse gradients iteratively through calculations of derivatives. However, in machine learning applications, it is possible to have millions of parameters for a given neural net and this would imply a combinatorially onerous number of derivatives to compute analytically. A numerical Newton's method approach (iteratively calculating through guesses of a limit) is likewise not a wise alternative because even for \"very small\"\n, the end result can be orders of magnitude off in error relative to machine precision.\n\nSo, one might think that a career in ML thus requires an extensive calculus background, but, Ryan P Adams, formerly of Twitter (and Harvard IACS), now of Princeton CS, describes automatic differentiation as [\"getting rid of the math that gets in the way of solving a [ML] problem.\"](https://www.youtube.com/watch?v=sq2gPzlrM0g) What we ultimately care about is tuning the hyperparameters of a machine learning algorithm, so if we can get a machine to do this for us, that is ultimately what we care about. What is implemented in this package is automatic differentiation which allows us to calculate derivatives of complex functions to machine precision 'without the math getting in the way.'\n\nOne of the most crucial applications of auto-differentiation is backpropagation in neural networks. Backpropagation is the process by which weights are optimized relative to a loss function. These steps are illustrated through a simple neural network example.\n\n## Background\nThe most important calculus derivative rule for automatic differentiation is the multivariate chain rule.\n\nThe basic chain rule states that the derivative of a composition of functions is:\n\n\n\nThat is, the derivative is a function of the incremental change in the outer function applied to the inner function, multiplied by the change in the inner function.\n\nIn the multivariate case, we can apply the chain rule as well as the rule of total differentiation. For instance, if we have a simple equation:\n\n\n\nThen,\n\n\n\nThe partial derivatives:\n\n\n\n\n\nThe total variation of y depends on both the variations in u, v and thus,\n\n\n\nWhat this trivial example illustrates is that the derivative of a multivariate function is ultimately the addition of the partial derivatives and computations of its component variables. If a machine can compute any given sub-function as well as the partial derivative between any sub-functions, then the machine need only add-up the product of a function and its derivatives to calculate the total derivative.\n\n[An intuitive way of understanding automatic differentiation is to think of any complicated function as ultimately a a graph of composite functions.](http://colah.github.io/posts/2015-08-Backprop/) Each node is a primitive operation -- one in which the derivative is readily known -- and the edges on this graph -- the relationship of change between any two variables -- are partial derivatives. The sum of the paths between any two nodes is thus the partial derivative between those two functions (this a graph restatement of the total derivative via the chain rule).\n\n### Forward Mode\n\nForward mode automatic differentiation thus begins at an input to a graph and sums the source paths. The below diagrams (from Christopher Olah's blog) provide an intuition for this process. The relationship between three variables (X, Y, Z) is defined by a number of paths (). Forward mode begins with a seed of 1, and then in each node derivative is the product of the sum of the previous steps.\n\n![](images/chain-def-greek.png)\n\n![](images/chain-forward-greek.png)\n\nConsequently, provided that within each node there is an elementary function, the machine can track the derivative through the computational graph.\n\nThere is one last piece of the puzzle: dual numbers which extend the reals by restating each real as , where . Symbolic evaluation, within a machine, can quickly become computational untenable because the machine must hold in memory variables and their derivatives in the successive expansions of the rules of calculus. Dual numbers allow us to track the derivative of even a complicated function, as a kind of data structure that caries both the derivative and the primal of a number.\n\nIn our chain rule equation, there are two pieces to the computation: the derivative of the outer function applied to the inner function and that value multiplied by the derivative of the inner function. This means that the full symbolic representation of an incredibly complicated function can grow to exponentially many terms. However, dual numbers allow us to parse that symbolic representation in bitesized pieces that can be analytically computed. [The reason for this is the Taylor series expansion of a function](http://jliszka.github.io/2013/10/24/exact-numeric-nth-derivatives.html):\n\n\n\nWhen one evaluates , given that , then all the higher order terms drop out (they are 0) and one is left with \n\n### Reverse Mode\n\nOne intuition for reverse mode auto differentiation is to consider again our chain-rule-as-graph notion.\n\nOne intuitive motivation for this (h/t [Rufflewind](https://rufflewind.com/2016-12-30/reverse-mode-automatic-differentiation)) is to think of reverse mode as an inversion of the chain-rule.\n\nIn this notation, the derivative for some output variable w to some variable *t* is a linear combination of derivatives for each u_i that w is connected to.\n\n![](images/forward_cr.png)\n\nBy flipping the numerator and the denominator, this is the notation of the partial derivative of a new parameter s with respect to the input variable u.\n\n![](images/backward_cr.png)\n\nThe graph theory intuition is perhaps the most straightforward: just as a machine computes an equation in a series of steps, reverse mode is the traversal of that computational graph in reverse. In the context of a neural network in which we wish to minimize weights relative to some loss function, this allows us to efficiently retrace our path without attempting to reevaluate the weights of what could be an incredibly complex network.\n\nTo recap: automatic differentiation is an algorithmic means of computing complicated derivatives by parsing those functions as a graph structures to be traversed. Dual numbers are used as a sort of mathematical data structure which allows the machine to analytically compute the derivative at any given node. It is superior to analytic or symbolic differentiation because it is actually computationally feasible on modern machines! And it is superior to numerical methods because automatic differentiation is far more accurate (it achieves machine precision). It is therefore extremely useful for applications like backpropagation on neutral networks.\n\n## How to Use Dotua\n\n### How to Install\nTo install our package, one can simply use pip install like so:\n\n```bash\n$ pip install Dotua\n```\n\n### Import and Usage Examples\n#### Forward Mode\nIn order to instantiate a forward mode auto-differentiation object from our package, the user shall first import the AutoDiff function from the Dotua library as such:\n\n```py\nfrom Dotua.autodiff import AutoDiff as ad\n```\n\nThe general workflow for the user is as follows:\n- Instantiate all variables as AutoDiff objects.\n- Input these variables into operators from the **Operator** class within the\nDotua library to create more complex expressions that propagate the derivative\nusing forward mode automatic differentiation.\n\nThe **AutoDiff** class is the core constructor for all variables in the function\nthat are to be differentiated. There are two options for instantiating\nvariables: **Scalar** and **Vector**, generated with *create_scalar()* and *create_vector()* respectively. **Scalar** variables have a single value per\nvariable, while **Vector** variables can have multiple associated values. The\ngeneral schematic for how the user shall instantiate **AutoDiff** objects is\noutlined below:\n\n1. Create either a **Scalar** or **Vector AutoDiff** object to generate seed\nvariables to later build the function to be differentiated. The initialization\nworks as follows:\n\n```python\nx, y = ad.create_scalar(vals = [1, 2])\nz = ad.create_vector(vals = [1, 2, 3])\n```\n\n2. Next, the user shall import the **Operator** class and pass in these\nvariables into elementary functions as follows:\n\n```python\nfrom Dotua.operator import Operator as op\nresult = op.sin(x * y)\nresults = op.sin(z)\n```\n\nSimple operators, such as sums and products, can be used normally:\n```python\nresult = 6 * x\nresults = z + 4\n```\n\n3. Finally, (as a continuation of the previous example), the user may access the value and derivative of a function using the *eval()* and *partial()* methods:\n\n```python\nprint(result.eval()) # 6\nprint(result.partial(x)) # 6\nprint(result.partial(y)) # 0\n```\nFor **Scalar** variables, *result.eval()* will return the value of the function,\nwhile *result.partial(v)* will return the partial derivative with respect to any variable, *v*. For **Vector** variables, *results.eval()* returns a list of tuples\n(value, jacobian), with one tuple for each function in the vector. The jacobian is a dictionary that represents the\nderivative of that element with respect to all of the elements in the vector.\n\n#### Reverse Mode\n\nThe initialization for reverse mode variables is very similar to forward mode.\nThe only difference is that there is an \"r\" in front of the module names. Additionally, for the initialization of reverse mode\nvariables, the user must instantiate an initializer object. This differs\nfrom the forward mode variables which can be initialized using static methods.\nOne can initialize a reverse mode scalar object as follows:\n\n```python\nfrom Dotua.rautodiff import rAutoDiff \nrad = rAutoDiff()\nx, y, z = rad.create_rscalar([1, 2, 3])\n```\n\nIn reverse mode, when the user calls the gradient function, they must specify\nthe variable they would like to differentiate with respect to. This time, the\ngradient function simply returns a numeric constant. An example of this is shown\nbelow:\n\n```python\nf = x + y + z\nf_gradx = rad.partial(f, x) # f_gradx = 1\n```\n\nThe following code shows how the user may interact with rVector. Note that rVector operates\ndifferently in reverse mode, as it is mainly an extension to allow one to compute rScalar \nfunctions for a vector of values. \n\n```python\nv = rad.create_rvector([1, 2 ,3])\ng = 2*v\ng_grad = rad.partial(g, v) # g_grad = [2, 2, 2]\n```\n\n### Examples\nThere are several files in the top level directory of the Dotua package that demonstrate the usage of the package. \n\nThe first file is an interactive jupyter notebook which contains an example use\ncase where the Dotua package performs well, namely, the Newton-Raphson method for approximating roots of functions. This notebook is titled \"newton_demo.ipynb\" and resides in \"examples\" folder in the top level directory of the package. The output of this demo is reproduced here for convenience:\n\n![](images/newton.png)\n\nA second file is an example of how our reverse mode auto differentiation package can be used to do backpropagation in a neural network. The file is called \"neuralnet_demo.ipynb\"\n\nFor some output y_hat and set of inputs X, the task of a neural network is to find a function which minimizes a loss function, like MSE:\n\n![](images/mse.png)\n\nWhere N is the number of data points, f_i the value returned by the model and y_i the true value for y at a given observation i. A 'dense' neural network will involve connections between every parameter and these edges each contain a weight value. The task is to find the weights which minimize the distance between y_hat and y. Because each node is connected to every other, this captures non-linearities as certain parameters may be more expressive than others. However, this also carries computational burdens.\n\nBackwards auto-differentiation is particularly useful means of traversing the graph and refitting those weights.\n\nHere we implement a toy neural network that has only one hidden layer. Using R.A. Fischer's well-known Iris dataset -- a collection of measurements of flowers and their respective species -- we predict species based on their floral measurements. As the first scatter plot indicates, there is a clear separation boundary between the setosa plants and the other two species, versicolor and virginica. To capture the boundary between each species would require more complexity, and so our simple neural network can only capture the biggest boundary -- a consequence of our package using reverse auto differentiation to fit the appropriate gradient. This is not a neural networks package, but an auto-differentiation package and this is just an illustration of one very useful application.\n\n![](images/iris_pca.png)\n\n![](images/cm_nnexample.png)\n\n## Software Organization\n\n### Directory Structure\n\nOur project will adhere to the directory structure outlined in the [python-packaging documentation](https://python-packaging.readthedocs.io/en/latest/index.html). At a high level, the project has the following structure:\n\n```python\nDotua/\n __init__.py\n autodiff.py\n operator.py\n rautodiff.py\n roperator.py\n nodes/\n __init__.py\n node.py\n rscalar.py\n rvector.py\n scalar.py\n vector.py\n tests/\n __init__.py\n test_initializer.py\n test_operator.py\n test_rautodiff.py\n test_roperator.py\n test_rscalar.py\n test_scalar.py\n test_vector.py\ndocs/\n documentation.md\n milestone1.md\n milestone2.md\nexamples/\n __init__.py\n newton_demo.py\n neural_network_demo.py\n ...\nLICENSE\nMANIFEST.in\nREADME.md\nrequirements.txt\nsetup.py\n.gitignore\n```\n\n### Modules\n\n#### Dotua/\nThe **Dotua** module contains the codes for forward mode implementation and reverse mode implementation.\n\nIt contains *AutoDiff* (autodiff.py), which is the driver of the forward mode autodifferentiation. The driver helps the users with getting access to the *Node* superclass (node.py) and associated subclasses (i.e., *Vector* (vector.py) and *Scalar* (scalar.py)) in the *nodes* file, and the *Operator* class (operator.py).\n\nIt also contains *rAutoDiff* (rautodiff.py), which is the driver of the reverse mode autodifferentiation. The driver helps the users with getting access to the *rScalar* class (rscalar.py) and *rVector* class (rvector.py) in the *nodes* file and the *rOperator* class (roperator.py).\n\n#### examples/\nThe **Examples** module has Python files with documented use cases of the library. Examples include an implementation of Newton\u2019s Method for approximating the roots of a non-linear function and a module which computes local extrema and an implementation of Neural Network for prediction problems.\n\n#### tests/\nThe **Tests** module contains the project\u2019s testing suite and is formatted according to the pytest requirements for automatic test discovery.\n\n### Testing\n\n#### Overview\nThe majority of the testing in Dotua's test suite consists of unit testing.\nThe aim is to verify the correctness of the application with thorough unit\ntesting of all simple usages of the forward and reverse modes\nof automatic differentiation. Essentially, this involves validating that our application produces correct calculations (evaluations and derivatives) for all elementary functions. Additionally, a range of more complex unit testing covers advanced scenarios such as functions with multidimensional domains and codomains\n(for forward mode) as well as functions with inherent complexity generated from\nthe composition of elementary functions.\n\n#### Test Automation\nDotua uses continuous integration testing through **Travis CI** to perform\nautomated, machine independent testing. Additionally, Dotua uses **Coveralls**\nto validate the high code coverage of our testing suite (currently 100%).\nTravis CI and Coveralls badges are embedded into the project README to provide transparency for users interacting with our project through GitHub.\n\n#### Installation\nTo install our package, one can simply use **pip install** like so:\n\n```bash\n$ pip install Dotua\n```\n\n#### User Verification\nThe entire Dotua test suite is included in the project distribution. Thus,\nusers are able to verify correctness for themselves using pytest after\ninstalling the Dotua package.\n\n### Distribution\n\n#### Licensing\nDotua is distributed under the GNU GPLv3 license to allow free \u201cas is\u201d usage\nwhile requiring all extensions to remain open source.\n\n\n## Implementation\n\nThe purpose of the Dotua library is to perform automatic differentation\non user defined functions, where the domain and codomain may be single- or\nmulti-dimensional (*n.b. this library provides support for both the forward\nand reverse modes of automatic differentation, but for the reverse mode only\nfunctions with single-dimensional codomains are supported*). At a high level,\nDotua serves as a partial replacement for NumPy in the sense that\nDotua provides methods for many of the mathematical functions\n(e.g., trigonometric, inverse trigonometric, hyperbolic, etc.) that NumPy\nimplements; however, while the NumPy versions of these methods can only provide function evaluation, the Dotua equivalents provide both evaluation and differentiation.\n\nTo achieve this, the Dotua library implements the following abstract\nideas:\n 1. Allowing users to be as expressive as they would like to be by providing\n our own versions of binary and unary operators.\n 2. Forward AD: keeping track of the value and derivative of user defined\n expressions and functions.\n 3. Reverse AD: constructing a computational graph from user defined functions\n that can be used to quickly compute gradients.\n\nWith these goals in mind, the Dotua forward mode implementation relies on\nthe **Nodes** modules and the **Operator** class and allows user interface\nthrough the **AutoDiff** class which serves as a **Node** factory for initializing instances of *Scalar* and *Vector*. Analogously, the Dotua reverse mode\nimplementation relies on the **Nodes** module and the **rOperator** class\nand facilitates user interface through the **rAutoDiff** class which serves as a\nfactory for initializing instances of *rScalar*.\n\n## Nodes\nThe **Nodes** module contains a *Node* superclass with the following basic design:\n\n```python\nclass Node():\n def eval(self):\n '''\n For the Scalar and Vector subclasses, this function returns the node's\n value as well as its derivative, both of which are guaranteed to be\n up to date by the class' operator overloads.\n\n Returns (self._val, self._jacobian)\n '''\n raise NotImplementedError\n\n def __add__(self, other):\n raise NotImplementedError\n\n def __radd__(self, other):\n raise NotImplementedError\n\n def __mul__(self, other):\n raise NotImplementedError\n\n def __rmul__(self, other):\n raise NotImplementedError\n\n ... # Additional operator overloads\n```\n\nEssentially, the role of the *Node* class (which in abstract terms is meant to\nrepresent a node in the computational graph underlying forward mode automatic differentiation of user defined expressions) is to serve as an interface for two\nother classes in the **Nodes** package: *Scalar*, *Vector*, *rScalar*, *rVector*. Each of these subclasses implements the required operator overloading as necessary for scalar\nand vector functions respectively (i.e., addition, multiplication, subtraction, division, power, etc.). This logic is separated into four separate classes to\nprovide increased organization for higher dimensional functions and to allow\nclass methods to use assumptions of specific properties of scalars and vectors\nto reduce implementation complexity.\n\nBoth the *Scalar* class and the *Vector* class have *_val* and *_jacobian* class attributes which allow for forward automatic differentiation by keeping track of\neach node's value and derivative. Both the *rScalar* class and the *rVector* class have *_roots* and *_grad_val* class attributes which allow for\nreverse auto differentiation by storing the computational graph and intermediate gradient value.\n\n\n\n### Scalar\nThe *Scalar* class is used for user defined one-dimensional variables.\nSpecifically, users can define functions of scalar variables (i.e., functions\ndefined over multiple scalar variables with a one-dimensional codomain) using\ninstances of *Scalar* in order to simultaneously calculate the function value\nand first derivative at a pre-chosen point of evaluation using forward mode\nautomatic differentiation. Objects of the *Scalar* class are initialized with a\nvalue (i.e., the point of evaluation) which is stored in the class attribute **self._val** (n.b., as the single underscore suggests, this attribute should\nnot be directly accessed or modified by the user). Additionally, *Scalar*\nobjects \u2013 which could be either individual scalar variables or expressions of\nscalar variables \u2013 keep track of their own derivatives in the class attribute **self._jacobian**. This derivative is implemented as a dictionary with *Scalar* objects serving as the keys and real numbers as values. Note that each *Scalar* object's **_jacobian** attribute has an entry for all scalar variables which the\nobject might interact with (see AutoDiff Initializer section for more\ninformation).\n\nUsers interact with *Scalar* objects in two ways:\n1. **eval(self)**: This method allows users to obtain the value\nfor a *Scalar* object at the point of evaluation defined when the user first\ninitialized their *Scalar* objects. Specifically, this method returns the float **self._val**.\n2. **partial(self, var)**: This method allows users to obtain a partial\nderivative of the given *Scalar* object with respect to **var**. If **self**\nis one of the *Scalar* objects directly initialized by the user (see AutoDiff Initializer section), then **partial()** returns 1 if **var == self** and 0\notherwise. If **self** is a *Scalar* object formed by an expression of other\n*Scalar* objects (e.g., **self = Scalar(1) + Scalar(2)**), then this method returns the correct partial\nderivative of **self** with respect to **var**.\n\nNote that these are the only methods that users should be calling for *Scalar*\nobjects and that users should not be directly accessing any of the object's\nclass attributes.\n\n*Scalar* objects support left- and right-sided addition, subtraction,\nmultiplication, division, exponentiation, and negation.\n\n### rScalar\n\nThe *rScalar* class is the reverse mode automatic differentiation analog of\nthe *Scalar* class. That is to say, *rScalar* is used for user defined\none-dimensional variables with which users can define scalar functions (i.e.,\nfunctions defined over multiple scalar variables with a one-dimensional\ncodomain) in order to calculate the function value and later easily determine\nthe function's gradient reverse mode automatic differentiation (see the\nrAutoDiff Initializer section for more details). Objects of the *rScalar*\nclass are initialized with a value (i.e., the point of evaluation) which is\nstored in the class attribute **self._val** (n.b., as the\nsingle underscore suggests, this attribute should not be directly accessed or\nmodified by the user). Additionally, *rScalar* objects \u2013 which could be either individual scalar variables or expressions of scalar variables \u2013 explicitly\nconstruct the computational graph used in automatic differentiation in the class\nattribute **self.roots()**. This attribute represents the computational graph\nas a dictionary where keys represent the children and the values represent the derivatives of the children \nwith respect to *rScalar* *self*. This dictionary\nis constructed explicitly through operator overloading whenever the user\ndefines functions using *rScalar* objects.\n\nUsers interact with *rScalar* objects in one way:\n1. **eval(self)**: This method allows users to obtain the value\nfor an *rScalar* object at the point of evaluation defined when the user first\ninitialized the *rScalar* object. Specifically, this method returns the value\nof the attribute **self._val**.\n\nNote that this is the only method that users should be calling for *rScalar*\nobjects and that users should not be directly accessing any of the object's\nclass attributes. While the *rScalar* class contains a *gradient()* method,\nthis method is for internal use only. Users should only be obtaining the\nderivatives of functions represented by *rScalar* objects through the\n*partial()* method provided in the *rAutoDiff* initializer class (see rAutoDiff\nInitializer section for more details).\n\n*rScalar* objects support left- and right-sided addition, subtraction,\nmultiplication, division, exponentiation, and negation.\n\n\n### Vector\n*Vector* is a subclass of *Node*. Every vector variable consists of a 1-d numpy array to store the values and a 2-d numpy array to store the jacobian matrix.\nUser can use index to acess specific element in a *Vector* instance. And operations between elements in the same vector instance and operations between vectors are implemented by overloading the operators of the class.\n\n## AutoDiff Initializer\n\nThe AutoDiff class functions as a **Node** factory, allowing the user to initialize\nvariables for the sake of constructing arbitrary functions. Because the **Node**\nclass serves only as an interface for the *Scalar* and *Vector* classes, users\nshould not instantiate objects of the *Node* class directly. Thus, we\ndefine the *AutoDiff* class in the following way to allow users to initialize\n*Scalar* and *Vector* variables:\n\n```Python\nfrom Dotua.nodes.scalar import Scalar\nfrom Dotua.nodes.vector import Vector\n\nclass AutoDiff():\n @staticmethod\n def create_scalar(vals):\n '''\n @vals denotes the evaluation points of variables for which the user\n would like to create Scalar variables. If @vals is a list,\n the function returns a list of Scalar variables with @vals\n values. If @vals is a single value, the user receives a single Scalar\n variable (not as a list). This function also initializes the jacobians\n of all variables allocated.\n '''\n pass\n\n @staticmethod\n def create_vector(vals):\n '''\n The idea is similar to create_scalar.\n This will allow the user to create vectors and specify initial\n values for the elements of the vectors.\n '''\n pass\n```\n\nUsing the *create_scalar* and *create_vector* methods, users are able to\ninitialize variables for use in constructing arbitrary functions. Additionally,\nusers are able to specify initial values for these variables. Creating variables\nin this way will ensure that users are able to use the Dotua defined\noperators to both evaluate functions and compute their derivatives.\n\n### Variable Universes\n\nThe implementaiton of the AutoDiff library makes the following assumption:\nfor each environment in which the user uses autodifferentiable variables\n(i.e., *Scalar* and *Vector* objects), the user initializes all such variables\nwith a single call to **create_scalar** or **create_vector**. This assumption\nallows *Scalar* and *Vector* objects to fully initialize their jacobians before\nbeing used by the user. This greatly reduces implementation complexity.\n\nThis design choice should not restrict users in their construction of arbitrary\nfunctions for the reason that in order to define a function, the user must\nknow how many primitive scalar variables they need to use in advance. Realize\nthat this does not mean that a user is prevented from defining new Python\nvariables as functions of previously created *Scalar* objects, but only that a\nuser, in defining a mathematical function **f(x, y, z)** must initialize\n**x, y, and z** with a single call to **create_scalar**. It is perfectly\nacceptable that in the definition of **f(x, y, z)** a Python variable such as\n**a = x + y** is created. The user is guaranteed that **a.eval()** and\n**a.partial(x), a.partial(y), and a.partial(z)** are all well defined and correct because **a** in this case is an instance of *Scalar*; however, it is not a\n\"primitive\" scalar variable and thus the user could not take a partial\nderivative with respect to **a**.\n\n## rAutoDiff Initializer\n\nThe rAutoDiff class functions as an **rScalar** factory, allowing the user to\ninitialize variables for the sake of constructing arbitrary functions of which\nthey want to later determine the derivative using reverse mode automatic\ndifferentiation. Because the same *rScalar* variables can be used to define\nmultiple functions, users must instantiate an rAutoDiff object to manage\nthe *rScalar* objects they create and calcuate the gradients of different\nfunctions of the same variables. Thus, we define the *rAutoDiff* class in the following ways:\n\n```Python\nfrom Dotua.nodes.rscalar import rScalar\n\nclass rAutoDiff():\n def __init__(self):\n self.func = None\n\n def create_rscalar(vals):\n '''\n @vals denotes the evaluation points of variables for which the user\n would like to create rScalar variables. If @vals is a list,\n the function returns a list of rScalar variables with @vals\n values. If @vals is a single value, the user receives a single rScalar\n variable (not as a list). This function also adds the new rScalar\n object(s) to the _universe of the rAutoDiff object.\n '''\n pass\n\n def partial(self, func, var):\n '''\n This method allows users to calculate the derivative of @func the\n function of rScalar objects with respect to the variable represented\n by the rScalar @var. This method also sets the self.func attribute\n of the rAutoDiff object to the given @func.\n '''\n pass\n\n def _reset_universe(self, func, var):\n '''\n This method is for internal use only. When a user calls partial(),\n the rAutoDiff object will first call _reset_universe() to reset then\n grad_val variables of the necessary rScalar objects before\n calculating the desired derivative.\n '''\n pass\n```\n\nBy instantiating an *rAutoDiff* object and using the *create_rscalar* method,\nusers are able to initialize variables for use in constructing arbitrary\nfunctions. Additionally, users are able to specify initial values for these\nvariables. Creating variables in this way will ensure that users are able to\nuse the Dotua defined operators to both evaluate functions and compute their\nderivatives. Furthermore, using the *partial* method, users are able to\ndetermine the derivative of their constructed function with respect to\na specified *rScalar* variable.\n\n## Operator\n\nThe *Operator* class defines static methods for elementary mathematical\nfunctions and operators (specifically those that cannot be overloaded in the\n*Scalar* and *Vector* classes) that can be called by users in constructing arbitrary functions. The *Operator* class will import the Nodes module in order to\nreturn new *Scalar* or *Vector* variables as appropriate. The design of the\n*Operator* class is as follows:\n\n```Python\nimport numpy as np\nfrom Dotua.nodes.scalar import Scalar\nfrom Dotua.nodes.vector import Vector\n\nclass Operator():\n @staticmethod\n def sin(x):\n pass\n\n @staticmethod\n def cos(x):\n pass\n\n ... # Other elementary functions\n```\n\nFor each method defined in the *Operator* class, our implementation uses\nducktyping to return the necessary object. If user passes a *Scalar* object\nto one of the methods then a new *Scalar* object is returned to the user\nwith the correct value and jacobian. If user passes a *Vector* object\nto one of the methods then a new *Vector* object is returned to the user\nwith the correct value and jacobian. On the other hand, if the user passes\na Python numeric type, then the method returns the evaluation of the\ncorresponding NumPy method on the given argument\n(e.g., **op.sin(1) = np.sin(1)**).\n\n## rOperator\n\nSimilarly, the *rOperator* class defines static methods for elementary mathematical\nfunctions and operators (specifically those that cannot be overloaded in the\n*rScalar* class) that can be called by users in constructing arbitrary functions. The design of the\n*rOperator* class is as follows:\n\n```Python\nimport numpy as np\nfrom Dotua.nodes.rscalar import rScalar\nfrom Dotua.nodes.rvector import rVector\n\nclass rOperator():\n @staticmethod\n def sin(x):\n pass\n\n @staticmethod\n def cos(x):\n pass\n\n ... # Other elementary functions\n```\n\nOnce again, for each method defined in the *rOperator* class, our implementation uses\nducktyping to return the necessary object. If user passes an *rScalar* object\nto one of the methods, then a new *rScalar* object is returned to the user\nwith the correct value and self/child link. If user passes an *rVector* object\nto one of the methods, then a new *rVector* object is returned to the user\nwith the correct value and parent/child links. On the other hand, if the user passes\na Python numeric type, then the method returns the evaluation of the\ncorresponding NumPy method on the given argument\n(e.g., **rop.sin(1) = np.sin(1)**).\n\n\n\n## A Note on Comparisons\n\nIt is important to note that the Dotua library intentionally does not overload\ncomparison operators for its variables class (i.e., *Scalar*, *rScalar*,\n*Vector*, and *rVector*). Users should only use the the equality and inequality operators == and !=\nto determine object equivalence. For users wishing to perform comparisons on\nthe values of functions or variables composed of *Scalar*, *rScalar*,\n*Vector*, or *rVector* variables with the values of functions or variables of the same type,\nthey can do so by accessing the values with the **eval()** function.\n\n\n## External Depencies\n\nDotua restricts dependencies on third-party libraries to the necessary\nminimum. Thus, the only external dependencies are NumPy, and SciPy\nNumPy is used as necessary within the library for mathematical computation (e.g., trigonometric functions). \nSciPy is used within the Newton-Raphson Demo as a comparison.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/VV-NS-CY-ZW-CS-207-Organization/cs207-FinalProject", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "Dotua", "package_url": "https://pypi.org/project/Dotua/", "platform": "", "project_url": "https://pypi.org/project/Dotua/", "project_urls": { "Homepage": "https://github.com/VV-NS-CY-ZW-CS-207-Organization/cs207-FinalProject" }, "release_url": "https://pypi.org/project/Dotua/1.0.0/", "requires_dist": [ "numpy", "scipy" ], "requires_python": "", "summary": "Package for Forward/Reverse Autodifferentiation", "version": "1.0.0" }, "last_serial": 4589872, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "1fcd6df07efadab9d7cb2a10d7386434", "sha256": "735184277ab7d628e799ed737117e676b29def32e764d09641bd39b10cbde645" }, "downloads": -1, "filename": "Dotua-0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "1fcd6df07efadab9d7cb2a10d7386434", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 25978, "upload_time": "2018-12-11T01:52:50", "url": "https://files.pythonhosted.org/packages/1e/18/e0575db118725478316c057a02ab1a2d96802bdf937152b80c95d6fadff3/Dotua-0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "edaaedfc63b4ad7c7a866e0568e08dc7", "sha256": "4cbdf77bc08c36641b91487a258f700db659c3e6a1d154e6d60238409f71ffab" }, "downloads": -1, "filename": "Dotua-0.1.tar.gz", "has_sig": false, "md5_digest": "edaaedfc63b4ad7c7a866e0568e08dc7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11526, "upload_time": "2018-12-11T01:52:52", "url": "https://files.pythonhosted.org/packages/0f/6f/743c15dd0bc11b0dc5deb312af1dab1562bf797baaff2a79aed41e0b4bb0/Dotua-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "e06023027f05357b80f36e1a7b1feabe", "sha256": "0da7a3cd389890d94815abdd03ba6f677eba00714ec0a56f33b911e632e3d977" }, "downloads": -1, "filename": "Dotua-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "e06023027f05357b80f36e1a7b1feabe", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 36427, "upload_time": "2018-12-11T20:59:10", "url": "https://files.pythonhosted.org/packages/39/47/5b22ec45ebec90ddec3a1b513621f454730e31d035d21b558195b6be1e5e/Dotua-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a5d934cefff66ec467c874369471675b", "sha256": "e4ae6be784ac30c8cbf67aeba8a03e3db9f9d406eed15b33b920e451785a574e" }, "downloads": -1, "filename": "Dotua-0.1.1.tar.gz", "has_sig": false, "md5_digest": "a5d934cefff66ec467c874369471675b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26858, "upload_time": "2018-12-11T20:59:11", "url": "https://files.pythonhosted.org/packages/3a/bf/3ceec1e0f58bfae66811004bd1eece0d1d84c893b2e894e1d1a2bf940c7b/Dotua-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "d6802a0a5619acc02ec826aa6c5286f6", "sha256": "707d7a72bd4def5189b6f0ddf499617955d0d6459fda04979c7bc820c33ded66" }, "downloads": -1, "filename": "Dotua-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "d6802a0a5619acc02ec826aa6c5286f6", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 36516, "upload_time": "2018-12-11T22:21:16", "url": "https://files.pythonhosted.org/packages/ad/fc/246160f95bbeca1c84a950b07d3e5e95a0b2d216c743e2cd1fc6c28b2a30/Dotua-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f306a9473afb51b7c7b16d898694617a", "sha256": "b44ae064ff9a4d5f0d4a66ad02801815c158fbfb62cc0d31cc18db7da936c569" }, "downloads": -1, "filename": "Dotua-0.1.2.tar.gz", "has_sig": false, "md5_digest": "f306a9473afb51b7c7b16d898694617a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26940, "upload_time": "2018-12-11T22:21:24", "url": "https://files.pythonhosted.org/packages/b1/35/049c0b3e9fc927e0baa569ce86b2512f501d8353dd36603ebd7105c7b69c/Dotua-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "78485f10a027aa483b0030894755ca85", "sha256": "b2dc518c41d3da80a5196d663aaaec4c68c1ab98b9994291e01866377cadaf8f" }, "downloads": -1, "filename": "Dotua-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "78485f10a027aa483b0030894755ca85", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 36515, "upload_time": "2018-12-11T23:24:17", "url": "https://files.pythonhosted.org/packages/95/f1/62cadfa2378857ea81d1a22b28dd05e4fe5f4ab650d5aaa10bdbff804dcd/Dotua-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6d4cd781008e3a7024a751b46fce522d", "sha256": "e72984b1a789e570251cb1a35a6b75b313ffbb591deef81ee17843763cf82200" }, "downloads": -1, "filename": "Dotua-0.1.3.tar.gz", "has_sig": false, "md5_digest": "6d4cd781008e3a7024a751b46fce522d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26933, "upload_time": "2018-12-11T23:24:20", "url": "https://files.pythonhosted.org/packages/e2/a5/0da6747b2e0569db3e0a1689ce41151a945bfa146f33e1ba0829ac1a9b3b/Dotua-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "c2035d41c81ee05964f727e42c6b17dc", "sha256": "9264045ed6ebb61e594da3c6d884a16169095337ec30fcd830363fd67ad6ee57" }, "downloads": -1, "filename": "Dotua-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "c2035d41c81ee05964f727e42c6b17dc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 36515, "upload_time": "2018-12-11T23:24:19", "url": "https://files.pythonhosted.org/packages/74/30/ee14ac2ca5f9e28e146d9bff8968d1449324d869238797b6ca3e150838a8/Dotua-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4ca65341c06efe2fe2144c9f1735676c", "sha256": "ded233058cd0313fe93cf577bafd722d3b563183d289ba068ca86c8d827c49af" }, "downloads": -1, "filename": "Dotua-0.1.4.tar.gz", "has_sig": false, "md5_digest": "4ca65341c06efe2fe2144c9f1735676c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26943, "upload_time": "2018-12-11T23:24:21", "url": "https://files.pythonhosted.org/packages/82/79/fa1d45dd3d79b65ab3ba8e44b1811eccdbd7468abf670eee62bfbd5e171b/Dotua-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "4bd9d61eae0e60d61ef19f00e463a44b", "sha256": "7fe6c086bdc69b597220b8c68b7851e4d96d1fdcc11c7c2c0b50d04fab7974b5" }, "downloads": -1, "filename": "Dotua-0.1.5-py3-none-any.whl", "has_sig": false, "md5_digest": "4bd9d61eae0e60d61ef19f00e463a44b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 54494, "upload_time": "2018-12-11T23:43:03", "url": "https://files.pythonhosted.org/packages/cb/80/c20cbd5db9c2364c28790d6774a100b5958494a6570149d5b76ab103ea98/Dotua-0.1.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b0eb709c95d76ccc4c3af96611849f0b", "sha256": "b27450cf5909a426be0e9d73d5648b740c238093a3fabf02f72f8297128ee774" }, "downloads": -1, "filename": "Dotua-0.1.5.tar.gz", "has_sig": false, "md5_digest": "b0eb709c95d76ccc4c3af96611849f0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 39294, "upload_time": "2018-12-11T23:43:04", "url": "https://files.pythonhosted.org/packages/11/c1/564c92c295ece033ef1ad0338ae3981dec744f9693d595b8166b7bc7cb7e/Dotua-0.1.5.tar.gz" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "d6240bcef99d91a0cbf8334014f00eb9", "sha256": "1cf60b31ee45892c26f460938ee4cea80d842932edebee096c2626fd8fd7dda6" }, "downloads": -1, "filename": "Dotua-1.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "d6240bcef99d91a0cbf8334014f00eb9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 58748, "upload_time": "2018-12-12T11:59:56", "url": "https://files.pythonhosted.org/packages/94/16/1a7ae36a1c894c02e12ddf780fe036abc47420a4878a6ef66ede4668c624/Dotua-1.0.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b7ff504ef14bad86f5894918c61caa68", "sha256": "1eb5f4f1ef6ab3981931730dccf2f89aa9864c0653fbbac6fe5b773a0b22efc0" }, "downloads": -1, "filename": "Dotua-1.0.0.tar.gz", "has_sig": false, "md5_digest": "b7ff504ef14bad86f5894918c61caa68", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 43237, "upload_time": "2018-12-12T11:59:57", "url": "https://files.pythonhosted.org/packages/f8/99/3d0a65a496a91eef5f609ea3d7c63f9d900908d36f03f445a4ff19aba3ef/Dotua-1.0.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d6240bcef99d91a0cbf8334014f00eb9", "sha256": "1cf60b31ee45892c26f460938ee4cea80d842932edebee096c2626fd8fd7dda6" }, "downloads": -1, "filename": "Dotua-1.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "d6240bcef99d91a0cbf8334014f00eb9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 58748, "upload_time": "2018-12-12T11:59:56", "url": "https://files.pythonhosted.org/packages/94/16/1a7ae36a1c894c02e12ddf780fe036abc47420a4878a6ef66ede4668c624/Dotua-1.0.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b7ff504ef14bad86f5894918c61caa68", "sha256": "1eb5f4f1ef6ab3981931730dccf2f89aa9864c0653fbbac6fe5b773a0b22efc0" }, "downloads": -1, "filename": "Dotua-1.0.0.tar.gz", "has_sig": false, "md5_digest": "b7ff504ef14bad86f5894918c61caa68", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 43237, "upload_time": "2018-12-12T11:59:57", "url": "https://files.pythonhosted.org/packages/f8/99/3d0a65a496a91eef5f609ea3d7c63f9d900908d36f03f445a4ff19aba3ef/Dotua-1.0.0.tar.gz" } ] }