{ "info": { "author": "Alejandro Francisco Queiruga", "author_email": "", "bugtrack_url": null, "classifiers": [], "description": "# DETest\n\nA Differential Equation Testing Suite\n\nAlejandro Francisco Queiruga \nLawrence Berkeley National Lab \n2018\n\nThis repository contains a set of testing problems with known analytical solutions or reference solutions from a \"trusted\" code: oracles.\nIt can generate a testing framework based on Python's unittest test will automatically compare your code to these oracles, making sure on a variety of problems that your code are exactly correct, an approximation that converges at the expected rate, or at least gives you the same answer you got yesterday.\n\nThese tests were originally written to support various codes in the Earth and Environmental Sciences division at Lawrence Berkeley National Lab, and have been merged into this unifying framework.\nThe impetus was when I realized I kept copying and pasting the same Python files implementing analytical solutions, and hapharzardly implementing the same convergence testing. \nWhen you write an incredibly complicated test yourself and it does not agree with your code, you don't know if the problem is with the test or your code.\nThis makes the tests themselves problematic: I and collaborators have had plenty of trouble deriving, researching, and even typing up the analytical solutions themselves.\nHaving a centralized testing framework not only makes it **easier to follow best practices**, it also **validates the tests themselves** by cross referencing with multiple codes:\n\n![cross validation of tests](figures/test_reuse.png)\n\n\n## Installation\n\ndetest is compatible with both Python 2 and 3. Numpy is required, and\nsome of the oracles make use of Scipy and Sympy. Install it from PyPI,\n\n```bash\npip3 install detest\n```\n\nor clone it and use setuptools,\n\n```bash\npython3 setup.py install --prefix=/path/to/install/to\n```\n\nand then import it with `import detest`.\n\n## Introduction\n\nTesting numerical scientific and engineering codes has a further challenge than normal software.\nThe results are inexact and the expected behavior can be unknown.\nWhen we see something wrong with out codes, we don't know if it's a problem with:\n\n1. a bug in the code?\n2. a failure of the numerical model?\n3. a fundamental problem with the underlying theory?\n4. or incorrect expectations?\n\nWe address bullet point #4 by providing a trusted set of analytical solutions to be oracles in this library.\nThe oracles (their theory, mathematical solution, and implementation) are verified by being in agreement with multiple codes!\nWe can use these oracles and an understanding of the behavior we expect to see from our codes to help us diagnose issues #1-#3.\n\n1. Unit Tests - make sure the code works\n2. Known Analytical Oracles - tests with known solutions. The code could either get these solutions exactly correct, or only approximates the solution approach at an expected convergence rate.\n3. Self-Similarity Convergence - Problems without known solutions, but the testee should behave consistently with itself.\n4. Reference Tests - tests with no known solutions, but we compare the codes to an over-discretized trusted code, or experimental data.\n\nThe philosophy I have evolved has the following phases:\n\n### Unit Tests\n\nThese are software tests.\nThese tests are for things on the order of \"Does the code read an input file correctly?\"\nThis repository doesn't deal with them: they're unique to the codebase.\n\n### Oracle-based Exact Precision Tests\n\nThe methods we use to solve problems should be able to get certain solutions **Exactly Correct**.\n\n![assert](https://latex.codecogs.com/gif.latex?assert\\left(&space;\\left|\\left|&space;code&space;-&space;oracle&space;\\right|\\right|&space;<&space;10^{-12}&space;\\right))\n\n\nThese can be done in the same unit testing framework. They should be cheap.\nThis library will generate a unittest object for requested exact precision tests.\n\n### Oracle-based Convergence Tests\n\nThese problems should also have the property that they are Lipschitz continuous; i.e. the numerical problem should converge smoothly.\n\nThese require more computational effort to run.\nThere is some advice for designing simple problems that the testee should be able to execute extremely quickly.\n\nThis library will generate a unittest object for requested exact precision tests.\n\n![error](https://latex.codecogs.com/gif.latex?e(h)&space;=&space;\\left|\\left|&space;code(h)&space;-&space;oracle&space;\\right|\\right|)\n\n\n![assert](https://latex.codecogs.com/gif.latex?assert\\left(&space;regression(\\log(h),\\log(e))&space;\\approx&space;rate&space;\\right))\n\n\nfor an expected rate.\n\n### Self-Similarity Convergence Tests\n\nWe assume self similarity.\n\n![error](https://latex.codecogs.com/gif.latex?e(h)&space;=&space;\\left|\\left|&space;code(h)&space;-&space;code(H^*)&space;\\right|\\right|)\n\n\nwhere $H^*\\ll h$.\n\n![assert](https://latex.codecogs.com/gif.latex?assert\\left(&space;regression(\\log(h),\\log(e))&space;\\approx&space;rate&space;\\right))\n\n\n### Reference Tests\n\nWe can construct a reference solution by saving the solutions of a \"golden code\" in the database, and then comparing future codes to it.\n\n### Regression Tests\n\nRunning all of the above tests can be costly.\nMost of our everyday programming shouldn't effect the results for most of the codes capabilities.\nAfter a set of changes, we should assert\n\n![assert](https://latex.codecogs.com/gif.latex?assert\\left(&space;code(today)&space;\\approx&space;code(yesterday)&space;\\right))\n\n\nfor every problem we don't think we changed.\nThese can be very fast and short, and randomly generated every day.\n\nDETest will eventually have a class that runs these tests and maintains this history database.\n\n## What makes a good test?\n\nWhat happens when one of these tests isn't Lipschitz continuous? What if the physics itself has bifurcation points?\nHow do we verify a code that is supposed to be solving a nonsmooth problem?\nThese are hard problems to solve.\nA strategy is to examine ergodic properties, such as the center point of an attractor, the total amount of gas in the reservoir, the granular temperature of the system, etc.\nThese have to be coupled by making your code solve the simpler problems, too.\nCurrently, we have no such tests in this package.\n\n## The Tests\n\nRight now, the focus is on problems in subsurface flow and\nmechanics. The complete list of oracles is \n\n1. Constant strain modes in elasticity\n1. Constant flux flow\n1. Terzaghi's consolidation\n1. de Leeuw's consolidation\n1. Thin crack in tension / under pressure\n1. Poisson problem\n1. Radial production in a poroelastic system\n\nNo \"gold standard\" datasets for reference problems have been included yet.\n\nThe problems in the library each have their own module.\nEach module has a description, default parameter dictionary, and at least one class that implements the solution.\nThe class is meant to behave like a closure, where it is initialized with a parameter set and is callable.\nEach closure evaluates the analytical solution at a point that is some combination of $(x,y,z,t)$ for the given parameters.\nThe module will have a default set of parameters.\n\nThe units for default properties are all in SI.\nThis strictly doesn't matter, but for some codes the properties are not inputs, but instead intrinsic to the theoretical formulation.\n\n\n## Using the Test Suite\n\n### Wrapping the code\n\nThe developer needs to wrap their code into a Python script with one function call for each oracle.\nThe routine for your code takes two arguments: a dictionary of parameters, and an 'h' argument for discretization scale.\nThe parameters a specific to the oracle problem (e.g. domain size, conductivity, etc.); read the description of the oracle.\nTo check mesh refinement, 'h' will be a grid spacing or a timestep size. (I sometimes make it control both.)\nDetest will automatically call the script with a set of parameters with many different 'h's.\nThe output is a dictionary that matches the field names of the oracle, plus an additional field 'points'.\n\nFor a command line code, this will look something like this:\n\n```python\ndef myScript(parameters, h):\n make_tough_input(h,parameters)\n sp.call(['TH','millstone_input.py',str(h)])\n x,U,P = process_results_for_testing()\n return {'U':U,'P':P, 'points':x}\n```\n\nwhere the developer is responsible for autogenerating their input files.\n\n### Making an automated suite\n\nDetest will autogenerate a unittest suite.\nA snippet this short will populate the unittest framework:\n\n```python\nsuite = [\n detest.ExactTest( detest.oracles.Uniaxial, myUniaxial),\n detest.ExactTest( detest.oracles.Shear, myShear),\n detest.ConvergenceTest(detest.oracles.Terzaghi, myTerzaghi, 1),\n]\nMyTestSuite = detest.make_suite(suite)\n```\n\nThe file can then be executed with the unittest module,\n\n```bash\npython -m unittest MyTestSuite.py\n```\n\nThe power of this architecture is that list can be generated with a loop.\nAn example of this is in [afqsrungekutta](https://github.com/afqueiruga/afqsrungekutta/blob/master/test/de_test.py),\nwherein a seperate ConvergenceTest for every each tableau is made.\n\n### Just using the oracles\n\nDetest is also a library of analytical solutions.\nOne can just import the solutions if that's useful,\n```python\nimport detest\nf = detest.oracles.Terzaghi({'k':2.0e-12})\ndisplacement = f([[5.0,10.0]])['U']\n```\nThis is particularly useful for applying truncated boundary conditions for some of the convergence tests.\n\n### Computation Expense Concerns\n\nRunning numerical tests is expensive in terms of computing time, which is also a dollar-cost.\nThere are different strategies to minimize the cost and enable real-time continuous integration:\n\n1. Only test randomly with frequency, and save the rigorous-churn through tests for weekly tests.\n2. Use a scheduling environment to run tests in parallel on commodity resources.\n3. Schedule them for low-priority queues at a low-cost off-hours.\n\n## License\n\nCopyright (C) Alejandro Francisco Queiruga, 2015-2018 \nLawrence Berkeley National Lab\n\nDETest is released under version 3 of the GNU Lesser General Public License, as per LICENSE.txt.\n\n## Acknowledgements\n\nThis library was developed to support various projects in the Earth and Environmental Sciences division at Lawrence Berkeley National Lab.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/afqueiruga/detest", "keywords": "", "license": "GNU LGPL", "maintainer": "", "maintainer_email": "", "name": "detest", "package_url": "https://pypi.org/project/detest/", "platform": "", "project_url": "https://pypi.org/project/detest/", "project_urls": { "Homepage": "https://github.com/afqueiruga/detest" }, "release_url": "https://pypi.org/project/detest/0.2/", "requires_dist": null, "requires_python": "", "summary": "A differential equation testing suite", "version": "0.2", "yanked": false, "yanked_reason": null }, "last_serial": 6031348, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "170f7a018e43cbb90b965cea3565631a", "sha256": "897cf1ac72c8f838c2773f2a8a0b34861ae5ab58aa024018046dc0ca96d3f30e" }, "downloads": -1, "filename": "detest-0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "170f7a018e43cbb90b965cea3565631a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 25729, "upload_time": "2019-09-27T06:41:01", "upload_time_iso_8601": "2019-09-27T06:41:01.392602Z", "url": "https://files.pythonhosted.org/packages/06/a1/ac532d5ab5135c6871177baf17c188bff8df7cd5d0f58b92ec2fbf611062/detest-0.1-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "2fb6a49ad9dd2f371c7c7fb6820abc3d", "sha256": "5fe0bcb4ef20d8061bc7bb517048cf3ccc331ce6ba9470a942321d58be723bc1" }, "downloads": -1, "filename": "detest-0.1.tar.gz", "has_sig": false, "md5_digest": "2fb6a49ad9dd2f371c7c7fb6820abc3d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18032, "upload_time": "2019-09-27T06:41:04", "upload_time_iso_8601": "2019-09-27T06:41:04.587523Z", "url": "https://files.pythonhosted.org/packages/51/e9/dc4e1548e1b6ee17f2b3173198c2dcebbfbb68032bc70b469516678975fa/detest-0.1.tar.gz", "yanked": false, "yanked_reason": null } ], "0.2": [ { "comment_text": "", "digests": { "md5": "02dec8850dd2abda3f0092196dcbe09d", "sha256": "b50309da5a4716dbb791a05c60a391379ea432bc48b5aef722b4b38fcc7d5113" }, "downloads": -1, "filename": "detest-0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "02dec8850dd2abda3f0092196dcbe09d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26612, "upload_time": "2019-10-25T18:47:20", "upload_time_iso_8601": "2019-10-25T18:47:20.258319Z", "url": "https://files.pythonhosted.org/packages/67/8a/b41722d892da5867aa1f945793a93aa58f0e82cfbde9a9e4806869c5ab3e/detest-0.2-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "2de58691fc0d6ae04437d15abfd52b3d", "sha256": "e2bba462d8c04212fc7702247388d8666ac52eb44736efb9cecc25ec4be23ac0" }, "downloads": -1, "filename": "detest-0.2.tar.gz", "has_sig": false, "md5_digest": "2de58691fc0d6ae04437d15abfd52b3d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18603, "upload_time": "2019-10-25T18:47:21", "upload_time_iso_8601": "2019-10-25T18:47:21.597971Z", "url": "https://files.pythonhosted.org/packages/bf/bd/f02e451de79a03e3f09f79f9166f09c31a57787f1b4e4d5cbd09c5270fa4/detest-0.2.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "02dec8850dd2abda3f0092196dcbe09d", "sha256": "b50309da5a4716dbb791a05c60a391379ea432bc48b5aef722b4b38fcc7d5113" }, "downloads": -1, "filename": "detest-0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "02dec8850dd2abda3f0092196dcbe09d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26612, "upload_time": "2019-10-25T18:47:20", "upload_time_iso_8601": "2019-10-25T18:47:20.258319Z", "url": "https://files.pythonhosted.org/packages/67/8a/b41722d892da5867aa1f945793a93aa58f0e82cfbde9a9e4806869c5ab3e/detest-0.2-py3-none-any.whl", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "2de58691fc0d6ae04437d15abfd52b3d", "sha256": "e2bba462d8c04212fc7702247388d8666ac52eb44736efb9cecc25ec4be23ac0" }, "downloads": -1, "filename": "detest-0.2.tar.gz", "has_sig": false, "md5_digest": "2de58691fc0d6ae04437d15abfd52b3d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18603, "upload_time": "2019-10-25T18:47:21", "upload_time_iso_8601": "2019-10-25T18:47:21.597971Z", "url": "https://files.pythonhosted.org/packages/bf/bd/f02e451de79a03e3f09f79f9166f09c31a57787f1b4e4d5cbd09c5270fa4/detest-0.2.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }