QInfer/python-qinfer

..
    This work is licensed under the Creative Commons Attribution-
    NonCommercial-ShareAlike 3.0 Unported License. To view a copy of this
    license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/ or send a
    letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View,
    California, 94041, USA.
    
.. _unittesting_devguide:
    
.. currentmodule:: qinfer

Unit and Documentation Testing
==============================

Unit Tests
----------

.. _doctest_devguide:

Documentation Tests
-------------------

As described in :ref:`docs_devguide_snippets`, Sphinx integrates with
Python's :mod:`doctest` module to help ensure that both the documentation
and underlying library are correct. Doctests consist of short snippets
of code along with their expected output. A doctest *passes* if the
actual output of the snippet matches its expected output. For instance,
a doctest that ``1 + 1`` correctly produces ``2`` could be written as::

    Below, we show an example of addition in practice:

    >>> 1 + 1
    2

    Later, we will consider more advanced operators.

The blank lines above and below the doctest separate it from the surrounding
text, while the expected output appears immediately below the relevant code.

To run the doctests in the **QInfer** documentation using Linux or OS X:

.. code-block:: bash

    $ cd doc/
    $ make doctest

To run the doctests on Windows using PowerShell, use ``.\make`` instead:

.. code-block:: bash

    PS > cd doc/
    PS > .\make doctest

As with the unit tests, doctests are automatically run on pull requests,
to help ensure the correctness of contributed documentation.

Test Annotations
~~~~~~~~~~~~~~~~

A doctest snippet may be annotated with one or more comments that change
the behavior of that test. The :mod:`doctest` documentation goes into far
more detail, but the two we will commonly need are ``# doctest: +SKIP``
and ``# doctest: +ELLIPSIS``. The former causes a test to be skipped entirely.
Skipping tests can be useful if the output of a doctest is random, for instance.

The second, ``+ELLIPSIS``, causes any ellipsis (``...``) in the expected output
to act as a wild card. For instance, both of the following doctests would pass::

    >>> print([1, 2, 3]) # doctest: +ELLIPSIS
    [1, ..., 3]
    >>> print([1, 2, 3, 4, 'foo', 3]) # doctest: +ELLIPSIS
    [1, ..., 3]

Doctest Annoyances
~~~~~~~~~~~~~~~~~~

There are a few annoyances that come along with writing tests based
on string-equivalence of outputs, in particular for a cross-platform
and 2/3 compatible library. In particular:

- NumPy, 2to3 and ``type``/``class``: Python 2 and 3 differ on whether
  the type of NumPy :class:`~numpy.ndarray` prints as ``type`` (Python 2)
  ``class`` (Python 3)::

      >>> print(type(np.array([1])))
      <type 'numpy.ndarray'> # Python 2
      <class 'numpy.ndarray'> # Python 3

  Thus, to write a doctest that checks if something is a NumPy array or not,
  it is preferred to use an `isinstance` check instead::

      >>> isinstance(np.array([1]), np.ndarray)
      True

  Though this sacrifices some on readability, it gains on portability and correctness.

- `int` versus `long` in array shapes: The Windows and Linux versions of NumPy
  behave differently with respect to when a NumPy shape is represented as a
  `tuple` of `int` versus a `tuple` of `long` values. This can cause doctests
  to choke on spurious ``L`` suffixes::

      >>> print(np.zeros((1, 10)).shape)
      (1, 10) # tuple of int
      (1L, 10L) # tuple of long

  Since `int` and `long` respect ``==``, however, the same trick as above can
  help::

      >>> np.zeros((1, 10)).shape == (1, 10)
      True