:mod:`xdoctest.utils`
=====================

.. py:module:: xdoctest.utils

.. autoapi-nested-parse::

   Most of these utilities exist in ubelt, but we copy them here to keep xdoctest
   as a package with minimal dependencies, whereas ubelt includes a larger set of
   utilities.

   This __init__ file is generated using mkinit:

       mkinit xdoctest.utils



Submodules
----------
.. toctree::
   :titlesonly:
   :maxdepth: 1

   util_import/index.rst
   util_misc/index.rst
   util_mixins/index.rst
   util_notebook/index.rst
   util_path/index.rst
   util_str/index.rst
   util_stream/index.rst


Package Contents
----------------

Classes
~~~~~~~

.. autoapisummary::

   xdoctest.utils.PythonPathContext
   xdoctest.utils.TempDoctest
   xdoctest.utils.NiceRepr
   xdoctest.utils.TempDir
   xdoctest.utils.CaptureStdout
   xdoctest.utils.CaptureStream
   xdoctest.utils.TeeStringIO



Functions
~~~~~~~~~

.. autoapisummary::

   xdoctest.utils.import_module_from_name
   xdoctest.utils.import_module_from_path
   xdoctest.utils.is_modname_importable
   xdoctest.utils.modname_to_modpath
   xdoctest.utils.modpath_to_modname
   xdoctest.utils.normalize_modpath
   xdoctest.utils.split_modpath
   xdoctest.utils.ensuredir
   xdoctest.utils.add_line_numbers
   xdoctest.utils.codeblock
   xdoctest.utils.color_text
   xdoctest.utils.ensure_unicode
   xdoctest.utils.highlight_code
   xdoctest.utils.indent
   xdoctest.utils.strip_ansi


.. py:class:: PythonPathContext(dpath, index=0)

   Bases: :class:`object`

   Context for temporarily adding a dir to the PYTHONPATH. Used in testing

   :Parameters: * **dpath** (*str*) -- directory to insert into the PYTHONPATH
                * **index** (*int*) -- position to add to. Typically either -1 or 0.

   .. rubric:: Example

   >>> with PythonPathContext('foo', -1):
   >>>     assert sys.path[-1] == 'foo'
   >>> assert sys.path[-1] != 'foo'
   >>> with PythonPathContext('bar', 0):
   >>>     assert sys.path[0] == 'bar'
   >>> assert sys.path[0] != 'bar'

   .. rubric:: Example

   >>> # Mangle the path inside the context
   >>> # xdoctest: +REQUIRES(module:pytest)
   >>> self = PythonPathContext('foo', 0)
   >>> self.__enter__()
   >>> import pytest
   >>> with pytest.warns(UserWarning):
   >>>     sys.path.insert(0, 'mangled')
   >>>     self.__exit__(None, None, None)

   .. rubric:: Example

   >>> # xdoctest: +REQUIRES(module:pytest)
   >>> self = PythonPathContext('foo', 0)
   >>> self.__enter__()
   >>> sys.path.remove('foo')
   >>> import pytest
   >>> with pytest.raises(RuntimeError):
   >>>     self.__exit__(None, None, None)

   .. method:: __enter__(self)



   .. method:: __exit__(self, type, value, trace)




.. function:: import_module_from_name(modname)

   Imports a module from its string name (__name__)

   :Parameters: **modname** (*str*) -- module name

   :returns: module
   :rtype: module

   .. rubric:: Example

   >>> # test with modules that wont be imported in normal circumstances
   >>> # todo write a test where we gaurentee this
   >>> modname_list = [
   >>>     'pickletools',
   >>>     'lib2to3.fixes.fix_apply',
   >>> ]
   >>> #assert not any(m in sys.modules for m in modname_list)
   >>> modules = [import_module_from_name(modname) for modname in modname_list]
   >>> assert [m.__name__ for m in modules] == modname_list
   >>> assert all(m in sys.modules for m in modname_list)


.. function:: import_module_from_path(modpath, index=-1)

   Imports a module via its path

   :Parameters: * **modpath** (*PathLike*) -- path to the module on disk or within a zipfile.
                * **index** (*int*) -- location at which we modify PYTHONPATH if necessary.
                  If your module name does not conflict, the safest value is -1,
                  However, if there is a conflict, then use an index of 0.
                  The default may change to 0 in the future.

   :returns: the imported module
   :rtype: module

   .. rubric:: References

   https://stackoverflow.com/questions/67631/import-module-given-path

   .. rubric:: Notes

   If the module is part of a package, the package will be imported first.
   These modules may cause problems when reloading via IPython magic

   This can import a module from within a zipfile. To do this modpath
   should specify the path to the zipfile and the path to the module
   within that zipfile separated by a colon or pathsep.
   E.g. `/path/to/archive.zip:mymodule.py`

   .. warning::

      It is best to use this with paths that will not conflict with
      previously existing modules.
      
      If the modpath conflicts with a previously existing module name. And
      the target module does imports of its own relative to this conflicting
      path. In this case, the module that was loaded first will win.
      
      For example if you try to import '/foo/bar/pkg/mod.py' from the folder
      structure:
        - foo/
          +- bar/
             +- pkg/
                +  __init__.py
                |- mod.py
                |- helper.py

      If there exists another module named `pkg` already in sys.modules
      and mod.py does something like `from . import helper`, Python will
      assume helper belongs to the `pkg` module already in sys.modules.
      This can cause a NameError or worse --- a incorrect helper module.

   .. rubric:: Example

   >>> import xdoctest
   >>> modpath = xdoctest.__file__
   >>> module = import_module_from_path(modpath)
   >>> assert module is xdoctest

   .. rubric:: Example

   >>> # Test importing a module from within a zipfile
   >>> import zipfile
   >>> from xdoctest import utils
   >>> from os.path import join, expanduser
   >>> dpath = expanduser('~/.cache/xdoctest')
   >>> dpath = utils.ensuredir(dpath)
   >>> # Write to an external module named bar
   >>> external_modpath = join(dpath, 'bar.py')
   >>> # For pypy support we have to write this using with
   >>> with open(external_modpath, 'w') as file:
   >>>     file.write('testvar = 1')
   >>> assert open(external_modpath, 'r').read() == 'testvar = 1'
   >>> internal = 'folder/bar.py'
   >>> # Move the external bar module into a zipfile
   >>> zippath = join(dpath, 'myzip.zip')
   >>> with zipfile.ZipFile(zippath, 'w') as myzip:
   >>>     myzip.write(external_modpath, internal)
   >>> assert exists(zippath)
   >>> # Import the bar module from within the zipfile
   >>> modpath = zippath + ':' + internal
   >>> modpath = zippath + os.path.sep + internal
   >>> module = import_module_from_path(modpath)
   >>> assert module.__name__ == os.path.normpath('folder/bar')
   >>> print('module = {!r}'.format(module))
   >>> print('module.__file__ = {!r}'.format(module.__file__))
   >>> print(dir(module))
   >>> assert module.testvar == 1

   Doctest:
       >>> # xdoctest: +REQUIRES(module:pytest)
       >>> import pytest
       >>> with pytest.raises(IOError):
       >>>     import_module_from_path('does-not-exist')
       >>> with pytest.raises(IOError):
       >>>     import_module_from_path('does-not-exist.zip/')


.. function:: is_modname_importable(modname, sys_path=None, exclude=None)

   Determines if a modname is importable based on your current sys.path

   :Parameters: * **modname** (*str*) -- name of module to check
                * **sys_path** (*list, default=None*) -- if specified overrides `sys.path`
                * **exclude** (*list*) -- list of directory paths. if specified prevents these
                  directories from being searched.

   :returns: True if the module can be imported
   :rtype: bool

   .. rubric:: Example

   >>> is_modname_importable('xdoctest')
   True
   >>> is_modname_importable('not_a_real_module')
   False
   >>> is_modname_importable('xdoctest', sys_path=[])
   False


.. function:: modname_to_modpath(modname, hide_init=True, hide_main=False, sys_path=None)

   Finds the path to a python module from its name.

   Determines the path to a python module without directly import it

   Converts the name of a module (__name__) to the path (__file__) where it is
   located without importing the module. Returns None if the module does not
   exist.

   :Parameters: * **modname** (*str*) -- module filepath
                * **hide_init** (*bool*) -- if False, __init__.py will be returned for packages
                * **hide_main** (*bool*) -- if False, and hide_init is True, __main__.py will be
                  returned for packages, if it exists.
                * **sys_path** (*list, default=None*) -- if specified overrides `sys.path`

   :returns: modpath - path to the module, or None if it doesn't exist
   :rtype: str

   .. rubric:: Example

   >>> modname = 'xdoctest.__main__'
   >>> modpath = modname_to_modpath(modname, hide_main=False)
   >>> assert modpath.endswith('__main__.py')
   >>> modname = 'xdoctest'
   >>> modpath = modname_to_modpath(modname, hide_init=False)
   >>> assert modpath.endswith('__init__.py')
   >>> # xdoctest: +REQUIRES(CPython)
   >>> modpath = basename(modname_to_modpath('_ctypes'))
   >>> assert 'ctypes' in modpath


.. function:: modpath_to_modname(modpath, hide_init=True, hide_main=False, check=True, relativeto=None)

   Determines importable name from file path

   Converts the path to a module (__file__) to the importable python name
   (__name__) without importing the module.

   The filename is converted to a module name, and parent directories are
   recursively included until a directory without an __init__.py file is
   encountered.

   :Parameters: * **modpath** (*str*) -- module filepath
                * **hide_init** (*bool, default=True*) -- removes the __init__ suffix
                * **hide_main** (*bool, default=False*) -- removes the __main__ suffix
                * **check** (*bool, default=True*) -- if False, does not raise an error if
                  modpath is a dir and does not contain an __init__ file.
                * **relativeto** (*str, default=None*) -- if specified, all checks are ignored
                  and this is considered the path to the root module.

   :returns: modname
   :rtype: str

   :raises ValueError: if check is True and the path does not exist

   .. rubric:: Example

   >>> from xdoctest import static_analysis
   >>> modpath = static_analysis.__file__.replace('.pyc', '.py')
   >>> modpath = modpath.replace('.pyc', '.py')
   >>> modname = modpath_to_modname(modpath)
   >>> assert modname == 'xdoctest.static_analysis'

   .. rubric:: Example

   >>> import xdoctest
   >>> assert modpath_to_modname(xdoctest.__file__.replace('.pyc', '.py')) == 'xdoctest'
   >>> assert modpath_to_modname(dirname(xdoctest.__file__.replace('.pyc', '.py'))) == 'xdoctest'

   .. rubric:: Example

   >>> # xdoctest: +REQUIRES(CPython)
   >>> modpath = modname_to_modpath('_ctypes')
   >>> modname = modpath_to_modname(modpath)
   >>> assert modname == '_ctypes'

   .. rubric:: Example

   >>> modpath = '/foo/libfoobar.linux-x86_64-3.6.so'
   >>> modname = modpath_to_modname(modpath, check=False)
   >>> assert modname == 'libfoobar'


.. function:: normalize_modpath(modpath, hide_init=True, hide_main=False)

   Normalizes __init__ and __main__ paths.

   :Parameters: * **modpath** (*PathLike*) -- path to a module
                * **hide_init** (*bool, default=True*) -- if True, always return package modules
                  as __init__.py files otherwise always return the dpath.
                * **hide_main** (*bool, default=False*) -- if True, always strip away main files
                  otherwise ignore __main__.py.

   :returns: a normalized path to the module
   :rtype: PathLike

   .. rubric:: Notes

   Adds __init__ if reasonable, but only removes __main__ by default

   .. rubric:: Example

   >>> from xdoctest import static_analysis as module
   >>> modpath = module.__file__
   >>> assert normalize_modpath(modpath) == modpath.replace('.pyc', '.py')
   >>> dpath = dirname(modpath)
   >>> res0 = normalize_modpath(dpath, hide_init=0, hide_main=0)
   >>> res1 = normalize_modpath(dpath, hide_init=0, hide_main=1)
   >>> res2 = normalize_modpath(dpath, hide_init=1, hide_main=0)
   >>> res3 = normalize_modpath(dpath, hide_init=1, hide_main=1)
   >>> assert res0.endswith('__init__.py')
   >>> assert res1.endswith('__init__.py')
   >>> assert not res2.endswith('.py')
   >>> assert not res3.endswith('.py')


.. function:: split_modpath(modpath, check=True)

   Splits the modpath into the dir that must be in PYTHONPATH for the module
   to be imported and the modulepath relative to this directory.

   :Parameters: * **modpath** (*str*) -- module filepath
                * **check** (*bool*) -- if False, does not raise an error if modpath is a
                  directory and does not contain an `__init__.py` file.

   :returns: (directory, rel_modpath)
   :rtype: tuple

   :raises ValueError: if modpath does not exist or is not a package

   .. rubric:: Example

   >>> from xdoctest import static_analysis
   >>> modpath = static_analysis.__file__.replace('.pyc', '.py')
   >>> modpath = abspath(modpath)
   >>> dpath, rel_modpath = split_modpath(modpath)
   >>> recon = join(dpath, rel_modpath)
   >>> assert recon == modpath
   >>> assert rel_modpath == join('xdoctest', 'static_analysis.py')


.. py:class:: TempDoctest(docstr, modname=None)

   Bases: :class:`object`

   Creates a temporary file containing a module-level doctest for testing

   .. rubric:: Example

   >>> from xdoctest import core
   >>> self = TempDoctest('>>> a = 1')
   >>> doctests = list(core.parse_doctestables(self.modpath))
   >>> assert len(doctests) == 1


.. py:class:: NiceRepr

   Bases: :class:`object`

   Defines `__str__` and `__repr__` in terms of `__nice__` function
   Classes that inherit `NiceRepr` must define `__nice__`

   .. rubric:: Example

   >>> class Foo(NiceRepr):
   ...    pass
   >>> class Bar(NiceRepr):
   ...    def __nice__(self):
   ...        return 'info'
   >>> foo = Foo()
   >>> bar = Bar()
   >>> assert str(bar) == '<Bar(info)>'
   >>> assert repr(bar).startswith('<Bar(info) at ')
   >>> assert 'object at' in str(foo)
   >>> assert 'object at' in repr(foo)

   .. method:: __repr__(self)


      Return repr(self).


   .. method:: __str__(self)


      Return str(self).



.. py:class:: TempDir(persist=False)

   Bases: :class:`object`

   Context for creating and cleaning up temporary files. Used in testing.

   .. rubric:: Example

   >>> with TempDir() as self:
   >>>     dpath = self.dpath
   >>>     assert exists(dpath)
   >>> assert not exists(dpath)

   .. rubric:: Example

   >>> self = TempDir()
   >>> dpath = self.ensure()
   >>> assert exists(dpath)
   >>> self.cleanup()
   >>> assert not exists(dpath)

   .. method:: __del__(self)



   .. method:: ensure(self)



   .. method:: cleanup(self)



   .. method:: __enter__(self)



   .. method:: __exit__(self, type_, value, trace)




.. function:: ensuredir(dpath, mode=1023)

   Ensures that directory will exist. creates new dir with sticky bits by
   default

   :Parameters: * **dpath** (*str*) -- dir to ensure. Can also be a tuple to send to join
                * **mode** (*int*) -- octal mode of directory (default 0o1777)

   :returns: path - the ensured directory
   :rtype: str


.. function:: add_line_numbers(source, start=1, n_digits=None)

   Prefixes code with line numbers

   .. rubric:: Example

   >>> print(chr(10).join(add_line_numbers(['a', 'b', 'c'])))
   1 a
   2 b
   3 c
   >>> print(add_line_numbers(chr(10).join(['a', 'b', 'c'])))
   1 a
   2 b
   3 c


.. function:: codeblock(block_str)

   Wraps multiline string blocks and returns unindented code.
   Useful for templated code defined in indented parts of code.

   :Parameters: **block_str** (*str*) -- typically in the form of a multiline string

   :returns: the unindented string
   :rtype: str

   .. rubric:: Example

   >>> # Simulate an indented part of code
   >>> if True:
   ...     # notice the indentation on this will be normal
   ...     codeblock_version = codeblock(
   ...             '''
   ...             def foo():
   ...                 return 'bar'
   ...             '''
   ...         )
   ...     # notice the indentation and newlines on this will be odd
   ...     normal_version = ('''
   ...         def foo():
   ...             return 'bar'
   ...     ''')
   >>> assert normal_version != codeblock_version
   >>> print('Without codeblock')
   >>> print(normal_version)
   >>> print('With codeblock')
   >>> print(codeblock_version)


.. function:: color_text(text, color)

   Colorizes text a single color using ansii tags.

   :Parameters: * **text** (*str*) -- text to colorize
                * **color** (*str*) -- may be one of the following: yellow, blink, lightgray,
                  underline, darkyellow, blue, darkblue, faint, fuchsia, black,
                  white, red, brown, turquoise, bold, darkred, darkgreen, reset,
                  standout, darkteal, darkgray, overline, purple, green, teal, fuscia

   :returns:

             text : colorized text.
                 If pygments is not installed plain text is returned.
   :rtype: str

   .. rubric:: Example

   >>> import sys
   >>> if sys.platform.startswith('win32'):
   >>>     import pytest
   >>>     pytest.skip()
   >>> text = 'raw text'
   >>> from xdoctest import utils
   >>> from xdoctest.utils import util_str
   >>> if utils.modname_to_modpath('pygments') and not util_str.NO_COLOR:
   >>>     # Colors text only if pygments is installed
   >>>     import pygments
   >>>     print('pygments = {!r}'.format(pygments))
   >>>     ansi_text1 = color_text(text, 'red')
   >>>     print('ansi_text1 = {!r}'.format(ansi_text1))
   >>>     ansi_text = utils.ensure_unicode(ansi_text1)
   >>>     prefix = utils.ensure_unicode('\x1b[31')
   >>>     print('prefix = {!r}'.format(prefix))
   >>>     print('ansi_text = {!r}'.format(ansi_text))
   >>>     assert ansi_text.startswith(prefix)
   >>>     assert color_text(text, None) == 'raw text'
   >>> else:
   >>>     # Otherwise text passes through unchanged
   >>>     assert color_text(text, 'red') == 'raw text'
   >>>     assert color_text(text, None) == 'raw text'


.. function:: ensure_unicode(text)

   Casts bytes into utf8 (mostly for python2 compatibility)

   .. rubric:: References

   http://stackoverflow.com/questions/12561063/python-extract-data-from-file

   CommandLine:
       python -m xdoctest.utils ensure_unicode

   .. rubric:: Example

   >>> assert ensure_unicode('my ünicôdé strįng') == 'my ünicôdé strįng'
   >>> assert ensure_unicode('text1') == 'text1'
   >>> assert ensure_unicode('text1'.encode('utf8')) == 'text1'
   >>> assert ensure_unicode('ï»¿text1'.encode('utf8')) == 'ï»¿text1'
   >>> import codecs
   >>> assert (codecs.BOM_UTF8 + 'text»¿'.encode('utf8')).decode('utf8')


.. function:: highlight_code(text, lexer_name='python', **kwargs)

   Highlights a block of text using ansi tags based on language syntax.

   :Parameters: * **text** (*str*) -- plain text to highlight
                * **lexer_name** (*str*) -- name of language
                * **\*\*kwargs** -- passed to pygments.lexers.get_lexer_by_name

   :returns:

             text : highlighted text
                 If pygments is not installed, the plain text is returned.
   :rtype: str

   CommandLine:
       python -c "import pygments.formatters; print(list(pygments.formatters.get_all_formatters()))"

   .. rubric:: Example

   >>> text = 'import xdoctest as xdoc; print(xdoc)'
   >>> new_text = highlight_code(text)
   >>> print(new_text)


.. function:: indent(text, prefix='    ')

   Indents a block of text

   :Parameters: * **text** (*str*) -- text to indent
                * **prefix** (*str*) -- prefix to add to each line (default = '    ')

   :returns: indented text
   :rtype: str

   CommandLine:
       python -m xdoctest.utils ensure_unicode

   .. rubric:: Example

   >>> text = 'Lorem ipsum\ndolor sit amet'
   >>> prefix = '    '
   >>> result = indent(text, prefix)
   >>> assert all(t.startswith(prefix) for t in result.split('\n'))


.. function:: strip_ansi(text)

   Removes all ansi directives from the string.

   .. rubric:: References

   http://stackoverflow.com/questions/14693701/remove-ansi
   https://stackoverflow.com/questions/13506033/filtering-out-ansi-escape-sequences

   .. rubric:: Examples

   >>> line = '\t\u001b[0;35mBlabla\u001b[0m     \u001b[0;36m172.18.0.2\u001b[0m'
   >>> escaped_line = strip_ansi(line)
   >>> assert escaped_line == '\tBlabla     172.18.0.2'


.. py:class:: CaptureStdout(supress=True, enabled=True)

   Bases: :class:`xdoctest.utils.util_stream.CaptureStream`

   Context manager that captures stdout and stores it in an internal stream

   :Parameters: * **supress** (*bool, default=True*) -- if True, stdout is not printed while captured
                * **enabled** (*bool, default=True*) -- does nothing if this is False

   .. rubric:: Example

   >>> self = CaptureStdout(supress=True)
   >>> print('dont capture the table flip (╯°□°）╯︵ ┻━┻')
   >>> with self:
   ...     text = 'capture the heart ♥'
   ...     print(text)
   >>> print('dont capture look of disapproval ಠ_ಠ')
   >>> assert isinstance(self.text, six.text_type)
   >>> assert self.text == text + '\n', 'failed capture text'

   .. rubric:: Example

   >>> self = CaptureStdout(supress=False)
   >>> with self:
   ...     print('I am captured and printed in stdout')
   >>> assert self.text.strip() == 'I am captured and printed in stdout'

   .. rubric:: Example

   >>> self = CaptureStdout(supress=True, enabled=False)
   >>> with self:
   ...     print('dont capture')
   >>> assert self.text is None

   .. method:: log_part(self)


      Log what has been captured so far


   .. method:: start(self)



   .. method:: stop(self)


      .. rubric:: Example

      >>> CaptureStdout(enabled=False).stop()
      >>> CaptureStdout(enabled=True).stop()


   .. method:: __enter__(self)



   .. method:: __del__(self)



   .. method:: close(self)



   .. method:: __exit__(self, type_, value, trace)




.. py:class:: CaptureStream

   Bases: :class:`object`

   Generic class for capturing streaming output from stdout or stderr


.. py:class:: TeeStringIO(redirect=None)

   Bases: :class:`io.StringIO`

   An IO object that writes to itself and another IO stream.

   :ivar redirect: The other stream to write to.

   :vartype redirect: io.IOBase

   .. rubric:: Example

   >>> redirect = io.StringIO()
   >>> self = TeeStringIO(redirect)

   .. method:: isatty(self)


      Returns true of the redirect is a terminal.

      .. rubric:: Notes

      Needed for IPython.embed to work properly when this class is used
      to override stdout / stderr.


   .. method:: fileno(self)


      Returns underlying file descriptor of the redirected IOBase object
      if one exists.


   .. method:: encoding(self)
      :property:


      Gets the encoding of the `redirect` IO object

      .. rubric:: Example

      >>> redirect = io.StringIO()
      >>> assert TeeStringIO(redirect).encoding is None
      >>> assert TeeStringIO(None).encoding is None
      >>> assert TeeStringIO(sys.stdout).encoding is sys.stdout.encoding
      >>> redirect = io.TextIOWrapper(io.StringIO())
      >>> assert TeeStringIO(redirect).encoding is redirect.encoding


   .. method:: write(self, msg)


      Write to this and the redirected stream


   .. method:: flush(self)


      Flush to this and the redirected stream



