.. ======================================================================
CHLone - CGNS HDF5 LIBRARY only node edition
See license.txt in the root directory of this source release
======================================================================
.. _pythonCHLone:
Python module
=============
.. sidebar:: CGNS.MAP documentation
The quick start here gives some basic functions use, more detailled
examples are in
`CGNS.MAP <http://pycgns.sourceforge.net/MAP/readme.html>`_ documentation.
As a matter of fact, CGNS.MAP is no more than an import and a rename of
the `CHLone <http://chlone.sourceforge.net>`_ Python module, but actual
use of *CGNS/Python* trees requires Python functions for examples. Then
the actual use would be find in these
`CGNS.MAP <http://pycgns.sourceforge.net/MAP/readme.html>`_ pages.
The CHLone python module is the mapper. It has been moved from
the CGNS.MAP module of pyCGNS to have a self-contained service
for HDF5 and Python translation of CGNS trees. Now you can use
the CGNS/Python interface to CHLone without pyCGNS.
Quick start
-----------
The mapper is a module implementing
the :ref:`SIDS-to-Python <reference_sids_to_python>` mapping,
a CGNS/Python translation of the CGNS/HDF5 files, including links.
There are only two functions in the module: the **load** and the **save**,
the behavior of these function is driven by their arguments. Defaults
arguments insure a simple load/save of CGNS/HDF5 files.
A simple exemple to load a *CGNS/HDF5* file as a *CGNS/Python* tree::
import CHLone
T=CHLone.load("naca012.cgns")
The returned ``T`` is a tuple ``(tree,links,paths)``, the ``load``
argument is the file name, all other parameters are then set to their
default value.
The ``tree`` value contains the actual CGNS/Python tree with linked-to
files included (because the ``S2P_FOLLOWLINKS`` flag is *on*) and the
``links`` value is a list of links found during the *HDF5* file parse.
The last value of a link entry is the status of the link
(see :ref:`flags <lkflags>`).
The ``paths`` contains read time information about incomplete nodes,
for example no data had been retrieved for this node (if the max data
size is reached).
Saving a file is also quite easy::
import CHLone
CHLone.save("naca012.cgns",tree)
The ``tree`` value contains the actual CGNS/Python tree, for example the one
you had with the ``load``.
These two simple functions behaviors can be modified using ``flags`` and
other parameters, see below the *user interface* for a reference of these
arguments and
the `examples <http://pycgns.sourceforge.net/MAP/examples.html>`_
for a rather practical use.
User interface
--------------
It is a lightweight module, its purpose is to be as small as possible
in order to be embedded separatly in an application .
There are two functions: the ``load`` and the ``save``, an exception class
``CHLoneException`` and a set of constants. All other Python manipulations
can be performed using `pyCGNS <http://pycgns.sourceforge.net/index.html>`_
that provides a very large set of features on *CGNS/Python* trees.
Functions
~~~~~~~~~
The ``load`` reads
a *CGNS/HDF5* file and produces a *CGNS/Python* tree. The ``save`` takes a
*CGNS/Python* tree and writes the contents in a *CGNS/HDF5* file::
(tree,links,paths)=CHLone.load(filename,flags,depth,subtree,linkpaths,updatedict,maxdata)
status=CHLone.save(filename,tree,links,flags,depth,updatedict)
The default values are set to provide the simple interface if you have
*usual* files without links::
(tree,links,paths)=CHLone.load(filename)
status=CHLone.save(filename,tree)
The `S2P` prefix stands for *SIDS-to-Python*.
The complete interface of the load and the save are::
(tree,links,paths) <- load(filename,
flags = S2P_FDEFAULT,
depth = 0,
subtree = '',
linkpaths = [],
update = {},
maxdata = -1,
linkfull = False)
None <- save(filename,
tree,
links = [],
flags = S2P_FDEFAULT,
depth = 0,
linkpaths = [],
update = {},
skip = {},
linkfull = False)
The arguments and the return values are:
* **tree** (input/output):
The ``tree`` is the list representing the *CGNS/Python* tree.
The structure of a ``tree`` list is detailled
in :ref:`SIDS-to-Python <reference_sids_to_python>`.
There is no link information in this tree either for *load* or for *save*.
During the *load*, the links are silently replaced by the linked-to tree
they are referring. The ``links`` value keeps track of these link
references found while parsing the *CGNS/HDF5* file.
During the *save*, the sub-trees of the argument tree that refer to
other files by means of links are discarged. It is up to the user
application to save these sub-trees by an subsequent call to the API.
* **links** (input/output):
The ``links`` is a list with the link node information. It is returned
by a *load* and used as command parameters during the *save*. You can write
your own ``links`` list or change the list you obtain after a *load*.
The structure of a ``links`` list is detailled
in :ref:`SIDS-to-Python <reference_sids_to_python>` but *CHLone* adds a
flag at the end of each link entry list. These :ref:`flags <lkflags>`
indicate wether the link has been found or if the file was found but
not the target node or any other issue related to this link entry.
* **linkfull** (input/output):
A flag that sets the input/output format of a link entry in the list.
An entry is a sequence of two filenames (source, destination), the
directories where the files where found, wrt to the link search path,
and finally the nodes paths (source, destination). The status is a
integer that indicates wetehr the link has been found or not, and the reason
why it was not found. The ``linkfull`` flag is a boolean.
``False`` sets the old format, that is, for each link input or output
entry: ``[ destdir, destfile, destnode, srcnode, status ]`` (with
``status`` optional). The new format helps to handle links-to-links and
keeps tracks of the actual directories used during all the link traversals.
The entry format is then: `` [ srcdir, srcfile, srcnode, destdir, destfile,
destnode, status ]``.
* **paths** (output):
A list of ``( node-path, status )`` entries. The status indicates why
the node is in this returned list. So far, the only reason is because
the ``maxdata`` threshold and the ``NODATA`` flag have been set.
* **filename** (input):
The name of the target file, to read or to write. The ``filename`` can
be absolute or relative, it should be accessible in read/write depending
on the action you perform on it. The file extension is unused.
* **flags** (input):
You can control the behavior of a load/save using the
:ref:`flags <mapflags>`. You have to look a these carefully, the same
tree can load/save in a completely different way depending
on these ``flags``.
* **depth** (input):
This positive integer value sets the level of children the load/save
takes into account. For example, a depth of 2 would stop load/save
the CGNS tree once the children of the children of the start node
is reached. This level two child is used, its children are not.
If you want to have all the children, use a 0 ``depth`` which means
no limit on depth.
* **subtree** (input):
The ``subtree`` defines the start node of the load/save. It should be
an absolute path of an existing node in the argument filename.
All the nodes along this path are taken into account for load/save
actions.
* **linkpaths** (input):
The load may need a *link files search path* if your linked-to files
are not in the current directory. The ``linkpath`` argument is a list
of strings, during the load *CHLone* will look for linked-to files using
this list: it is parsed from the first element to the last,
the selected file is the first found in this directory list.
* **updatedict** (input):
A dictionnary of paths (string) as keys and *CGNS/Python* nodes as values.
When the load reaches a node with the path in the keys, the numpy value
is updated instead of creating a new array. You can pass your own array
with an already allocated memory zone or update and already loaded numpy.
* **maxdata** (input):
Set the maxmimum size of node data array to be read. To be used with
the ``NODATA`` flag. If a node has a data size above this max, the
node data type is set to ``MT`` (no data) and its path is stored into
the returned ``paths`` list to keep traceability on it.
* **skip** (input):
A list of *CGNS/Python* nodes paths (string) to skip at save time.
This list is used when you skp large data and you do not want the save to
overwrite the actual data you have on the disk.
The ''paths'' list you get with the load is the good candidate for this
parameter.
This parameter is **only** used for an update, not for a new file save.
.. warning::
The current directory is **not** in the link search path. So if your
linked-to file is in current directory, you should add `.` in the
link search path list. This is to avoid side effect while you are
running tools that create local copies.
.. warning::
If the filename is an absolute path name (not recommended !) then
you should add and empty path in the search path list.
.. _mapflags:
Flags
~~~~~
The interface choice was to reduce the number of functions and to tune
function call parameters to change the behavior or the ``load`` and ``save``.
This actually leads to a quite tricky flags manipulation
The flags are integers that can be OR-ed or XOR-ed to set/unset specific
parts in the read/write loops. Some falgs combinations are impossible, however
the irrelevant falgs are silently ignored and you may limit your
combinations to the examples hereafter.
The boolean operators are used for the flag settings::
flags=CHLone.FFOLLOWLINKS|CHLone.FTRACE
flags =flags&~CHLone.FTRACE
flags&=~CHLone.FTRACE
The table below gives all the `CHLone` flags.
+--------------------+------------------------------------------------------+
| *Flag variable* | *Function* |
+====================+======================================================+
| ``FNONE`` | Clear all flags, set to zero. |
+--------------------+------------------------------------------------------+
| ``FALL`` | Set all flags, set to one. |
+--------------------+------------------------------------------------------+
| ``FTRACE`` | Set the trace on, messages are sent to 'stdout' |
+--------------------+------------------------------------------------------+
| ``FFOLLOWLINKS`` | Continue to parse the linked-to tree \(1) |
+--------------------+------------------------------------------------------+
| ``FNODATA`` | Do not load large 'DataArray_t' \(1) \(3) |
+--------------------+------------------------------------------------------+
| ``FIGNORELINKS`` | Forget all link specifications. \(2) |
+--------------------+------------------------------------------------------+
| ``FCOMPRESS`` | Force chunking/compress \(8) |
+--------------------+------------------------------------------------------+
| ``FREVERSEDIMS`` | Reserved \(5) |
+--------------------+------------------------------------------------------+
| ``FOWNDATA`` | Forces the `numpy` flag ``\~NPY_OWNDATA`` \(1) |
+--------------------+------------------------------------------------------+
| ``FUPDATE`` | Save updates existing file \(6) \(2) |
+--------------------+------------------------------------------------------+
| ``FDELETEMISSING`` | Remove nodes not in input CGNS/Python tree \(2) |
+--------------------+------------------------------------------------------+
| ``FALTERNATESIDS`` | Changes some node types \(4) |
+--------------------+------------------------------------------------------+
| ``FUPDATEONLY`` | Create/update nodes only from update list \(1) |
+--------------------+------------------------------------------------------+
| ``FFORTRANFLAG`` | Forces the `numpy` flag ``\~NPY_FCONTIGUOUS`` \(1) |
+--------------------+------------------------------------------------------+
| ``FREADONLY`` | Reserved \(5) |
+--------------------+------------------------------------------------------+
| ``FNEW`` | Reserved \(5) |
+--------------------+------------------------------------------------------+
| ``FPROPAGATE`` | Force linked-to file write \(2) \(7) |
+--------------------+------------------------------------------------------+
| ``FDEBUG`` | Low level debug trace |
+--------------------+------------------------------------------------------+
The ``FDEFAULT`` flag corresponds
to ``( (FFOLLOWLINKS|FDELETEMISSING|FOWNDATA) & ~FREVERSEDIMS)``
There is no requirements or check on which flag can or cannot be associated
with another flag.
**Remarks:**
(1) Only when you are *loading* a tree.
(2) Only when you are *saving* a tree.
(3) Which means all ``DataArray_t`` actual memory zones will **NOT** be
released by Python.
(3) The term `large` has to be defined using the threshold parameter
``maxdata``. The *save* will **NOT** check if
the CGNS/Python tree was performed with the ``FNODATA`` flag on,
then you have to check by yourself that your *save* will not overwrite
an existing file with empty data! The last returned argument of the
``load``, the ``paths`` gives the list of the nodata nodes.
(4) The alternate types are not CGNS/SIDS types.
See :ref:`CGNS/Python <legacytypes>` mapping.
(5) Internal flag
(6) The file should exist, all new nodes are added, thus modifying the
children list of their parents. Existing nodes are updated only in
the case of value change. There no children removal, name or label
change.
(7) Not implemented yet or incompletely implemented.
(8) Compression is with a chunk size of 1024 for 1D and takes the last N-1
dimensions axis as chunk size for others (first axis is then size 1).
**Examples:**
Without any flag, the default load gets all the tree in the CGNS/HDF5 file,
follows the linked-to files as far as they are in the **current** directory.
See the link flags hereafter for details about bad link diagnostics::
import CHLone
(t,l,p)=CHLone.load('124Disk.hdf')
print t,l,p
The flag ``FFOLLOWLINKS`` is in the default flags, if we unset it the read
of the file will return the status ``LKIGNORED`` (see link flags). The linked-to
files are not parsed. The ``FFOLLOWLINKS`` is used for the load, while the
``FIGNORELINKS`` is for the save. It says that the link list passed as argument
to the save has to be ignored::
import CHLone
flags=CHLone.FDEFAULT&~CHLone.FFOLLOWLINKS
(t,l,p)=CHLone.load('124Disk.hdf',flags=flags)
print t,l,p
The CGNS/SIDS standard defines node types with the pattern ``<TYPE>_t`` for
most of the nodes. Some special nodes do have, for historical reasons, a
weird type name, for example ``"int[IndexDimension]"``. With the
``FALTERNATESIDS`` flag, you change the weird types with their ``<TYPE>_t``
as defined in the CGNS/Python mapping. This is not CGNS/SIDS compliant, the
flag is not set as default::
import CHLone
import CGNS.PAT.cgnsutils as CGU
flags=CHLone.FDEFAULT|CHLone.FALTERNATESIDS
(t,l,p)=CHLone.load('124Disk.hdf',flags=flags)
print 'load 1', CGU.getPathsByTypeSet(t,['Transform_t'])
(t,l,p)=CHLone.load('124Disk.hdf')
print 'load 2', CGU.getPathsByTypeSet(t,['Transform_t'])
print 'load 2', CGU.getPathsByTypeSet(t,['"int[IndexDimension]"'])
The ``FNODATA`` is one of the most useful flag, it allows to load the file
skeletton without the actual data. The falg has to be used with the ``maxdata``
parameter. If you define ``maxdata`` without ``FNODATA`` this parameter is
ignored. The third value of the load return is the list of paths of nodes
having their data incomplete. Then you actually have the node in the tree,
but the data is ``None`` if its size is more than ``maxdata``. The size is
the ``ndarray.size`` as returned by *numpy*::
import CHLone
import CGNS.PAT.cgnsutils as CGU
flags=CHLone.FDEFAULT|CHLone.FNODATA
(t,l,p)=CHLone.load('124Disk.hdf',flags=flags,maxdata=30)
print t,l,p
print 'value:',CGU.getValueByPath(t,'/Disk/zone3/GridCoordinates/CoordinateX')
More examples can be found in the
CGNS.MAP `examples <http://pycgns.sourceforge.net/MAP/examples.html>`_.
The use of *pyCGNS* is
helpful for paths manipulation, data search and parse and so on.
.. _lkflags:
Link flags
~~~~~~~~~~
The link flags can be OR-ed, this is required when the link fails, the actual
reason of the failure is described by another flag. The flags are defined as
a bitfield so that first value (in parenthesis) is 0, second is 1, then 2...
+-------------------+------------------------------------------------------+
| *Flag variable* | *Function* |
+===================+======================================================+
| S2P_LKOK | Link has been found and is OK (0) |
+-------------------+------------------------------------------------------+
| S2P_LKFAIL | Link has failed, check flags to get the error (1) |
+-------------------+------------------------------------------------------+
| S2P_LKNOFILE | Linked-to file not found (2) |
+-------------------+------------------------------------------------------+
| S2P_LKFILENOREAD | Linked-to file found but not readable (4) |
+-------------------+------------------------------------------------------+
| S2P_LKNONODE | Linked-to file found but linked-to node not found (8)|
+-------------------+------------------------------------------------------+
| S2P_LKLOOP | Linked-to file and node found, but loop detected (16)|
+-------------------+------------------------------------------------------+
| S2P_LKIGNORED | Link ignored by user request (64) |
+-------------------+------------------------------------------------------+
The flags ar OR-ed on failure, for example if you retrieve a link list from
a load with a status of 5, this means you have a failure on a file not
found (``S2P_LKFAIL`` and ``S2P_LKNOFILE``).
Links management
~~~~~~~~~~~~~~~~
The CGNS/HDF5 implementation defines the links as a symbolic link from
one node to another node. Both nodes can be in the same file or not.
In the case of a local link (a node that refers to another node in the
same file), the link entry has no directory/file.
The ``load`` parses all the links (it follows links) unless you unset
the ``S2P_FOLLOWLINKS``.
The ``save`` stops its actual file save when it encounters a link. Then
the user has to insure the save of the rest of the *CGNS/Python** tree
by himself
(see `examples <http://pycgns.sourceforge.net/MAP/examples.html>`_.
SIDS-to-Python Mapping
----------------------
.. toctree::
sids-to-python
.. warning::
The *root* node of an *HDF5* file is the ``/`` group with an attribute
name of ``HDF5 MotherNode``. This is an exception in the *CGNS/HDF5* tree,
all other nodes have the same *group name* as the value of
the ``name`` attribute. Then, if you want to use ``h5dump`` on a
*CGNS/HDF5* tree, keep in mind that the name ``HDF5 MotherNode`` is an
internal name and this should *not* be used by applications.
.. -------------------------------------------------------------------------