Python module

CGNS.MAP documentation

The quick start here gives some basic functions use, more detailled examples are in CGNS.MAP documentation. As a matter of fact, CGNS.MAP is no more than an import and a rename of the CHLone Python module, but actual use of CGNS/Python trees requires Python functions for examples. Then the actual use would be find in these CGNS.MAP pages.

The CHLone python module is the mapper. It has been moved from the CGNS.MAP module of pyCGNS to have a self-contained service for HDF5 and Python translation of CGNS trees. Now you can use the CGNS/Python interface to CHLone without pyCGNS.

Quick start

The mapper is a module implementing the SIDS-to-Python mapping, a CGNS/Python translation of the CGNS/HDF5 files, including links. There are only two functions in the module: the load and the save, the behavior of these function is driven by their arguments. Defaults arguments insure a simple load/save of CGNS/HDF5 files.

A simple exemple to load a CGNS/HDF5 file as a CGNS/Python tree:

import CHLone

T=CHLone.load("naca012.cgns")

The returned T is a tuple (tree,links,paths), the load argument is the file name, all other parameters are then set to their default value.

The tree value contains the actual CGNS/Python tree with linked-to files included (because the S2P_FOLLOWLINKS flag is on) and the links value is a list of links found during the HDF5 file parse. The last value of a link entry is the status of the link (see flags). The paths contains read time information about incomplete nodes, for example no data had been retrieved for this node (if the max data size is reached).

Saving a file is also quite easy:

import CHLone

CHLone.save("naca012.cgns",tree)

The tree value contains the actual CGNS/Python tree, for example the one you had with the load.

These two simple functions behaviors can be modified using flags and other parameters, see below the user interface for a reference of these arguments and the examples for a rather practical use.

User interface

It is a lightweight module, its purpose is to be as small as possible in order to be embedded separatly in an application . There are two functions: the load and the save, an exception class CHLoneException and a set of constants. All other Python manipulations can be performed using pyCGNS that provides a very large set of features on CGNS/Python trees.

Functions

The load reads a CGNS/HDF5 file and produces a CGNS/Python tree. The save takes a CGNS/Python tree and writes the contents in a CGNS/HDF5 file:

(tree,links,paths)=CHLone.load(filename,flags,depth,subtree,linkpaths,updatedict,maxdata)

status=CHLone.save(filename,tree,links,flags,depth,updatedict)

The default values are set to provide the simple interface if you have usual files without links:

(tree,links,paths)=CHLone.load(filename)

 status=CHLone.save(filename,tree)

The S2P prefix stands for SIDS-to-Python.

The complete interface of the load and the save are:

(tree,links,paths) <- load(filename,
                           flags       = S2P_FDEFAULT,
                           depth       = 0,
                           subtree     = '',
                           linkpaths   = [],
                           update      = {},
                           maxdata     = -1,
                           linkfull    = False)

              None <- save(filename,
                           tree,
                           links       = [],
                           flags       = S2P_FDEFAULT,
                           depth       = 0,
                           linkpaths   = [],
                           update      = {},
                           skip        = {},
                           linkfull    = False)

The arguments and the return values are:

  • tree (input/output): The tree is the list representing the CGNS/Python tree. The structure of a tree list is detailled in SIDS-to-Python. There is no link information in this tree either for load or for save.

    During the load, the links are silently replaced by the linked-to tree they are referring. The links value keeps track of these link references found while parsing the CGNS/HDF5 file.

    During the save, the sub-trees of the argument tree that refer to other files by means of links are discarged. It is up to the user application to save these sub-trees by an subsequent call to the API.

  • links (input/output): The links is a list with the link node information. It is returned by a load and used as command parameters during the save. You can write your own links list or change the list you obtain after a load. The structure of a links list is detailled in SIDS-to-Python but CHLone adds a flag at the end of each link entry list. These flags indicate wether the link has been found or if the file was found but not the target node or any other issue related to this link entry.

  • linkfull (input/output): A flag that sets the input/output format of a link entry in the list. An entry is a sequence of two filenames (source, destination), the directories where the files where found, wrt to the link search path, and finally the nodes paths (source, destination). The status is a integer that indicates wetehr the link has been found or not, and the reason why it was not found. The linkfull flag is a boolean. False sets the old format, that is, for each link input or output entry: [ destdir, destfile, destnode, srcnode, status ] (with status optional). The new format helps to handle links-to-links and keeps tracks of the actual directories used during all the link traversals. The entry format is then: `` [ srcdir, srcfile, srcnode, destdir, destfile, destnode, status ]``.

  • paths (output): A list of ( node-path, status ) entries. The status indicates why the node is in this returned list. So far, the only reason is because the maxdata threshold and the NODATA flag have been set.

  • filename (input): The name of the target file, to read or to write. The filename can be absolute or relative, it should be accessible in read/write depending on the action you perform on it. The file extension is unused.

  • flags (input): You can control the behavior of a load/save using the flags. You have to look a these carefully, the same tree can load/save in a completely different way depending on these flags.

  • depth (input): This positive integer value sets the level of children the load/save takes into account. For example, a depth of 2 would stop load/save the CGNS tree once the children of the children of the start node is reached. This level two child is used, its children are not. If you want to have all the children, use a 0 depth which means no limit on depth.

  • subtree (input): The subtree defines the start node of the load/save. It should be an absolute path of an existing node in the argument filename. All the nodes along this path are taken into account for load/save actions.

  • linkpaths (input): The load may need a link files search path if your linked-to files are not in the current directory. The linkpath argument is a list of strings, during the load CHLone will look for linked-to files using this list: it is parsed from the first element to the last, the selected file is the first found in this directory list.

  • updatedict (input): A dictionnary of paths (string) as keys and CGNS/Python nodes as values. When the load reaches a node with the path in the keys, the numpy value is updated instead of creating a new array. You can pass your own array with an already allocated memory zone or update and already loaded numpy.

  • maxdata (input): Set the maxmimum size of node data array to be read. To be used with the NODATA flag. If a node has a data size above this max, the node data type is set to MT (no data) and its path is stored into the returned paths list to keep traceability on it.

  • skip (input): A list of CGNS/Python nodes paths (string) to skip at save time. This list is used when you skp large data and you do not want the save to overwrite the actual data you have on the disk. The ‘’paths’’ list you get with the load is the good candidate for this parameter. This parameter is only used for an update, not for a new file save.

Warning

The current directory is not in the link search path. So if your linked-to file is in current directory, you should add . in the link search path list. This is to avoid side effect while you are running tools that create local copies.

Warning

If the filename is an absolute path name (not recommended !) then you should add and empty path in the search path list.

Flags

The interface choice was to reduce the number of functions and to tune function call parameters to change the behavior or the load and save. This actually leads to a quite tricky flags manipulation

The flags are integers that can be OR-ed or XOR-ed to set/unset specific parts in the read/write loops. Some falgs combinations are impossible, however the irrelevant falgs are silently ignored and you may limit your combinations to the examples hereafter.

The boolean operators are used for the flag settings:

flags=CHLone.FFOLLOWLINKS|CHLone.FTRACE

flags =flags&~CHLone.FTRACE
flags&=~CHLone.FTRACE

The table below gives all the CHLone flags.

Flag variable Function
FNONE Clear all flags, set to zero.
FALL Set all flags, set to one.
FTRACE Set the trace on, messages are sent to ‘stdout’
FFOLLOWLINKS Continue to parse the linked-to tree (1)
FNODATA Do not load large ‘DataArray_t’ (1) (3)
FIGNORELINKS Forget all link specifications. (2)
FCOMPRESS Force chunking/compress (8)
FREVERSEDIMS Reserved (5)
FOWNDATA Forces the numpy flag \~NPY_OWNDATA (1)
FUPDATE Save updates existing file (6) (2)
FDELETEMISSING Remove nodes not in input CGNS/Python tree (2)
FALTERNATESIDS Changes some node types (4)
FUPDATEONLY Create/update nodes only from update list (1)
FFORTRANFLAG Forces the numpy flag \~NPY_FCONTIGUOUS (1)
FREADONLY Reserved (5)
FNEW Reserved (5)
FPROPAGATE Force linked-to file write (2) (7)
FDEBUG Low level debug trace

The FDEFAULT flag corresponds to ( (FFOLLOWLINKS|FDELETEMISSING|FOWNDATA) & ~FREVERSEDIMS)

There is no requirements or check on which flag can or cannot be associated with another flag.

Remarks:

  1. Only when you are loading a tree.
  2. Only when you are saving a tree.
  3. Which means all DataArray_t actual memory zones will NOT be released by Python.
  1. The term large has to be defined using the threshold parameter maxdata. The save will NOT check if the CGNS/Python tree was performed with the FNODATA flag on, then you have to check by yourself that your save will not overwrite an existing file with empty data! The last returned argument of the load, the paths gives the list of the nodata nodes.
  2. The alternate types are not CGNS/SIDS types. See CGNS/Python mapping.
  3. Internal flag
  4. The file should exist, all new nodes are added, thus modifying the children list of their parents. Existing nodes are updated only in the case of value change. There no children removal, name or label change.
  5. Not implemented yet or incompletely implemented.
  6. Compression is with a chunk size of 1024 for 1D and takes the last N-1 dimensions axis as chunk size for others (first axis is then size 1).

Examples:

Without any flag, the default load gets all the tree in the CGNS/HDF5 file, follows the linked-to files as far as they are in the current directory. See the link flags hereafter for details about bad link diagnostics:

import CHLone
(t,l,p)=CHLone.load('124Disk.hdf')
print t,l,p

The flag FFOLLOWLINKS is in the default flags, if we unset it the read of the file will return the status LKIGNORED (see link flags). The linked-to files are not parsed. The FFOLLOWLINKS is used for the load, while the FIGNORELINKS is for the save. It says that the link list passed as argument to the save has to be ignored:

import CHLone
flags=CHLone.FDEFAULT&~CHLone.FFOLLOWLINKS
(t,l,p)=CHLone.load('124Disk.hdf',flags=flags)
print t,l,p

The CGNS/SIDS standard defines node types with the pattern <TYPE>_t for most of the nodes. Some special nodes do have, for historical reasons, a weird type name, for example "int[IndexDimension]". With the FALTERNATESIDS flag, you change the weird types with their <TYPE>_t as defined in the CGNS/Python mapping. This is not CGNS/SIDS compliant, the flag is not set as default:

import CHLone
import CGNS.PAT.cgnsutils as CGU

flags=CHLone.FDEFAULT|CHLone.FALTERNATESIDS
(t,l,p)=CHLone.load('124Disk.hdf',flags=flags)
print 'load 1', CGU.getPathsByTypeSet(t,['Transform_t'])

(t,l,p)=CHLone.load('124Disk.hdf')
print 'load 2', CGU.getPathsByTypeSet(t,['Transform_t'])
print 'load 2', CGU.getPathsByTypeSet(t,['"int[IndexDimension]"'])

The FNODATA is one of the most useful flag, it allows to load the file skeletton without the actual data. The falg has to be used with the maxdata parameter. If you define maxdata without FNODATA this parameter is ignored. The third value of the load return is the list of paths of nodes having their data incomplete. Then you actually have the node in the tree, but the data is None if its size is more than maxdata. The size is the ndarray.size as returned by numpy:

import CHLone
import CGNS.PAT.cgnsutils as CGU

flags=CHLone.FDEFAULT|CHLone.FNODATA
(t,l,p)=CHLone.load('124Disk.hdf',flags=flags,maxdata=30)
print t,l,p

print 'value:',CGU.getValueByPath(t,'/Disk/zone3/GridCoordinates/CoordinateX')

More examples can be found in the CGNS.MAP examples. The use of pyCGNS is helpful for paths manipulation, data search and parse and so on.

SIDS-to-Python Mapping

Warning

The root node of an HDF5 file is the / group with an attribute name of HDF5 MotherNode. This is an exception in the CGNS/HDF5 tree, all other nodes have the same group name as the value of the name attribute. Then, if you want to use h5dump on a CGNS/HDF5 tree, keep in mind that the name HDF5 MotherNode is an internal name and this should not be used by applications.