API Reference¶

Reading and Writing¶

Functions to read, inspect and write x3p files.

The types of the data are in x3pio.model.

x3pio.read(path: str | PathLike[str] | IO[bytes], *, verify: bool = True) → X3pFile¶

Read an x3p file into memory.

A set of known deviations from the standard is tolerated, see x3pio.codec; other deviations raise an error.

Parameters:

path – Path to an .x3p file, or an open binary file object. The content of a file in memory is read with X3pFile.loads().
verify – Check the MD5 checksums stored in the file, of main.xml and of the binary files. A checksum that is missing or malformed is not checked.

Returns:

A Profile, a Surface or a PointCloud, with the coordinates in metres (NaN for points that are not measured), metadata and extension files.

Raises:

X3pFormatError – If the file is not an x3p file, or is damaged beyond what can be tolerated.
X3pChecksumError – If verify and a stored checksum does not match, so the file was changed or damaged after it was written.

x3pio.info(path: str | PathLike[str] | IO[bytes], *, verify: bool = True) → FileInfo¶

Read the description of an x3p file without its coordinates.

The zip directory, main.xml and the vendor specific files are read. The binary data and validity files are not, so damage to them is not detected.

Parameters:

path – Path to an .x3p file, or an open binary file object that can seek, from which only the description is read.
verify – Check the MD5 checksum of the description, main.xml. The checksums of the data files are not checked, since the data is not read.

Returns:

The description as a FileInfo.

Raises:

io.UnsupportedOperation – If path is a stream that cannot seek: the description of a zip archive cannot be found without reading it whole.
X3pFormatError – If the file is not an x3p file, or is damaged beyond what can be tolerated.
X3pChecksumError – If verify and the checksum of main.xml does not match.

>>> import io
>>> import numpy as np
>>> import x3pio
>>> buf = io.BytesIO()
>>> x3pio.write(buf, np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6)
>>> summary = x3pio.info(buf)
>>> summary.feature_type.name, summary.shape, summary.layer_count
('SURFACE', (2, 3), 1)

class x3pio.FileDescription(*args, **kwargs)¶

The facts about an x3p file that do not depend on its coordinates.

X3pFile and FileInfo both provide them, so a function that needs only these facts takes either. The members are read-only.

property header: Header¶: The axes of Record1.

property revision: Revision¶: The revision of the standard that the file follows.

property placement: Placement¶: Where the view coordinate system lies in the global one.

property feature_type: FeatureType¶: The class of the data.

property storage: DataStorage¶: Whether the coordinates are stored as XML text or binary.

property metadata: Metadata | None¶: Record2, or None if the file has none.

class x3pio.FileInfo(header: Header, revision: Revision, placement: Placement, feature_type: FeatureType, layer_count: int, shape: tuple[int, ...], storage: DataStorage, metadata: Metadata | None, extensions: tuple[ExtensionFile, ...])¶

The description of an x3p file: everything except the coordinates and the validity bits.

Returned by x3pio.info(). It provides a FileDescription and adds the number and the shape of the layers. The fields cannot be assigned. metadata is a mutable object, and a copy of what was read.

Parameters:

header – The axes of Record1.
revision – The revision of the standard that the file follows.
placement – Where the view coordinate system lies in the global one.
feature_type – The class of the data.
layer_count – The number of layers: one for a point cloud.
shape – The shape of a layer: (rows, columns) of a surface, (points,) of a profile and of a point cloud.
storage – Whether the coordinates are stored as XML text or binary.
metadata – Record2, or None if the file has none.
extensions – The vendor specific IDs and the files that they name, in file order. Their content is not read.

class x3pio.ExtensionFile(vendor_id: str, path: str | None = None, size: int | None = None)¶

A VendorSpecificID of a file and the vendor file that it names, as info() lists it.

The content of the file is not read. Use read() and the extensions of the file for it.

Parameters:

vendor_id – The VendorSpecificID. In ISO 25178-72:2017 it is the one of the vendor, shared by all its files.
path – The path of the file in the container, relative to main.xml; None for an ID that names no file.
size – The size of the file in bytes, as the archive declares it; None without a file.

x3pio.write(path: str | PathLike[str] | IO[bytes], data: ArrayLike, *, x_scale: float, y_scale: float | None = None, **options: Unpack[ArrayOptions]) → None¶

Write a NumPy array of heights to an x3p file.

Parameters:

path – Destination path, or an open binary file object.
data – Heights in metres: (points,) for a profile, (rows, columns) for a surface or (layers, rows, columns) for a surface of several layers. NaN marks a point that is not measured. To write a profile of several layers, make it with Profile.from_array() and save it.
x_scale – Sampling interval along x, in metres.
y_scale – Sampling interval along y, in metres; required for a surface, and not given for a profile.
options – See ArrayOptions. To leave the metadata out, build the file with Surface.from_array(), set its metadata to None and save it.

Raises:

X3pFormatError – If data has the wrong shape, a scale is not positive, data does not fit z_type without overflow, or the offset or rotation is invalid.
TypeError – If placement is given without coordinate_system.

path also accepts an open binary stream, e.g. to avoid touching disk:

>>> import io
>>> import numpy as np
>>> import x3pio
>>> buf = io.BytesIO()
>>> x3pio.write(buf, np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6)
>>> file = x3pio.read(io.BytesIO(buf.getvalue()))
>>> file.layers[0].z.shape
(2, 3)

x3pio.write_points(path: str | PathLike[str] | IO[bytes], points: ArrayLike, **options: Unpack[PointOptions]) → None¶

Write an array of 3-D points to an x3p point cloud file.

Parameters:

path – Destination path, or an open binary file object.
points – (points, 3) array of x, y, z in metres.
options – See PointOptions.

Raises:

X3pFormatError – If points is not a finite (N, 3) array, or does not fit data_type without overflow.
TypeError – If placement is given without coordinate_system.

>>> import io
>>> import numpy as np
>>> import x3pio
>>> buf = io.BytesIO()
>>> x3pio.write_points(buf, np.array([[0.0, 0.0, 1.0], [1.0, 2.0, 3.0]]))
>>> x3pio.read(io.BytesIO(buf.getvalue())).layers[0].points().tolist()
[[0.0, 0.0, 1.0], [1.0, 2.0, 3.0]]

File Types¶

File types: profile, surface and point cloud.

An x3p file holds the 3-D coordinates of measured points in one of three shapes. The standard calls them feature types, and each is a type here:

A Profile is a line of points in a given order. The points need not lie on a straight line; each has the points before and after it as neighbours.
A Surface is a data matrix: a matrix of points with a neighbourhood relation, so that the neighbours of a point in the matrix are its neighbours in space. The points can lie on a regular grid, but do not have to.
A PointCloud is a list of points without any order or neighbourhood.

Each axis of a file is either incremental or absolute. The coordinate along an incremental axis is the index of the point in the matrix times the increment: a regular grid, with nothing stored for the axis. An absolute axis stores a coordinate for each point, as a number that is multiplied by the increment to give metres. The x and y axes of a profile and of a surface can be of either kind, the z axis, the height, is always absolute, and all three axes of a point cloud are absolute.

A point of a data matrix has the indices u, v and w: the column, the row and the layer. The matrix has SizeX, SizeY and SizeZ points along them, and the points are stored with u varying fastest, then v, then w. The arrays of a layer are therefore shaped (rows, columns). Only an incremental axis follows the index: x from u and y from v. The coordinates of an absolute axis are independent of the index.

Profiles and surfaces can have several layers (the standard speaks of multilayer data), which lie next to each other, so that each point also has the points at the same place in the adjacent layers as neighbours. Most files have a single layer.

The data of every file is reached the same way, through its layers: each layer holds its z, x and y arrays and gives its points. A point cloud presents its list of points as one layer, since the format has no layers for it. The layers share the axes and the placement. The file itself holds the layers and what belongs to the whole file: the revision of the standard, the metadata and the vendor extensions.

The stored coordinates, in metres, are in the view coordinate system of the file, and a Placement says where that system lies in the global one. Only the instrument or the software that wrote the file knows what the global system is.

class x3pio.X3pFile(layers: Iterable[Layer], *, revision: Revision = Revision.ISO25178_72_2017_DAM1, metadata: Metadata | None = None, extensions: VendorExtensions | None = None, storage: DataStorage = DataStorage.BINARY)¶

What a profile, a surface and a point cloud share: the file-level part of an x3p file.

There is no file of this type itself. read() returns a Profile, a Surface or a PointCloud, which are made with their own from_ methods. The data is in the layers; the file holds what belongs to all of them: the axes and the placement, which its layers share, the metadata, the vendor extensions and the storage form.

read() returns the type that matches the FeatureType of the file. The properties and methods of this class exist on all three types, so code that handles any file needs no check of the type:

>>> import io
>>> import numpy as np
>>> import x3pio
>>> blob = x3pio.Surface.from_array(np.zeros((2, 3)), x_scale=1.0, y_scale=1.0).dumps()
>>> x3p = x3pio.read(io.BytesIO(blob))
>>> type(x3p).__name__, x3p.feature_type.name
('Surface', 'SURFACE')
>>> [layer.points().shape for layer in x3p.layers]
[(2, 3, 3)]

The arrays of a layer belong to the file, so changing them changes the file, and it can be written again. The methods that return a copy, with_metadata, with_revision, with_z_type, with_placement and the others, share the arrays with the file as read-only views; make a writable file of one with copy(), or give a file other layers with with_layers():

>>> layer = x3p.layers[0]
>>> layer.z -= 1.0
>>> described = x3p.with_metadata(comment="levelled")
>>> float(layer.z[0, 0]), described.layers[0].z.flags.writeable
(-1.0, False)
>>> float(described.copy().layers[0].z[0, 0])
-1.0

The coordinates of a layer are in metres in the view coordinate system: the stored values multiplied by the axis increment. The placement says where that system lies in the global one, and CoordinateSystem names the two; no function applies it implicitly. NaN marks a point that is not measured.

A file keeps its layers, so file.layers[0] is the same object each time, and changing the arrays of a layer changes the file. The axes (header) and the placement belong to the layers and are immutable values; a copy with other ones is made with a with_ method of the file. The metadata, the extensions and the storage form are attributes of the file that can be changed.

The constructor is the low-level one and takes the layers as they are, without copying them: the from_ methods of the types make a file from heights, layers or points.

Parameters:

layers – The layers of the file, of the layer class of its type: they have to agree in shape, axes and placement, and a point cloud has one.
revision – The revision of the standard that the file follows, see Revision; Amd 1:2020 by default. Convert a file to another with with_revision().
metadata – Record2, or None if the file has none.
extensions – The vendor specific extension files, by their VendorSpecificID such as "http://www.vendor.com/mypath/a.xml": a mapping to bytes, with None for an ID without a file, see VendorExtensions. They are kept when a file is read and written again, and not interpreted. Without them the file has none.
storage – Whether the coordinates are written as XML text or binary. Not part of the comparison of two files.

Raises:

X3pFormatError – If the layers do not agree, or there are none.
TypeError – If a layer is not of the layer class of the type.

property storage: DataStorage¶: Whether the coordinates are written as XML text or binary.

property header: Header¶: The axes of the file, shared by the layers; an immutable value.

property revision: Revision¶: The revision of the standard that the file follows, see with_revision().

property placement: Placement¶: Where the view coordinate system lies in the global one; shared by the layers.

abstract property layers: Sequence[Layer]¶

The layers, which hold the data: several for a profile or a surface, one for a cloud.

Each layer gives its z, x, y, its valid mask and its points. The file keeps them: the same objects come back on every access.

property feature_type: FeatureType¶: The class of the data, as written in the FeatureType element.

with_z_type(data_type: DataType, *, increment: float | Literal['auto'] = 'auto') → Self¶

Return a copy that stores the heights in a different type.

The copy is independent, except that it shares the arrays with this file as read-only views.

Parameters:

data_type – The new storage type of the z axis.
increment – The new z increment, or "auto" for the smallest that fits the heights into data_type without overflow (1 for a floating point type).

Raises:

X3pFormatError – If the increment is not positive.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.array([[0.0, 1e-6]]), x_scale=1e-6, y_scale=1e-6)
>>> file.with_z_type(x3pio.DataType.INT16).header.z.data_type.name
'INT16'

globalized() → Self¶

Return a copy whose stored coordinates are the global ones.

The placement is applied to the coordinates once, so that z, x and y of the result are the global coordinates and its placement is the identity, except for incremental axes: their offset is the position of the first column or row and stays in the placement (the x_values of a layer plus placement.offset[0] are the global x coordinates).

Without rotation, the result keeps the matrix and its regular grid. An axis with a stored integer type and an offset becomes float64, since the shifted values no longer fall on multiples of its increment. With a rotation, a regular grid is turned into a skew one, so the result has absolute axes with x, y and z for all points.

The points of a layer of the result in CoordinateSystem.VIEW equal those of the same layer of this file in the global coordinate system. The result is independent of this file, except that it may share arrays with it as read-only views.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((1, 2)), x_scale=1.0, y_scale=1.0)
>>> moved = file.with_placement(x3pio.Placement.translation(0.0, 0.0, 5.0))
>>> moved.globalized().layers[0].z.tolist()
[[5.0, 5.0]]

with_placement(placement: Placement) → Self¶

Return a copy with another placement; the stored coordinates stay as they are.

The points move in the global coordinate system, since the view system that they are stored in is now somewhere else. Use transformed() to move the points relative to where they are now.

Parameters:: placement – Where the view coordinate system lies in the global one.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Profile.from_array(np.zeros(2), x_scale=1.0)
>>> moved = file.with_placement(x3pio.Placement.translation(5.0, 0.0, 0.0))
>>> moved.layers[0].points()[:, 0].tolist()
[5.0, 6.0]

transformed(transform: Placement) → Self¶

Return a copy whose points are moved in the global coordinate system.

The global coordinates become transform.apply(old global coordinates), and the stored coordinates stay as they are: the new placement is transform @ placement.

Parameters:: transform – The rigid transformation to apply to the points.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Profile.from_array(np.zeros(2), x_scale=1.0)
>>> up = x3pio.Placement.translation(0.0, 0.0, 1.0)
>>> file.transformed(up).layers[0].points()[:, 2].tolist()
[1.0, 1.0]

centered() → Self¶

Return a copy whose stored coordinates are centred on the origin.

The standard recommends choosing the offset so that the stored points are centred on the origin of the coordinate system. Each absolute axis is shifted by the middle of its range, and the placement moves by the same amount, so the global coordinates stay as they are. An incremental axis counts from the first point and is left alone.

The increment of an axis that is stored as an integer is chosen again for the centred range, as "auto" does, since the symmetric range uses the type best and the increment for the old range would waste most of it. An increment that was given for such an axis is replaced. The increments of other axes are not changed.

>>> import numpy as np
>>> import x3pio
>>> cloud = x3pio.PointCloud.from_points(np.array([[10.0, 0.0, 5.0], [12.0, 2.0, 7.0]]))
>>> centered = cloud.centered()
>>> centered.layer.x.tolist(), centered.placement.offset.tolist()
([-1.0, 1.0], [11.0, 1.0, 6.0])
>>> centered.layer.points().tolist() == cloud.layer.points().tolist()
True

copy() → Self¶

Return a copy that shares nothing with this file, with arrays that can be written to.

The with_ methods share the arrays of the file with their result as read-only views, to save memory. Make a copy of such a result to change its values in place.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((1, 2)), x_scale=1.0, y_scale=1.0)
>>> edited = file.with_metadata(comment="edited").copy()
>>> edited.layers[0].z[0, 0] = 1.0
>>> float(file.layers[0].z[0, 0]), float(edited.layers[0].z[0, 0])
(0.0, 1.0)

with_layers(layers: Iterable[Layer]) → Self¶

Return a file of other layers that keeps everything else of this one.

The metadata, the vendor extensions, the revision and the storage form stay; the layers bring their axes, their placement and their values, and are copied. This is the way to write back data whose shape or points differ from the ones that were read, such as a cropped array or a surface of other points, without losing the metadata.

Parameters:

layers – The layers of the new file, of the layer class of its type: they have to agree in shape, axes and placement, and a point cloud has one.

Raises:

X3pFormatError – If there is no layer, or the layers do not agree.
TypeError – If a layer is not of the layer class of the type.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.ones((4, 4)), x_scale=1e-6, y_scale=1e-6)
>>> cropped = x3pio.SurfaceLayer.from_array(
...     file.layers[0].z[:2, :2], x_scale=1e-6, y_scale=1e-6
... )
>>> smaller = file.with_layers([cropped])
>>> smaller.layers[0].z.shape, smaller.metadata == file.metadata
((2, 2), True)

with_metadata(**changes: Unpack[MetadataChanges]) → Self¶

Return a copy with changed metadata (Record2); this file stays as it is.

The copy is independent, except that it shares the arrays with this file as read-only views. A file without metadata is given a new record first, with the current UTC time as date, an empty instrument, and the probing type ProbingType.SOFTWARE; set the fields that describe the data set truthfully. To replace the whole record, assign file.metadata.

Parameters:: changes – The fields to change, see MetadataChanges.
Returns:: The changed copy.
Raises:: TypeError – If a field is not one of MetadataChanges.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((1, 2)), x_scale=1e-6, y_scale=1e-6)
>>> described = file.with_metadata(creator="Jane Doe", comment="Test surface")
>>> described.metadata.creator, file.metadata.creator
('Jane Doe', None)

with_revision(revision: Revision, *, drop_extensions: bool = False) → Self¶

Return a copy that follows another revision of the standard.

Data and metadata are kept, but ISO5436_2000 requires a calibration date, which is taken from the date of creation if there is none. The arrays are shared as read-only views. The vendor extensions are converted to Amd 1:2020, where each file becomes an ID of the vendor, a / and its path. They are not converted to ISO5436_2000; see drop_extensions.

Parameters:

revision – The revision to convert to.
drop_extensions – Drop the extensions when converting to ISO5436_2000, instead of failing.

Raises:

X3pFormatError – If there are extensions that would have to be dropped, and drop_extensions is not set, or a file cannot get an ID of its own.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((1, 2)), x_scale=1.0, y_scale=1.0)
>>> file.revision
<Revision.ISO25178_72_2017_DAM1: 'ISO25178-72:2017/DAM1'>
>>> file.with_revision(x3pio.Revision.ISO5436_2000).revision.value
'ISO5436 - 2000'

extensions_fit(revision: Revision) → bool¶

Return whether the vendor extensions are kept by with_revision().

That is the case without extensions, in the revision the file already follows, and in revision if the extensions can be converted to it. They cannot be converted to ISO5436_2000, where with_revision fails for them unless it drops them.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((1, 2)), x_scale=1.0, y_scale=1.0)
>>> file.extensions_fit(x3pio.Revision.ISO5436_2000)
True
>>> file.extensions["http://vendor.example/a.xml"] = b"<a/>"
>>> file.extensions_fit(x3pio.Revision.ISO5436_2000)
False

static loads(data: bytes, *, verify: bool = True) → X3pFile¶

Parse an in-memory x3p file; the counterpart of dumps().

read() reads a path or a stream.

Parameters:

data – The content of an .x3p file.
verify – Check the MD5 checksums stored in the file, of main.xml and of the binary files. A checksum that is missing or malformed is not checked.

Returns:

A Profile, a Surface or a PointCloud, as the file says.

Raises:

X3pFormatError – If data is not an x3p file, or is damaged beyond what can be tolerated.
X3pChecksumError – If verify and a stored checksum does not match the content, so the file was changed or damaged after it was written.

raw_data() → NDArray[Any]¶

Coded coordinates in the file’s native on-disk layout.

This is what a binary data file holds: one record per point in storage order (x fastest), with a field "x", "y" and "z" for each absolute axis, in the little-endian type of the axis. Each is coordinate / increment, rounded for an integer type. A point that is not measured is NaN in a floating point field, and 0 in an integer one, which relies on the validity file instead. Recomputed on each call, so it always reflects in-place edits.

Raises:: X3pFormatError – If the file cannot be written (see dumps()), for example if a value does not fit the type of its axis.

dumps(*, storage: DataStorage | None = None) → bytes¶

Serialize this file to an in-memory blob, overriding the storage form if given.

Parameters:

storage – Defaults to the form the file was read with (or storage if constructed directly).

Returns:

The serialized .x3p file contents. They always conform to the standard, whatever deviations the file had when read.

Raises:

X3pFormatError – If the file cannot be written as a conforming file: inconsistent data or axes, a value that does not fit its storage type, or missing required metadata.
TypeError – If storage is given and is not a DataStorage member.

save(path: str | PathLike[str] | IO[bytes], *, storage: DataStorage | None = None) → None¶

Write this file to disk or a binary stream, optionally overriding the storage form.

Parameters:

path – Destination path, or an open binary file object.
storage – Defaults to the form the file was read with (or storage if constructed directly).

Raises:

X3pFormatError – See dumps().

class x3pio.Profile(layers: Iterable[Layer], *, revision: Revision = Revision.ISO25178_72_2017_DAM1, metadata: Metadata | None = None, extensions: VendorExtensions | None = None, storage: DataStorage = DataStorage.BINARY)¶

A profile: a linear sequence of points, in one or several layers.

The data is in layers, one ProfileLayer for each layer, whose z is shaped (points,). The y axis of a profile has a single row, so y does not vary along a regular profile. The points need not lie on a straight line.

Make a profile from heights at equal steps along x:

>>> import numpy as np
>>> import x3pio
>>> scan = x3pio.Profile.from_array(np.sin(np.linspace(0, 6, 100)) * 1e-6, x_scale=1e-6)
>>> len(scan.layers), scan.layers[0].z.shape
(1, (100,))
>>> scan.layers[0].x_values[:2].tolist()
[0.0, 1e-06]

Several scans of the same line, as the layers of one file:

>>> profile = x3pio.Profile.from_layers([np.zeros(50), np.ones(50)], x_scale=1e-6)
>>> [float(layer.z[0]) for layer in profile.layers]
[0.0, 1.0]

A profile that follows an arbitrary path, from the coordinates of its points; each layer gives its points in the same shape again, points():

>>> path = np.array([[0.0, 0.0, 1.0], [1.0, 1.0, 2.0], [2.0, 1.0, 3.0]]) * 1e-3
>>> curved = x3pio.Profile.from_points(path)
>>> curved.layers[0].points().shape, curved.layers[0].x.tolist()
((3, 3), [0.0, 0.001, 0.002])

classmethod from_array(data: ArrayLike, *, x_scale: float, **options: Unpack[ArrayOptions]) → Profile¶

Build a profile from an array of heights.

The array is used as it is when it is already float64, not copied, so do not change it while the file is in use.

Parameters:

data – Heights in metres: (points,) for a profile of one layer, or (layers, points). NaN marks a point that is not measured.
x_scale – Sampling interval along x, in metres. A profile has a single row, so its y axis, which it does not use, gets the same increment.
options – See ArrayOptions.

Raises:

X3pFormatError – If data is not one- or two-dimensional, a scale is not positive, or global heights are given with a rotated placement.
TypeError – If placement is given without coordinate_system.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Profile.from_array(np.zeros((2, 10)), x_scale=1e-6)
>>> len(file.layers), file.layers[0].z.shape
(2, (10,))

classmethod from_layers(layers: Iterable[ArrayLike | ProfileLayer], *, x_scale: float | None = None, **options: Unpack[ArrayOptions]) → Profile¶

Build a profile of several layers, from arrays of heights or from layers.

The layers have to have the same number of points. Given as arrays, each is one layer of heights and x_scale says how far apart the points are. Given as ProfileLayer, they bring their axes and their placement, which have to agree, and no scale is given.

Parameters:

layers – Arrays of heights shaped (points,) in metres, with NaN for a point that is not measured, or layers, for example from other files.
x_scale – Sampling interval along x, in metres, for arrays.
options – See ArrayOptions; with layers only metadata, storage and revision apply.

Raises:

X3pFormatError – If there is no layer, arrays are not one-dimensional or layers differ in their number of points, their axes or their placement.
TypeError – If x_scale is left out for arrays, or given with layers.

>>> import numpy as np
>>> import x3pio
>>> first, second = np.zeros(5), np.ones(5)
>>> file = x3pio.Profile.from_layers([first, second], x_scale=1e-6)
>>> len(file.layers), file.layers[1].z.tolist()
(2, [1.0, 1.0, 1.0, 1.0, 1.0])
>>> again = x3pio.Profile.from_layers(file.layers)
>>> again.layers[1].z.tolist()
[1.0, 1.0, 1.0, 1.0, 1.0]

classmethod from_points(points: ArrayLike, **options: Unpack[PointOptions]) → Profile¶

Build a profile from the coordinates of its points, which may follow any path.

All three axes are absolute: every point stores its x, y and z, in the order along the profile. Use from_array() for a profile along a straight line at equal steps.

Parameters:

points – (points, 3) array of x, y, z in metres for a profile of one layer, or (layers, points, 3). NaN in z marks a point that is not measured.
options – See PointOptions.

Raises:

X3pFormatError – If points has another shape, does not fit data_type or does not fit data_type without overflow.
TypeError – If placement is given without coordinate_system.

>>> import numpy as np
>>> import x3pio
>>> path = np.array([[0.0, 0.0, 1.0], [1.0, 1.0, 2.0], [2.0, 1.0, 3.0]])
>>> file = x3pio.Profile.from_points(path)
>>> file.layers[0].z.shape, file.layers[0].x.tolist()
((3,), [0.0, 1.0, 2.0])

property layers: tuple[ProfileLayer, ...]¶: The layers, which the file keeps: the same objects come back on every access.

property is_regular_grid: bool¶

Whether the x and y axes are both incremental, so the points are evenly spaced.

The positions are then described completely by x_values of the layers. A profile that stores a coordinate for each point, such as one with unevenly spaced points or one along a diagonal, is not regular. x_values and spacing need only an incremental x axis.

>>> import numpy as np
>>> import x3pio
>>> x3pio.Profile.from_array(np.zeros(4), x_scale=1e-6).is_regular_grid
True
>>> x3pio.Profile.from_points(np.zeros((4, 3))).is_regular_grid
False

class x3pio.Surface(layers: Iterable[Layer], *, revision: Revision = Revision.ISO25178_72_2017_DAM1, metadata: Metadata | None = None, extensions: VendorExtensions | None = None, storage: DataStorage = DataStorage.BINARY)¶

A surface: a matrix of points, in one or several layers.

The data is in layers, one SurfaceLayer for each layer, whose z is shaped (rows, columns): y increases with the row index and x with the column index.

Make a surface on a regular grid from the heights and the steps along x and y. Then is_regular_grid is true, and the axes describe the grid completely:

>>> import numpy as np
>>> import x3pio
>>> heights = np.zeros((480, 640))
>>> surface = x3pio.Surface.from_array(heights, x_scale=1e-6, y_scale=1e-6)
>>> surface.is_regular_grid, surface.layers[0].z.shape
(True, (480, 640))
>>> layer = surface.layers[0]
>>> round(float(layer.x_values[-1]), 9), round(float(layer.y_values[-1]), 9)
(0.000639, 0.000479)

Several surfaces of the same size, as the layers of one file:

>>> layered = x3pio.Surface.from_layers([heights, heights], x_scale=1e-6, y_scale=1e-6)
>>> len(layered.layers), layered.layers[1].points().shape
(2, (480, 640, 3))

An irregular surface, whose points keep their rows and columns but not their even spacing, from the coordinates of its points; each layer gives them again, points():

>>> row0 = [[0.0, 0.0, 1.0], [1.0, 0.0, 2.0]]
>>> row1 = [[0.0, 1.5, 3.0], [1.2, 1.5, 4.0]]
>>> irregular = x3pio.Surface.from_points([row0, row1])
>>> irregular.is_regular_grid, irregular.layers[0].points().shape
(False, (2, 2, 3))

A surface that lies tilted or moved in the global coordinate system keeps its stored coordinates and carries a placement; the layers give either system:

>>> tilt = x3pio.Placement.from_axis_angle([1.0, 0.0, 0.0], 0.1)
>>> tilted = surface.with_placement(tilt).layers[0]
>>> view = tilted.points(coordinate_system=x3pio.CoordinateSystem.VIEW)
>>> view.shape == tilted.points().shape
True

classmethod from_array(data: ArrayLike, *, x_scale: float, y_scale: float | None = None, **options: Unpack[ArrayOptions]) → Surface¶

Build a surface from an array of heights.

The array is used as it is when it is already float64, not copied, so do not change it while the file is in use.

Parameters:

data – Heights in metres: (rows, columns) for a surface of one layer, or (layers, rows, columns). NaN marks a point that is not measured.
x_scale – Sampling interval along x, in metres.
y_scale – Sampling interval along y, in metres.
options – See ArrayOptions.

Raises:

X3pFormatError – If data is not two- or three-dimensional, a scale is not positive, or global heights are given with a rotated placement.
TypeError – If y_scale is left out, or placement is given without coordinate_system.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6)
>>> file.layers[0].z.shape
(2, 3)

classmethod from_layers(layers: Iterable[ArrayLike | SurfaceLayer], *, x_scale: float | None = None, y_scale: float | None = None, **options: Unpack[ArrayOptions]) → Surface¶

Build a surface of several layers, from arrays of heights or from layers.

The layers have to have the same shape. Given as arrays, each is one layer of heights and the scales say how far apart the points are. Given as SurfaceLayer, they bring their axes and their placement, which have to agree, and no scale is given.

Parameters:

layers – Arrays of heights shaped (rows, columns) in metres, with NaN for a point that is not measured, or layers, for example from other files.
x_scale – Sampling interval along x, in metres, for arrays.
y_scale – Sampling interval along y, in metres, for arrays.
options – See ArrayOptions; with layers only metadata, storage and revision apply.

Raises:

X3pFormatError – If there is no layer, arrays are not two-dimensional or layers differ in their shape, their axes or their placement.
TypeError – If a scale is left out for arrays, or given with layers.

>>> import numpy as np
>>> import x3pio
>>> scans = [np.zeros((3, 4)), np.ones((3, 4))]
>>> file = x3pio.Surface.from_layers(scans, x_scale=1e-6, y_scale=1e-6)
>>> len(file.layers), file.layers[0].z.shape
(2, (3, 4))
>>> float(x3pio.Surface.from_layers(list(reversed(file.layers))).layers[0].z.max())
1.0

classmethod from_points(points: ArrayLike, **options: Unpack[PointOptions]) → Surface¶

Build a surface from the coordinates of its points, on a grid that need not be regular.

This is the inverse of points(). All three axes are absolute: every point stores its x, y and z, and the points keep the neighbours of their rows and columns, which the standard makes the duty of the writer: a point’s neighbours in space have to be those in the array. Use from_array() for a regular grid.

Parameters:

points – (rows, columns, 3) array of x, y, z in metres for a surface of one layer, or (layers, rows, columns, 3). NaN in z marks a point that is not measured.
options – See PointOptions.

Raises:

X3pFormatError – If points has another shape, or does not fit data_type without overflow.
TypeError – If placement is given without coordinate_system.

>>> import numpy as np
>>> import x3pio
>>> row0 = [[0.0, 0.0, 1.0], [1.0, 0.0, 2.0]]
>>> row1 = [[0.0, 1.5, 3.0], [1.2, 1.5, 4.0]]
>>> file = x3pio.Surface.from_points([row0, row1])
>>> file.layers[0].z.shape, file.is_regular_grid
((2, 2), False)

property layers: tuple[SurfaceLayer, ...]¶: The layers, which the file keeps: the same objects come back on every access.

property is_regular_grid: bool¶

Whether the x and y axes are both incremental, so every layer is a regular grid.

The grid is then described completely by x_values and y_values of the layers.

class x3pio.PointCloud(layers: Iterable[Layer], *, revision: Revision = Revision.ISO25178_72_2017_DAM1, metadata: Metadata | None = None, extensions: VendorExtensions | None = None, storage: DataStorage = DataStorage.BINARY)¶

A point cloud: an unordered list of points without any topology.

The points are in layers, which holds the one PointLayer that a point cloud presents, with z, x and y shaped (points,). All three axes are absolute and a point cannot be missing.

Make a point cloud from an (N, 3) array of x, y and z in metres. points() of the layer returns an array of the same shape:

>>> import numpy as np
>>> import x3pio
>>> points = np.random.default_rng(0).uniform(0.0, 1e-3, (1000, 3))
>>> cloud = x3pio.PointCloud.from_points(points)
>>> cloud.layer.points().shape, cloud.layer.x.shape
((1000, 3), (1000,))

A point that is not measured cannot be kept, as a surface does with NaN: leave it out. The storage type decides the precision of the file, float64 by default:

>>> compact = x3pio.PointCloud.from_points(points, data_type=x3pio.DataType.INT16)
>>> stored = x3pio.X3pFile.loads(compact.dumps())
>>> bool(np.abs(stored.layer.points() - points).max() < 1e-7)
True

classmethod from_points(points: ArrayLike, **options: Unpack[PointOptions]) → PointCloud¶

Build a point cloud from an array of points.

Parameters:

points – (points, 3) array of x, y, z in metres.
options – See PointOptions.

Raises:

X3pFormatError – If points is not a finite (N, 3) array, or does not fit data_type without overflow.
TypeError – If placement is given without coordinate_system.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.PointCloud.from_points(np.array([[0.0, 1.0, 2.0]]))
>>> file.layer.points().tolist()
[[0.0, 1.0, 2.0]]

property layers: tuple[PointLayer, ...]¶: The one layer that a point cloud presents.

property layer: PointLayer¶: The one layer of the point cloud.

class x3pio.ArrayOptions¶

The options of Profile.from_array(), Surface.from_array() and write().

Parameters:

z_scale – Increment of the stored heights, or "auto" (the default) for the smallest one that fits the heights into z_type without overflow (1 for a floating point type, which stores metres).
z_type – Storage type of the heights, float64 by default.
metadata – Record2; defaults to a record naming x3pio as software with the current UTC time. The metadata is copied, and gets the current UTC time as its date if it has none.
storage – Whether to write the heights as XML text or binary, binary by default.
placement – Where the view coordinate system lies in the global one, see Placement. Without it the two systems are the same.
coordinate_system – Required with placement: the coordinate system that the heights are in. CoordinateSystem.VIEW: they are stored as they are, like z of a file that is read. CoordinateSystem.GLOBAL: they are global, so the z offset of the placement is taken off. That is impossible with a rotation, since the global heights of a rotated grid are not on a regular grid.
revision – The revision of the standard to write, see Revision; Amd 1:2020 by default.

class x3pio.PointOptions¶

The options that PointCloud.from_points() and write_points() share.

Parameters:

data_type – Storage type of all three coordinates, float64 by default.
scale – Increment of the stored coordinates of each axis, or "auto" (the default) for the smallest one that fits the axis into data_type without overflow (1 for a floating point type).
metadata – Record2; defaults to a record naming x3pio as software with the current UTC time. The metadata is copied, and gets the current UTC time as its date if it has none.
storage – Whether to write the coordinates as XML text or binary, binary by default.
placement – Where the view coordinate system lies in the global one, see Placement. Without it the two systems are the same.
coordinate_system – Required with placement: the coordinate system that the points are in. CoordinateSystem.GLOBAL: they are global, and the stored view coordinates are the inverse of the placement applied to them, so layer.points() returns them again. CoordinateSystem.VIEW: they are stored as they are, like x, y and z of a file that is read.
revision – The revision of the standard to write, see Revision; Amd 1:2020 by default.

class x3pio.MetadataChanges¶

The fields of the metadata (Record2) that X3pFile.with_metadata() can change.

Each field is optional; a field that is left out stays as it is. For the fields that may be absent, None removes the value.

Parameters:

date – Date and time of creation.
creator – Name of the person or institution that created the data.
calibration_date – Date and time of the last calibration.
comment – Free text describing the data set.
manufacturer – Manufacturer of the instrument or software.
model – Model of the instrument or software.
serial – Serial number of the instrument; may be empty for software.
version – Version number of the instrument, of the software, or of both.
probing_type – Kind of probing system.
probing_identification – Description of the probing system.

Layers¶

Layers of profiles and surfaces.

A profile or a surface of x3p can have several layers, which lie next to each other: the points keep the neighbours of their place in the layer, and each point also has the points at the same place of the adjacent layers as neighbours. Most files have a single layer. A point cloud has no layers in the format; x3pio presents its list of points as one layer.

A layer is a complete topography: it holds the arrays of its points together with the axes and the placement of the file, so it gives its points without the file. A file keeps its layers: asking for them again gives the same objects, and layers are equal if their axes, placement and values are.

The axes and the placement of a layer are immutable values, and its arrays can be written to. To make a layer of other data, such as a cropped array or other points, make it with from_array or from_points and give it to a file with X3pFile.with_layers.

class x3pio.Layer(z: ArrayLike, x: ArrayLike | None = None, y: ArrayLike | None = None, *, header: Header, placement: Placement | None = None)¶

A layer of a profile, a surface or a point cloud: the data and the geometry that all share.

A layer knows its axes and its placement, so it can give its points without the file. The axes and the placement are immutable values; the arrays can be written to, see z. Two layers are equal if they are of the same kind and have equal axes, placement and values, NaN equal to NaN. Since the values can change, a layer is not hashable.

The arrays are held as float64. An array of that type is kept as it is, so that writing to the layer writes to it; an array of another type is converted, which makes a copy.

Parameters:

z – The heights, shaped like the layer; NaN for a point that is not measured.
x – The x coordinates if the x axis is absolute, otherwise None.
y – The y coordinates if the y axis is absolute, otherwise None.
header – The axes of the file, with their increments.
placement – Where the view coordinate system lies in the global one; the identity by default.

property header: Header¶: The axes of the layer, with their increments; an immutable value.

property placement: Placement¶: Where the view coordinate system lies in the global one; an immutable value.

property z: NDArray[np.float64]¶

The heights in the stored shape; NaN for a point that is not measured.

The array can be written to: layer.z[0, 1] = 1.0, layer.z -= 1.0 and layer.z = other all change the file that holds the layer. Assigning fills the array and so needs a value of the same shape, or a single number; it does not replace the array. A layer of a copy that shares its arrays is read-only; X3pFile.copy() makes a writable copy.

Raises:: ValueError – On assigning a value of another shape, or to a read-only layer.

property x: NDArray[np.float64] | None¶

The stored x coordinates in metres if the x axis is absolute, otherwise None.

With an incremental axis the coordinates follow from the place of a point, see x_values, and there is nothing to write to. Assigning works like for z.

Raises:: X3pFormatError – On assigning to a layer whose x axis is incremental; make a layer from the coordinates of its points, with from_points.

property y: NDArray[np.float64] | None¶

The stored y coordinates in metres if the y axis is absolute, otherwise None.

See x.

property shape: tuple[int, ...]¶: The shape of the layer.

property valid: NDArray[np.bool_]¶: Boolean mask of the points that are measured.

property is_regular_grid: bool¶

Whether the x and y axes are both incremental, so the points lie on a regular grid.

The grid is then described completely by x_values, y_values, spacing and extent of the layer. It is never so for a point cloud, whose axes are absolute, and not for a profile or a surface with a stored x or y coordinate for each point.

points(*, coordinate_system: CoordinateSystem = CoordinateSystem.GLOBAL) → NDArray[np.float64]¶

Return the points as an array of x, y, z in metres, shaped like the layer.

The shape is z.shape + (3,): (rows, columns, 3) for a surface layer, (points, 3) for a profile layer and for a point cloud. A point that is not measured has NaN in z, and in x and y too if the placement turns the points, since the rotation couples them. The from_points constructors take this shape.

The array is made for this layer alone and is a new one, three times the size of z. points[layer.valid] gives the (N, 3) array of the measured points that point cloud libraries take, and reshape(-1, 3) all of them.

Parameters:: coordinate_system – CoordinateSystem.GLOBAL, with the placement applied, which is the default, or CoordinateSystem.VIEW, as stored.

>>> import numpy as np
>>> import x3pio
>>> layer = x3pio.Surface.from_array(
...     np.array([[0.0, 1.0], [np.nan, 3.0]]), x_scale=1.0, y_scale=1.0
... ).layers[0]
>>> layer.points().shape
(2, 2, 3)
>>> layer.points()[layer.valid].shape
(3, 3)

class x3pio.ProfileLayer(z: ArrayLike, x: ArrayLike | None = None, y: ArrayLike | None = None, *, header: Header, placement: Placement | None = None)¶

One layer of a Profile.

Writing to the arrays changes the file that holds the layer. z, x and y are shaped (points,).

property x_values: NDArray[np.float64]¶

X coordinates of the points in metres: x_u = (u - 1) * increment.

Raises:: ValueError – If the x axis is absolute.

property spacing: float¶

The distance between neighbouring points along x in metres: the x increment.

Raises:: ValueError – If the x axis is absolute.

classmethod from_array(data: ArrayLike, *, x_scale: float, **options: Unpack[ArrayOptions]) → ProfileLayer¶

Make a layer alone from an array of heights, see Profile.from_array().

>>> import numpy as np
>>> import x3pio
>>> layer = x3pio.ProfileLayer.from_array(np.zeros(4), x_scale=1e-6)
>>> layer.points().shape
(4, 3)

classmethod from_points(points: ArrayLike, **options: Unpack[PointOptions]) → ProfileLayer¶: Make a layer alone from its points, see Profile.from_points().

property header: Header¶: The axes of the layer, with their increments; an immutable value.

property is_regular_grid: bool¶

Whether the x and y axes are both incremental, so the points lie on a regular grid.

property placement: Placement¶: Where the view coordinate system lies in the global one; an immutable value.

points(*, coordinate_system: CoordinateSystem = CoordinateSystem.GLOBAL) → NDArray[np.float64]¶

Return the points as an array of x, y, z in metres, shaped like the layer.

Parameters:: coordinate_system – CoordinateSystem.GLOBAL, with the placement applied, which is the default, or CoordinateSystem.VIEW, as stored.

>>> import numpy as np
>>> import x3pio
>>> layer = x3pio.Surface.from_array(
...     np.array([[0.0, 1.0], [np.nan, 3.0]]), x_scale=1.0, y_scale=1.0
... ).layers[0]
>>> layer.points().shape
(2, 2, 3)
>>> layer.points()[layer.valid].shape
(3, 3)

property shape: tuple[int, ...]¶: The shape of the layer.

property valid: NDArray[np.bool_]¶: Boolean mask of the points that are measured.

property x: NDArray[np.float64] | None¶

The stored x coordinates in metres if the x axis is absolute, otherwise None.

With an incremental axis the coordinates follow from the place of a point, see x_values, and there is nothing to write to. Assigning works like for z.

Raises:: X3pFormatError – On assigning to a layer whose x axis is incremental; make a layer from the coordinates of its points, with from_points.

property y: NDArray[np.float64] | None¶

The stored y coordinates in metres if the y axis is absolute, otherwise None.

See x.

property z: NDArray[np.float64]¶

The heights in the stored shape; NaN for a point that is not measured.

Raises:: ValueError – On assigning a value of another shape, or to a read-only layer.

class x3pio.SurfaceLayer(z: ArrayLike, x: ArrayLike | None = None, y: ArrayLike | None = None, *, header: Header, placement: Placement | None = None)¶

One layer of a Surface.

Writing to the arrays changes the file that holds the layer. z, x and y are shaped (rows, columns).

Make a layer of heights on a regular grid. spacing is the distance between the points and extent the cell edges in the form that the extent argument of an image plot takes; y increases with the row index:

>>> import numpy as np
>>> import x3pio
>>> heights = np.array([[0.0, 1.0, np.nan], [2.0, 3.0, 4.0]]) * 1e-6
>>> layer = x3pio.SurfaceLayer.from_array(heights, x_scale=2e-6, y_scale=1e-6)
>>> layer.spacing, layer.valid.tolist()
((2e-06, 1e-06), [[True, True, False], [True, True, True]])
>>> [round(value * 1e6, 3) for value in layer.extent]
[-1.0, 5.0, -0.5, 1.5]
>>> layer.points().shape
(2, 3, 3)

property x_values: NDArray[np.float64]¶

X coordinates of the columns in metres: x_u = (u - 1) * increment.

Raises:: ValueError – If the x axis is absolute.

property y_values: NDArray[np.float64]¶

Y coordinates of the rows in metres: y_v = (v - 1) * increment.

y increases with the row index.

Raises:: ValueError – If the y axis is absolute.

property spacing: tuple[float, float]¶

The distance between neighbouring points along x and along y in metres.

These are the increments of the axes, so nobody has to multiply them by the size.

Raises:: ValueError – If the x or the y axis is absolute.

property extent: tuple[float, float, float, float]¶

The edges of the grid cells as (x0, x1, y0, y1) in metres, in the view system.

A point lies in the middle of its cell, so the extent reaches half a step beyond the first and the last point: for x_values it is -dx / 2 to x_values[-1] + dx / 2. This is what the extent of an image plot of the heights takes, with the origin in the lower left corner, since y grows with the row:

plt.imshow(layer.z, origin="lower", extent=layer.extent)

Raises:: ValueError – If the x or the y axis is absolute.

classmethod from_array(data: ArrayLike, *, x_scale: float, y_scale: float, **options: Unpack[ArrayOptions]) → SurfaceLayer¶

Make a layer alone from an array of heights, see Surface.from_array().

>>> import numpy as np
>>> import x3pio
>>> layer = x3pio.SurfaceLayer.from_array(np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6)
>>> layer.points().shape
(2, 3, 3)

classmethod from_points(points: ArrayLike, **options: Unpack[PointOptions]) → SurfaceLayer¶: Make a layer alone from its points, see Surface.from_points().

property header: Header¶: The axes of the layer, with their increments; an immutable value.

property is_regular_grid: bool¶

Whether the x and y axes are both incremental, so the points lie on a regular grid.

property placement: Placement¶: Where the view coordinate system lies in the global one; an immutable value.

points(*, coordinate_system: CoordinateSystem = CoordinateSystem.GLOBAL) → NDArray[np.float64]¶

Return the points as an array of x, y, z in metres, shaped like the layer.

Parameters:: coordinate_system – CoordinateSystem.GLOBAL, with the placement applied, which is the default, or CoordinateSystem.VIEW, as stored.

>>> import numpy as np
>>> import x3pio
>>> layer = x3pio.Surface.from_array(
...     np.array([[0.0, 1.0], [np.nan, 3.0]]), x_scale=1.0, y_scale=1.0
... ).layers[0]
>>> layer.points().shape
(2, 2, 3)
>>> layer.points()[layer.valid].shape
(3, 3)

property shape: tuple[int, ...]¶: The shape of the layer.

property valid: NDArray[np.bool_]¶: Boolean mask of the points that are measured.

property x: NDArray[np.float64] | None¶

The stored x coordinates in metres if the x axis is absolute, otherwise None.

With an incremental axis the coordinates follow from the place of a point, see x_values, and there is nothing to write to. Assigning works like for z.

Raises:: X3pFormatError – On assigning to a layer whose x axis is incremental; make a layer from the coordinates of its points, with from_points.

property y: NDArray[np.float64] | None¶

The stored y coordinates in metres if the y axis is absolute, otherwise None.

See x.

property z: NDArray[np.float64]¶

The heights in the stored shape; NaN for a point that is not measured.

Raises:: ValueError – On assigning a value of another shape, or to a read-only layer.

class x3pio.PointLayer(z: ArrayLike, x: ArrayLike | None = None, y: ArrayLike | None = None, *, header: Header, placement: Placement | None = None)¶

The points of a PointCloud, as the one layer that it presents.

The format has no layers for a point cloud. x3pio presents its list of points as a single layer, so that every file is used the same way. z, x and y are shaped (points,) and all three axes are absolute.

classmethod from_points(points: ArrayLike, **options: Unpack[PointOptions]) → PointLayer¶

Make a layer alone from an (N, 3) array, see PointCloud.from_points().

>>> import numpy as np
>>> import x3pio
>>> layer = x3pio.PointLayer.from_points(np.array([[0.0, 1.0, 2.0]]))
>>> layer.points().tolist()
[[0.0, 1.0, 2.0]]

property header: Header¶: The axes of the layer, with their increments; an immutable value.

property is_regular_grid: bool¶

Whether the x and y axes are both incremental, so the points lie on a regular grid.

property placement: Placement¶: Where the view coordinate system lies in the global one; an immutable value.

points(*, coordinate_system: CoordinateSystem = CoordinateSystem.GLOBAL) → NDArray[np.float64]¶

Return the points as an array of x, y, z in metres, shaped like the layer.

Parameters:: coordinate_system – CoordinateSystem.GLOBAL, with the placement applied, which is the default, or CoordinateSystem.VIEW, as stored.

>>> import numpy as np
>>> import x3pio
>>> layer = x3pio.Surface.from_array(
...     np.array([[0.0, 1.0], [np.nan, 3.0]]), x_scale=1.0, y_scale=1.0
... ).layers[0]
>>> layer.points().shape
(2, 2, 3)
>>> layer.points()[layer.valid].shape
(3, 3)

property shape: tuple[int, ...]¶: The shape of the layer.

property valid: NDArray[np.bool_]¶: Boolean mask of the points that are measured.

property x: NDArray[np.float64] | None¶

The stored x coordinates in metres if the x axis is absolute, otherwise None.

With an incremental axis the coordinates follow from the place of a point, see x_values, and there is nothing to write to. Assigning works like for z.

Raises:: X3pFormatError – On assigning to a layer whose x axis is incremental; make a layer from the coordinates of its points, with from_points.

property y: NDArray[np.float64] | None¶

The stored y coordinates in metres if the y axis is absolute, otherwise None.

See x.

property z: NDArray[np.float64]¶

The heights in the stored shape; NaN for a point that is not measured.

Raises:: ValueError – On assigning a value of another shape, or to a read-only layer.

Coordinates¶

Coordinate systems and the placement of the view system in the global one.

An x3p file stores its coordinates in its own system, the view coordinate system: the system in which the instrument or the software described the points. The file also says where that system lies in the global coordinate system, the one the data set belongs to, by a rotation and an offset. Both systems are right-handed, and a pure rotation keeps that. The standard writes this as one formula for the stored values s of the three axes, their increments I and the offsets O:

global = R @ (I * s) + O

I * s is what this library calls the view coordinates, in metres, and R and O are what Placement holds. Nothing in a file is transformed implicitly: the stored coordinates stay as they are, and a CoordinateSystem says which of the two systems a function returns or expects.

Without a placement the two systems are the same. points(coordinate_system=...) of a layer returns its points in either system, with_placement and transformed change where the points lie in the global system, globalized makes the global coordinates the stored ones, and centered centres the stored coordinates on the origin, as the standard recommends. To write a file with a placement, pass placement= and coordinate_system= to the constructors; coordinate_system says which system the values you give are in.

class x3pio.CoordinateSystem(*values)¶

The coordinate system that coordinates are given in, or are asked for.

The same points have different coordinates in the two systems, unless the placement of the file is the identity.

VIEW = 1¶: The system the file stores its coordinates in: the stored values times their increments.

GLOBAL = 2¶: The system the data set belongs to: the view system turned and moved by the placement.

class x3pio.Placement(rotation: ArrayLike | None = None, offset: ArrayLike | None = None)¶

The pose of the view coordinate system in the global one: a rotation and an offset.

This is the rotation matrix and the offset of the standard, which together make up its coordinate transformation, without the increments: they scale the stored numbers and belong to the axes of the file.

A point with the view coordinates p has the global coordinates rotation @ p + offset. A placement is immutable. Placements compose with @, the way matrices do: (a @ b).apply(p) is a.apply(b.apply(p)).

The offset of an x3p file is the global position of the origin of the view system. For an axis that is incremental, whose coordinates are the position of the point in the matrix times the increment, the origin is the first point.

>>> import numpy as np
>>> import x3pio
>>> quarter_turn = x3pio.Placement.from_axis_angle([0, 0, 1], np.pi / 2)
>>> p = x3pio.Placement.translation(1.0, 0.0, 0.0) @ quarter_turn
>>> p.apply([[1.0, 0.0, 0.0]]).round(12).tolist()
[[1.0, 1.0, 0.0]]
>>> p.inverse().apply(p.apply([[1.0, 2.0, 3.0]])).round(12).tolist()
[[1.0, 2.0, 3.0]]

Parameters:

rotation – The 3 x 3 rotation of the view system into the global one, the identity by default. It must be a pure rotation, without mirroring or scaling.
offset – The offsets of the x, y and z axis in metres, zero by default.

Raises:

X3pFormatError – If the rotation is not a rotation, or the offset is not three finite numbers.

classmethod identity() → Placement¶: Return the placement that leaves the coordinates as they are.

classmethod translation(x: float, y: float, z: float) → Placement¶: Return a placement that only moves, by x, y and z in metres.

classmethod from_axis_angle(axis: ArrayLike, angle: float) → Placement¶

Return a placement that only turns, by angle radians about axis.

The turn is counterclockwise when the axis points at the viewer.

Parameters:

axis – The direction of the axis, three numbers that need not be of length one.
angle – The angle in radians.

Raises:

X3pFormatError – If the axis is not three finite numbers that are not all zero.

property rotation: NDArray[float64]¶: The 3 x 3 rotation matrix, read-only.

property offset: NDArray[float64]¶: The offsets of the x, y and z axis in metres, read-only.

property has_rotation: bool¶: Whether the placement turns anything, which the identity rotation does not.

property is_identity: bool¶: Whether the placement leaves the coordinates as they are.

apply(points: ArrayLike) → NDArray[float64]¶

Turn view coordinates into global ones: rotation @ point + offset for each point.

Parameters:: points – View coordinates in metres, shaped (..., 3). A point with NaN in any coordinate has NaN in all three of its result if the placement has a rotation, since the rotation mixes them; without a rotation, the others stay known.
Returns:: The global coordinates, shaped like points.

inverse() → Placement¶: Return the placement that undoes this one: global coordinates into view ones.

Header and Metadata¶

Axes, revision and metadata of an x3p file.

The main.xml document of a file has four records. The first says how the three axes are stored, here Header of Axis: whether each is incremental or absolute, with what data type, and by which increment its stored numbers are multiplied to give metres. It also names the revision of the standard that the file follows, its Revision, which belongs to the file as a whole. The second, here Metadata, is optional: when and by whom the data was made, with which instrument and probing system, and when the instrument was last calibrated. Where the coordinates lie in space is the placement of the file, Placement. The third record holds the data, and the fourth the checksum of the document; those are the business of the codec.

The standard asks for a creation date and a probing type in the metadata, so metadata without them cannot be written. The probing system is one of three kinds: contacting, non contacting, and software, for data that was made or changed by software and not measured.

class x3pio.Header(x: Axis = <factory>, y: Axis = <factory>, z: Axis = <factory>)¶

The three axes of x3p (Record1/Axes), an immutable value.

The layers of a file share one header. To change an axis, make another header with dataclasses.replace(), or use the methods of the file that do it. The revision of the standard is not part of it, since it belongs to the whole file: see revision.

The class of the data is not part of it: it is the type of the file, see Profile, Surface and PointCloud, and so is the rotation and the offset of the view coordinate system, see Placement.

Parameters:

x – Description of the x axis.
y – Description of the y axis.
z – Description of the z axis, which must be absolute.

class x3pio.Axis(axis_type: AxisType = AxisType.INCREMENTAL, data_type: DataType = DataType.FLOAT64, increment: float = 1.0)¶

Description of one coordinate axis (CX, CY or CZ), an immutable value.

A stored coordinate s (a point’s matrix index for an incremental axis) gives the view coordinate increment * s in metres. Where the view system lies in the global one is the Placement of the file, not a property of an axis.

Parameters:

axis_type – Incremental or absolute; the z axis is always absolute.
data_type – Storage type of the stored coordinates. Irrelevant for an incremental axis, which stores none.
increment – Scale factor in metres: the sampling interval of an incremental axis, or the scale of the stored values of an absolute axis (1 to store metres directly).

property is_incremental: bool¶: Whether the coordinate is derived from the matrix index.

class x3pio.AxisType(*values)¶

Kind of an axis, as written in the AxisType element.

An incremental axis derives its coordinate from the matrix index and stores nothing; an absolute axis stores a coordinate for each point.

class x3pio.FeatureType(*values)¶

Class of the 3-D data in a file, as written in the FeatureType element.

A profile or a surface is stored as a matrix of points whose neighbours in the matrix are also neighbours in space; a point cloud is an unordered list of points without any topology.

class x3pio.Revision(*values)¶

Revision of the standard a file follows, as embedded verbatim in its Revision element.

Members are named after the standard; their string value is the exact text of the Revision element. The two revisions differ in how a file holds vendor specific extensions: ISO 25178-72:2017 has a single VendorSpecificID for the extension as a whole, and its files lie in the container under any name. Its Amd 1:2020 registers each extension file by an ID of its own. Both are held by VendorExtensions.

X3pFile.revision is the revision of a file. A file is written in the revision it was read in, and a new file follows Amd 1:2020 unless revision= says otherwise.

Reading: a Revision with the text of Amd 1:2020 means ISO25178_72_2017_DAM1 and one with the text of ISO 25178-72:2017 means ISO5436_2000. A missing or empty Revision means ISO5436_2000 too. Any other text raises X3pFormatError, since a revision that is not known may store its data differently. A file with several VendorSpecificID is read as Amd 1:2020, since only Amd 1:2020 allows that.
Writing: the revision is written as the Revision element and decides how vendor extensions are written. Axes always have DataType, Increment and Offset, which both schemas accept.
Calibration date: the text of the standard leaves the CalibrationDate out when there was no calibration, but its schema requires it, and the schema of Amd 1:2020 makes it optional. ISO25178_72_2017_DAM1 leaves it out when calibration_date is None, while ISO5436_2000 writes the date of creation to be valid against its schema. This states a calibration that did not happen, so set the date if you know it. A file without metadata has no CalibrationDate.
Coordinates: both revisions use the definition of Amd 1:2020. The Increment scales the stored values of an absolute axis, and the first point of an incremental axis is at its offset. The text of ISO 25178-72:2017 does not say where the index of an incremental axis starts.

with_revision() converts a file to the other revision.

ISO5436_2000 = 'ISO5436 - 2000'¶: ISO 25178-72:2017, the ISO 5436-2 XML format of openGPS.

ISO25178_72_2017_DAM1 = 'ISO25178-72:2017/DAM1'¶: ISO 25178-72:2017/Amd 1:2020.

classmethod resolve(revision: str) → Revision¶

Return the revision of the text of a Revision element.

The text of Amd 1:2020 gives ISO25178_72_2017_DAM1 and the ISO5436 - 2000 of ISO 25178-72:2017 gives ISO5436_2000, each also with another dash than the hyphen, such as the en dash that some software writes. An empty text, as of a missing element, gives ISO5436_2000.

Raises:: X3pFormatError – If the text is none of these.

property has_vendor_ids: bool¶: Whether every vendor extension file has a VendorSpecificID of its own.

property requires_calibration_date: bool¶

Whether the schema requires a CalibrationDate in the metadata.

The text of the standard says the element is missing for an instrument that has not been calibrated, but its schema requires it; Amd 1:2020 makes it optional.

class x3pio.Metadata(date: datetime | None = None, creator: str | None = None, instrument: Instrument = <factory>, calibration_date: datetime | None = None, probing_system: ProbingSystem = <factory>, comment: str | None = None)¶

Record2 of x3p: the optional metadata of a data set.

Parameters:

date – Date and time of creation. It is written as YYYY-MM-DDThh:mm:ss.sTZD, with a decimal fraction and a time zone offset such as +02:00. A value without a time zone is a local time, as in ISO 8601, and is written with the offset of the local time zone; a date without a zone that is read from a file is taken the same way. None if a file read held no recognizable value, which must be fixed before writing; a record needs one to be written, and for_software() sets the current UTC time.
creator – Name of the person or institution that created the data.
instrument – The instrument or software that created the data.
calibration_date – Date and time of the last calibration, written like date; None if there was no calibration. The schema of the revision ISO5436_2000 requires the element, so a file in it is written with the date of creation if there is none, and with_revision() sets it so.
probing_system – The probing system used.
comment – Free text describing the data set.

classmethod for_software(name: str, version: str, *, date: datetime | None = None) → Metadata¶

Return the record of data that software made, not a measurement.

The software is both the manufacturer and the model of the instrument, without a serial number, and the probing system is of the kind ProbingType.SOFTWARE. Set what else is known, such as the creator and a comment, with with_metadata().

Parameters:

name – Name of the software.
version – Version of the software.
date – Date and time of creation; the current UTC time by default.

>>> metadata = Metadata.for_software("analyser", "1.2")
>>> metadata.instrument.model, metadata.probing_system.type.name
('analyser', 'SOFTWARE')

class x3pio.Instrument(manufacturer: str = '', model: str = '', serial: str = '', version: str = '')¶

The instrument or software that created the data (Record2/Instrument).

All elements are required by the standard; serial may be empty for software-created data.

class x3pio.ProbingSystem(type: ProbingType | None = ProbingType.SOFTWARE, identification: str = '')¶

The probing system used (Record2/ProbingSystem).

Parameters:

type – Kind of probing system; ProbingType.SOFTWARE for synthetic or filtered data and software gauges. None if a file read held no recognizable value, which must be fixed before writing.
identification – Description as specific as possible, e.g. the stylus or objective used.

class x3pio.ProbingType(*values)¶: Kind of probing system, as written in the ProbingSystem/Type element.

Data Types and Storage¶

Storage data types of the axes and their conversion to metres.

A file stores the numbers of an absolute axis as one of four types: a 16-bit or a 32-bit integer, or a 32-bit or a 64-bit floating point number, always with the least significant byte first. The stored number times the increment of the axis is the coordinate in metres, so an integer axis chooses its increment to fit the range of the data into the range of the type, and a floating point axis stores metres with the increment 1 by default.

A point that is not measured is NaN in a floating point type. An integer has no value for it, so a file with integer axes lists the valid points in a separate file of bits. The x and y of a missing point are not kept then and read as NaN; floating point axes keep them.

class x3pio.DataType(*values)¶

Storage type of the coordinates of one axis, as written in the DataType element.

Members are named after the stored representation; their string value is the exact letter written into main.xml.

x3pio.suggest_scale(values: ndarray, data_type: DataType) → float¶

Pick the increment that stores values in data_type at the highest resolution.

A floating point type stores metres directly, so the increment is 1. An integer type is given the smallest increment for which the largest magnitude still fits without overflow.

Parameters:

values – Coordinates in metres; non-finite values are ignored.
data_type – Target storage type.

>>> suggest_scale(np.array([0.0, 1.0]), DataType.FLOAT64)
1.0
>>> suggest_scale(np.array([0.0, 32767e-6]), DataType.INT16)
1e-06

class x3pio.DataStorage(*values)¶

How an x3p file stores its coordinates.

The standard recommends the binary form for data sets of more than 10000 points, and allows it for smaller ones too.

Vendor Extensions¶

Vendor specific extensions of an x3p file, registered by VendorSpecificID.

A vendor can put files of its own into the zip archive of an x3p file, such as the settings of the instrument or notes on the measurement. A VendorSpecificID in main.xml is a URI that the vendor owns and that identifies such an extension, so that two vendors do not take the same name. Software that does not know an ID ignores it and its files, and a file that no ID registers is not an extension.

The two revisions of the standard differ in what an ID stands for. In ISO 25178-72:2017 there is a single ID for the extension of a vendor as a whole, and its files lie in the container under any name. Its Amd 1:2020 gives every file an ID of its own and derives the name of the file in the container from the ID. VendorExtensions holds both, as one mapping from an ID to the content of its file; see Revision for what differs.

class x3pio.VendorExtensions(revision: Revision = Revision.ISO25178_72_2017_DAM1)¶

The vendor specific extension files of an x3p file, by their VendorSpecificID.

The standard registers vendor files by a VendorSpecificID, a URI that the vendor owns: a domain name, an optional path and the name of the file. This is a mutable mapping from the IDs, exactly as written in the file, with the content of their files as values. The value is None for an ID that has no file, such as an ID that is only a domain name. The key is never changed, so give the ID as the software that reads the file expects to find it.

Files are added with add(), which takes the vendor and the path of the file below it and makes the ID from them, or with extensions[uri] = content for an ID that is known in full.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((1, 2)), x_scale=1.0, y_scale=1.0)
>>> uri = "http://www.vendor.com/mypath/a.xml"
>>> file.extensions[uri] = b"<a/>"
>>> file.extensions.add("http://www.vendor.com", "b.xml", b"<b/>")
>>> list(file.extensions)
['http://www.vendor.com/mypath/a.xml', 'http://www.vendor.com/b.xml']
>>> file.extensions[uri]
b'<a/>'
>>> list(file.extensions.below("http://www.vendor.com/mypath"))
['http://www.vendor.com/mypath/a.xml']
>>> del file.extensions[uri]
>>> len(file.extensions)
1

The two revisions hold them differently. In Amd 1:2020 every file has an ID of its own, and any number of vendors can add files. ISO 25178-72:2017 has a single VendorSpecificID, which identifies the extension of the vendor as a whole, and its files lie in the container under any name. That ID is the vendor of the first file that is added, a file of another vendor is refused, and the key of a file is the vendor, a / and its path.

Different URIs can name the same file in the container, such as http://a.b/c and https://a.b/c, because the name leaves out the scheme. They share its content. Adding a URI for a file that another ID names, with other content, raises an error.

The files of a file that was read stay compressed until their content is requested, so listing the IDs unpacks nothing. A damaged file raises X3pFormatError when its content is requested, not when the x3p file is read.

property vendor_ids: list[str]¶

The VendorSpecificID elements that a file written with these extensions has.

In Amd 1:2020 there is one for every file, the same as the keys of the mapping. In ISO 25178-72:2017 there is at most one, which identifies the extension of the vendor as a whole.

>>> import x3pio
>>> file = x3pio.Surface.from_array(
...     [[0.0]], x_scale=1.0, y_scale=1.0, revision=x3pio.Revision.ISO5436_2000
... )
>>> file.extensions.vendor_ids
[]
>>> file.extensions.add("http://www.vendor.com", "a.xml", b"<a/>")
>>> file.extensions.vendor_ids
['http://www.vendor.com']

extract(uri: str, destination: str | PathLike[str] | IO[bytes]) → None¶

Write the content of a file to a file or a binary stream.

Parameters:

uri – The ID of the file.
destination – The path of the file to write, or an open binary stream.

Raises:

KeyError – If uri is not registered.
X3pFormatError – If uri is registered without a file, or the file is damaged.
OSError – If the file cannot be written.

below(prefix: str) → VendorExtensions¶

Return a live view of the IDs that lie below a domain name or a path.

The view has the same keys. Reading, adding and removing through it changes this mapping, and a new ID has to lie below the prefix. prefix is matched against whole segments: the host name, which is everything up to the first /, and then the segments of the path. The scheme is ignored, so http and https IDs of one vendor are both found.

Parameters:: prefix – A URI of a vendor, such as "http://www.vendor.com", or of a directory of it, such as "http://www.vendor.com/mypath".
Raises:: X3pFormatError – If prefix is not such a URI.

>>> import numpy as np
>>> import x3pio
>>> file = x3pio.Surface.from_array(np.zeros((1, 2)), x_scale=1.0, y_scale=1.0)
>>> file.extensions["http://www.vendor.com/mypath/a.xml"] = b"<a/>"
>>> file.extensions["https://www.vendor.com/b.xml"] = b"<b/>"
>>> file.extensions["http://example.org/c.xml"] = b"<c/>"
>>> len(file.extensions.below("www.vendor.com"))
2
>>> file.extensions.below("www.vendor.com").clear()
>>> list(file.extensions)
['http://example.org/c.xml']

Add a file of a vendor, or replace the content of the one at path.

In Amd 1:2020 the ID of the file is vendor, a / and path. In the standard before it vendor is the single VendorSpecificID of the file and path is the path of the file in the container. The first file that is added fixes the vendor, and a file of another vendor is refused; the key of the file is again vendor, a / and path. A / at the end of vendor does not matter.

Parameters:

vendor – The vendor, a domain name or a URI that the vendor owns, usually with a scheme such as http://. It is written to the file exactly as given.
path – The path of the file below the vendor, such as "mypath/a.xml".
content – The content of the file: its bytes, the path of a file to read (a str is always a path, never data), or an open binary stream, which is read from its current position and left open. The content is read immediately.

Raises:

X3pFormatError – If vendor or path cannot name a file, the file would take the name of a file of the format, is both a file and a directory, or names the same file as another ID with other content, the vendor is another than the one of the file, or the ID does not lie below what this view is limited to.
OSError – If a file cannot be read.

clear() → None¶

Remove every ID of this mapping, or of this view, and the files that no ID names.

Clearing the whole mapping also removes the vendor of ISO 25178-72:2017.

Files With Deviations¶

Serialization of the model: zip container, main.xml, binary files and checksums.

This package turns bytes into the types of x3pio.model and back. It depends on the model, never the other way round.

When reading, x3pio tolerates the deviations from the standard that are listed here and corrects them silently. Other deviations raise X3pFormatError. Elements that the schema does not define are ignored wherever they occur.

Reading does not validate main.xml against the XSD schema of the standard: the elements are looked up by name and checked as far as the model needs. Writing always follows the standard. That is ensured by tests that validate the written main.xml against the schemas of ISO 25178-72:2017 and Amd 1:2020.

Checksums. The MD5 checksums of main.xml, of the binary data file and of the validity file are verified. A mismatch raises X3pChecksumError: the file was changed or damaged after it was written. read(path, verify=False) and X3pFile.loads skip the check. A checksum that is missing or malformed, a checksum file that is missing, outside the container, with another name or with only the digest, in either case and with or without the file name that md5sum writes, is accepted.

Axes:

A missing Increment is read as 1, and a missing DataType of an incremental axis as float64. ISO 25178-72:2017 allows both, Amd 1:2020 requires them, and some files and instruments omit the increment of the z axis. An empty Increment is read as 1 and an empty Offset as 0.
A negative Increment is accepted, a zero one is an error.
A rotation element slightly outside of [-1, 1], by up to 1e-6, is clamped.

Document structure:

A missing or empty Revision is read as ISO 25178-72:2017, and the en dash and other dashes in the text of a revision are read as the hyphen, see Revision.
A root element outside of the namespace of the schema is accepted.
A missing Record4, Instrument or ProbingSystem, or a missing element of the instrument, is empty.
A FeatureType that does not agree with the data is corrected: a list is a point cloud, a matrix is a surface, and a profile has a single row.
Placeholder text instead of a date or a probing type (N/A, Date of Calibration, Type) is dropped, leaving None. The probing type is matched without regard to case. Metadata without a date or probing type cannot be written until they are set.

Numbers:

A Datum outside of the number pattern of the schema, such as 2.e-06, is parsed anyway.
A floating point value of inf, -inf or NaN in a binary file is a point that is not measured, although the standard only mentions NaN.

Container:

A UTF-8 byte order mark before the XML declaration is ignored.
Files packed by hand, for example by compressing the folder of an extracted file in the file manager, have all their content in a single folder instead of the root of the zip. This is accepted. Such a container may also hold what the operating system adds: a __MACOSX/ folder at any depth, ._name companions and .DS_Store of macOS, and Thumbs.db and desktop.ini of Windows, in any case. These are dropped; other names are not touched.
Files that no VendorSpecificID registers are ignored. The VendorSpecificID of any file is kept as written, whether it names a file or not.

Some problems cannot be corrected and always raise X3pFormatError: an incremental z axis, a zero increment, an unknown revision, feature, axis or data type, data that does not fit the dimensions, a point cloud with an incremental axis, a missing main.xml, Record1 or Record3, links that leave the container (absolute paths, .., URLs), and anything that is not XML.

Command Line¶

Command line interface.

x3pio info FILE prints the header, the metadata and the sizes of a file, without reading its data.

x3pio convert SOURCE DESTINATION changes the storage form (-s, xml or binary), the z data type (-t) and the revision (-r) independently. Whatever is not given is kept from the source file, and deviations of the source are repaired. -f/--fix drops what cannot be written instead of failing: metadata without a date or a probing type, and the vendor extensions when converting to ISO5436_2000.

The command line is also runnable as python -m x3pio.

Exceptions¶

exception x3pio.X3pError¶: Base class for all x3pio errors.

exception x3pio.X3pFormatError¶

Raised when file content or a value given for a file is not valid in x3p.

It is a ValueError, like json.JSONDecodeError, so code that catches that also catches a bad value passed to x3pio.

exception x3pio.X3pChecksumError¶: Raised when an MD5 checksum stored in an x3p file does not match its content.