API Reference

sdfio: read and write ISO 25178-71 SDF surface data files.

The Surface Data File (SDF) format is defined by ISO 25178-71, “Geometrical product specifications (GPS) – Surface texture: Areal – Part 71: Surface data file (SDF) file format”. It stores areal (or profile) surface topography measurements as a rectangular grid of height values, in either a human-readable ASCII or a compact binary format.

Basic usage:

>>> import os
>>> import tempfile
>>> import numpy as np
>>> import sdfio
>>> path = os.path.join(tempfile.mkdtemp(), "example.sdf")
>>> sdfio.write(path, np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6)
>>> sdf = sdfio.read(path)
>>> sdf.data.shape
(2, 3)
>>> sdf.header.z_scale
1.0

The examples below write to memory, so that they leave no files behind.

A surface is a (rows, columns) array of heights in metres. y grows with the row and x with the column. Points that are not measured are NaN:

>>> import io
>>> heights = np.array([[0.0, 1e-6, 2e-6], [3e-6, np.nan, 5e-6]])
>>> buffer = io.BytesIO()
>>> sdfio.write(buffer, heights, x_scale=2e-6, y_scale=3e-6)
>>> surface = sdfio.read(io.BytesIO(buffer.getvalue()))
>>> surface.data.shape, bool(np.isnan(surface.data[1, 1]))
((2, 3), True)
>>> surface.x_axis.tolist(), surface.y_axis.tolist()
([0.0, 2e-06, 4e-06], [0.0, 3e-06])

A profile is a surface with one row:

>>> profile = np.sin(np.linspace(0.0, 6.0, 100)).reshape(1, -1) * 1e-6
>>> buffer = io.BytesIO()
>>> sdfio.write(buffer, profile, x_scale=1e-6, y_scale=1e-6)
>>> sdfio.read(io.BytesIO(buffer.getvalue())).data.shape
(1, 100)

The file is described by the header. The revision, the data type, the manufacturer, the dates and the quantization step come from a template, and the representation from format. The heights are stored as integers of data_type times z_scale, which is by default the smallest scale that fits the data:

>>> metadata = sdfio.Metadata(
...     revision=sdfio.Revision.ISO_2_0,
...     manufacturer_id="acme",
...     data_type=sdfio.DataType.INT16,
... )
>>> buffer = io.BytesIO()
>>> sdfio.write(
...     buffer, heights, x_scale=2e-6, y_scale=3e-6,
...     format=sdfio.FileFormat.ASCII, metadata=metadata,
... )
>>> text = buffer.getvalue().decode("ascii")
>>> text.splitlines()[0], "DataType = 5" in text, "BAD" in text
('aISO-2.0', True, True)

The trailer holds free text, or “Name = Value” fields, which ISO-2.0 requires:

>>> sdf = sdfio.read(io.BytesIO(buffer.getvalue()))
>>> sdf.trailer_fields = {"OperatorName": "Jane Doe"}
>>> sdfio.read(io.BytesIO(sdf.dumps())).trailer_fields["OperatorName"]
'Jane Doe'

A file is converted to another revision, data type or representation with SdfFile.with_revision() and SdfFile.dumps():

>>> converted = sdf.with_revision(sdfio.Revision.ISO_1_0, trailer="")
>>> str(converted.header.revision)
'ISO-1.0'
>>> binary = sdfio.read(io.BytesIO(converted.dumps(format=sdfio.FileFormat.BINARY)))
>>> binary.header.format.name
'BINARY'

SdfFile.points() gives the points of the grid with their x and y coordinates, as the (rows, columns, 3) array that plotting libraries take. With drop_invalid it is the (N, 3) list of the measured points that point cloud libraries take:

>>> points = surface.points()
>>> points.shape
(2, 3, 3)
>>> points[1, 2].tolist()
[4e-06, 3e-06, 5e-06]
>>> surface.points(drop_invalid=True).shape
(5, 3)
class sdfio.DataType(*values)

Bases: IntEnum

Storage types of the data area, by their DataType code in the header.

Codes 0 to 2 are the unsigned integer types of BCR-1.0, which ISO dropped. Which revision has which type is RevisionRules.data_types.

class sdfio.FileFormat(*values)

Bases: Enum

ASCII or binary SDF file representation.

class sdfio.Header(revision: Revision = Revision.ISO_2_0, format: FileFormat = FileFormat.BINARY, manufacturer_id: str = 'sdfio', create_date: datetime | None = <factory>, mod_date: datetime | None = <factory>, x_scale: float = 1.0, y_scale: float = 1.0, z_scale: float = 1.0, z_resolution: float = -1.0, data_type: DataType = DataType.BINARY64)

Bases: object

SDF record 1 header fields.

Compression and CheckType are not modelled as fields here: neither compression nor a checksummed data area is supported, for any revision, on read or write (rejected outright, always written as 0). NumPoints and NumProfiles are not fields either: they are the shape of the data (SdfFile.data), which is written with them.

A header is checked when it is made and when a field is set: the revision, the file format, the data type, which the revision must have, a positive z_scale and a 7-bit ASCII manufacturer_id. To change the revision together with a data type that the old revision does not have, make a new header, see SdfFile.with_revision(). A header that a reader returns is not, because the readers accept files with such values, so that they can be inspected and fixed; the write rejects them.

Parameters:

  • revision – Revision of the SDF format (see Revision). Accepts a raw "ISO-2.0"-shaped string as a convenience, which is coerced to Revision.

  • format – Whether the file uses the binary or the ASCII representation, see FileFormat.

  • manufacturer_id – Measurement instrument manufacturer’s identifier; may include the source of the data, hardware and software identifiers.

  • create_date – Original creation date and time, or None if not recorded in the file. It is always held as a timezone-aware UTC datetime with the resolution of a minute, as the field has, whatever the revision: any timezone-aware value is converted when it is set, and a naive one is taken as local time. A revision for which RevisionRules.requires_utc is False writes it as local time.

  • mod_date – Last modification date and time, or None if not recorded in the file; held like create_date.

  • x_scale – Sampling interval in metres along x.

  • y_scale – Sampling interval in metres along y.

  • z_scale – Scale factor applied to the coded height values.

  • z_resolution – The measurement instrument’s original quantization step in metres, or a negative value if unknown. Distinct from z_scale: data can be rescaled or reprocessed (e.g. datum removal) after acquisition, changing z_scale, while z_resolution stays as a record of the instrument’s original base resolution.

  • data_type – DataType code, see DataType.

Raises:
property magic: str

8-character file magic (family and version) written at the start of the file.

class sdfio.Metadata(revision: Revision = Revision.ISO_2_0, manufacturer_id: str = 'sdfio', create_date: datetime | Literal['now'] | None = 'now', mod_date: datetime | Literal['now'] | None = 'now', z_resolution: float = -1.0, data_type: DataType = DataType.BINARY64)

Bases: object

Reusable sdfio.write() header template.

Holds exactly the Header fields that are meaningful to reuse across many sdfio.write() calls – as opposed to x_scale/ y_scale/z_scale/format, which differ on every call and are write()’s own parameters. Every field here is used as given, with no overriding.

Parameters:

  • revision – Revision of the SDF format (see Revision).

  • manufacturer_id – Measurement instrument manufacturer’s identifier; may include the source of the data, hardware and software identifiers.

  • create_date – Creation timestamp, or "now" (the default) for the time the file is built, or None for not recorded. Held like Header.create_date: as UTC to the minute.

  • mod_date – Modification timestamp, held and defaulting like create_date.

  • z_resolution – Quantization step in metres, or a negative value if unknown.

  • data_type – DataType code, see DataType; must be valid for revision.

class sdfio.Revision(*values)

Bases: StrEnum

A revision of the SDF format, as written in the magic of a file, such as aISO-2.0.

BCR is the proposal from before the standard, which ISO succeeded; it has one version. The value of a member is the text of the magic.

classmethod resolve(family_text: str, version_text: str, magic: str) → Revision

Return the revision of the family and version text of a magic.

Parameters:

magic – The whole magic, for the error message.

Raises:

SdfFormatError – If the two do not name a revision.

property rules: RevisionRules

How this revision differs from the others, see RevisionRules.

class sdfio.RevisionRules(requires_utc: bool, max_binary_count: int, requires_tagged_trailer: bool, sentinel: Sentinel, data_types: tuple[DataType, ...], ascii_files: AsciiRules, reverses_profile_order: bool)

Bases: object

What a revision of the SDF format decides, one instance per Revision.

Parameters:

  • requires_utc – Whether the file stores create_date/mod_date as UTC. The standard mandates UTC from version 2.0; the other revisions store local time. In memory the dates are UTC for every revision, see Header.

  • max_binary_count – Largest number of points per profile, and of profiles, of the binary encoding: 65535 where the binary header holds both in 2 bytes, and 4294967295 for 4 bytes (ISO 25178-71, Table 2, where the data types of these fields are listed for the binary format only). The standard sets no limit for the ASCII encoding.

  • requires_tagged_trailer – Whether a non-empty trailer written to disk must use the tagged Name = Value format of the header. Enforced on write only: a file with a non-conforming trailer can still be read, see validate_trailer_tagged().

  • sentinel – How a point that was not measured is marked in the raw values: ISO reserves the minimum of a data type, and the 1993 BCR proposal the maximum, since it still had unsigned integer types whose minimum (0) is a valid low bound.

  • data_types – The data types the revision has, by their codes.

  • ascii_files – What the ASCII encoding allows, see AsciiRules.

  • reverses_profile_order – Whether the first profile stored on disk is at the maximum y, not at y=0. BCR never fixed a direction; ISO 25178-71 fixed row 1 at y=0. This reflects observed behaviour, not a requirement: MountainsMap renders BCR-1.0 with the first profile at the maximum y, and sdfio matches that on read and write, so SdfFile.y_axis increases with the row for every revision.

>>> [data_type.name for data_type in Revision.ISO_1_0.rules.data_types]
['INT16', 'INT32', 'BINARY64']
>>> Revision.ISO_1_0.rules.max_binary_count
65535
>>> Revision.ISO_2_0.rules.max_binary_count
4294967295
order_rows(data: NDArray[float64]) → NDArray[float64]

Return data with the rows in the order of the file, or back in increasing y.

The same operation converts in both directions: a reversal for a revision for which reverses_profile_order is True, and data as it is otherwise.

exception sdfio.SdfError

Bases: Exception

Base class for all sdfio errors.

class sdfio.SdfFile(header: Header, data: NDArray[float64], trailer: str | bytes = '')

Bases: object

An SDF surface data file in memory.

Parameters:

  • header – Record 1.

  • data – (num_profiles, num_points) array of height values in metres, with NaN for a point that was not measured. It is held as a 2-D float64 array whenever it is set, and its shape is the number of profiles and points of the file. y increases with the row, see y_axis.

  • trailer – Record 3: text from an ASCII file, bytes from a binary file.

property x_axis: NDArray[float64]

Return the x coordinates of the columns in metres: x_j = (j - 1) * x_scale.

There is one per point of a row. One sentence of the 2026 edition of the standard pairs the numbers the other way round, against its formula and figure; a later edition fixes it.

property y_axis: NDArray[float64]

Return the y coordinates of the rows in metres: y_i = (i - 1) * y_scale.

y increases with the row, in the right-handed coordinate system of the standard, for every revision, see RevisionRules.reverses_profile_order. For a profile the standard calls y_scale meaningless, and the result is [0.0].

points(*, drop_invalid: bool = False) → NDArray[float64]

Return the points of the grid as an (num_profiles, num_points, 3) array in metres.

The last axis holds x, y and z: x from x_axis along the columns, y from y_axis along the rows, and z from data. The array is new, so changing it does not change the file. A point that is not measured has NaN in z and keeps its x and y. A profile has one row, with y equal to 0.

reshape(-1, 3) gives the (N, 3) array of points that point cloud libraries take.

Parameters:

drop_invalid – Leave out the points that are not measured. The result is then the (N, 3) array of the measured points, row by row, and the shape of the grid is lost.

>>> import numpy as np
>>> from sdfio import SdfFile, Header
>>> header = Header(x_scale=1e-6, y_scale=2e-6)
>>> sdf = SdfFile(header=header, data=np.array([[0.0, 1.0, 2.0], [3.0, np.nan, 5.0]]))
>>> points = sdf.points()
>>> points.shape
(2, 3, 3)
>>> points[1, 2].tolist()
[2e-06, 2e-06, 5.0]
>>> sdf.points(drop_invalid=True).shape
(5, 3)
property is_profile: bool

Whether the file holds a profile and not an areal surface.

The standard defines a profile as a file with one row (“in case of a profile, the value of ‘Number of rows in the data matrix’ is 1”) and calls y_scale meaningless for it. A surface with a single row is therefore a profile.

>>> import numpy as np
>>> from sdfio import SdfFile, Header
>>> profile = SdfFile(header=Header(), data=np.zeros((1, 3)))
>>> surface = SdfFile(header=Header(), data=np.zeros((2, 3)))
>>> profile.is_profile, surface.is_profile
(True, False)
with_revision(revision: Revision, *, data_type: DataType | int | None = None, trailer: str | bytes | None = None, refit_z_scale: bool = False) → SdfFile

Return a copy of this file retargeted to another revision.

What the new revision does not take is not guessed; each is a parameter:

  • data_type: some data types exist in some revisions only, such as binary32 and int8 in ISO-2.0 and BCR-1.0. Pass one the new revision has.

  • trailer: ISO-2.0 requires a tagged “Name = Value” trailer, or none. Pass a replacement, such as "", if the current one doesn’t fit.

  • refit_z_scale: the data may not fit the new data type at the current z_scale, which the write would report. With True the z_scale is kept if the data fits, and otherwise replaced by the smallest that does, see sdfio.suggest_z_scale(). The data is not changed.

The dates need no parameter. They are UTC in every revision and converted to the convention of the revision when the file is written. The header is new; the data is shared with this file, not copied.

Raises:

SdfFormatError – If data_type or the trailer doesn’t fit revision.

Converting to a different revision:

>>> import io
>>> import numpy as np
>>> import sdfio
>>> buf = io.BytesIO()
>>> sdfio.write(buf, np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6,
...              metadata=sdfio.Metadata(revision=sdfio.Revision.ISO_1_0))
>>> sdf = sdfio.read(io.BytesIO(buf.getvalue()))
>>> converted = sdf.with_revision(sdfio.Revision.ISO_2_0)
>>> str(converted.header.revision)
'ISO-2.0'

To change only the data type, pass the current revision:

>>> retyped = sdf.with_revision(sdf.header.revision, data_type=sdfio.DataType.INT16)
>>> retyped.header.data_type.name
'INT16'

refit_z_scale takes a scale that fits:

>>> tall = sdfio.SdfFile.from_array(
...     np.array([[0.0, 0.5]]), x_scale=1e-6, y_scale=1e-6, z_scale=1e-9
... )
>>> tall.with_revision(
...     sdfio.Revision.ISO_2_0, data_type=sdfio.DataType.INT16, refit_z_scale=True
... ).header.z_scale > 1e-9
True
with_data(data: ArrayLike, *, x_scale: float | None = None, y_scale: float | None = None) → SdfFile

Return a copy with other heights, for example a part of this file.

The rest of the header and the trailer are kept. Like from_array(), this only checks the shape of data; whether it fits the data type and the z_scale is checked when the file is written. float64 data is shared, not copied, so a slice of this file’s data is a view of it: changing the part changes this file. Copy it first with .copy() to avoid that.

Parameters:

  • data – (num_profiles, num_points) array of height values in metres.

  • x_scale – New sampling interval along x, in metres, if it is not the one of the file.

  • y_scale – New sampling interval along y, in metres, if it is not the one of the file.

Raises:

SdfFormatError – If data is not 2-D.

>>> import numpy as np
>>> sdf = SdfFile.from_array(np.zeros((4, 6)), x_scale=1e-6, y_scale=1e-6)
>>> part = sdf.with_data(sdf.data[1:3, ::2], x_scale=2e-6)
>>> part.data.shape, part.header.x_scale
((2, 3), 2e-06)
raw_data() → NDArray[Any]

Return the coded heights in the on-disk dtype of the data type.

That is data / z_scale, rounded, with the marker of the revision instead of NaN for a point that was not measured. It is computed on each call, so it follows changes of data and header.

Raises:

SdfFormatError – If a value is out of range for header.data_type, or collides with its invalid-point sentinel.

property trailer_fields: Mapping[str, str]

Return the trailer as “Name = Value” fields, the tagged format of the header.

ISO-2.0 requires this form on write. A line of another form is left out, see sdfio.trailer.sanitize_tagged_fields(); use sdfio.trailer.parse_tagged_fields() to detect it instead. The result is a read-only snapshot. Assign the whole property to change the trailer: sdf.trailer_fields = {**sdf.trailer_fields, "X": "Y"}.

>>> import numpy as np
>>> from sdfio import SdfFile, Header
>>> sdf = SdfFile(header=Header(), data=np.zeros((1, 1)))
>>> sdf.trailer_fields = {"OperatorName": "Jane Doe"}
>>> sdf.trailer_fields
mappingproxy({'OperatorName': 'Jane Doe'})
classmethod from_array(data: ArrayLike, *, x_scale: float, y_scale: float, z_scale: float | Literal['auto'] = 'auto', format: FileFormat = FileFormat.BINARY, metadata: Metadata | None = None, trailer: str | bytes = '') → SdfFile

Build an in-memory SDF file from a NumPy array, without writing it.

The arguments are those of sdfio.write(), which builds the file this way and saves it. Only the shape of data is checked here, and metadata has checked its data type against its revision. Whether data fits the data type, the manufacturer ID and the trailer are checked when the file is written, see save().

Parameters:

  • data – (num_profiles, num_points) array of height values in metres. NaN marks points that were not measured. A float64 array is used as it is, not copied.

  • x_scale – Sampling interval along x, in metres.

  • y_scale – Sampling interval along y, in metres.

  • z_scale – Scale factor of the coded heights, or "auto" (the default) for the smallest that fits data into metadata.data_type, which is 1 for a float type, see sdfio.suggest_z_scale().

  • format – The encoding, which becomes Header.format.

  • metadata – Header template, shared by files that have the same metadata, such as Metadata(manufacturer_id="acme"). The defaults are ISO-2.0, binary64 and the current time. A binary file holds 10 characters of the manufacturer ID.

  • trailer – Record 3, the trailer.

Raises:

SdfFormatError – If data is not 2-D.

>>> import numpy as np
>>> sdf = SdfFile.from_array(np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6)
>>> sdf.data.shape, sdf.header.revision.value, sdf.header.format.name
((2, 3), 'ISO-2.0', 'BINARY')
classmethod loads(data: bytes) → SdfFile

Parse an in-memory SDF blob, detecting the ASCII/binary format from its magic.

Parameters:

data – Full contents of a .sdf file, in either format. A UTF-8 byte order mark in front of the magic, which some tools write, is ignored.

Raises:

SdfFormatError – If data does not start with a recognized SDF magic, or its contents are otherwise malformed.

dumps(*, format: FileFormat | None = None) → bytes

Serialize this file to an in-memory blob, overriding the ASCII/binary format if given.

Parameters:

format – Defaults to the format the data was read with (or header.format if constructed directly).

Returns:

The serialized .sdf file contents, always as bytes (the ASCII format is 7-bit ASCII text, so it is returned encoded rather than as str).

Raises:
save(path: str | PathLike[str] | IO[bytes], *, format: FileFormat | None = None) → None

Write this file to disk or a binary stream, optionally overriding the format.

Parameters:

  • path – Destination path, or an open binary file object.

  • format – Defaults to the format the data was read with (or header.format if constructed directly).

Raises:

SdfFormatError – See dumps().

exception sdfio.SdfFormatError

Bases: SdfError, ValueError

Raised when file content or a value given for a file is not valid in the SDF format.

It is a ValueError, like json.JSONDecodeError, so code that catches that also catches a bad value passed to sdfio.

exception sdfio.SdfVersionError

Bases: SdfFormatError

Raised when an SDF revision string is not recognized.

sdfio.format_tagged_fields(fields: Mapping[str, str]) → str

Format a {name: value} mapping as <CRLF>-terminated “Name = Value” lines.

sdfio.parse_tagged_fields(text: str) → dict[str, str]

Parse “Name = Value” lines into a {name: value} dict.

That is the form of the header, and the form that ISO-2.0 requires of the trailer on write. Blank lines are skipped. Names are kept as written; compare them case-insensitively if needed.

Raises:

SdfFormatError – If a line is not “Name = Value”.

sdfio.read(path: str | PathLike[str] | IO[bytes]) → SdfFile

Read an SDF surface data file into memory.

Parameters:

path – Path to a .sdf file, or an open binary file object, in either ASCII or binary format.

Returns:

The parsed header, data matrix (in metres, NaN for non-measured or spurious points) and raw trailer.

Raises:

SdfFormatError – If the file does not start with a recognized SDF magic, or its contents are otherwise malformed.

sdfio.sanitize_tagged_fields(text: str) → str

Reformat text, keeping the lines that are “Name = Value” and leaving out the rest.

Unlike parse_tagged_fields() this never raises, which salvages a trailer that is mostly tagged, such as when converting a file to ISO-2.0.

sdfio.suggest_z_scale(data: NDArray[float64], data_type: DataType, revision: Revision = Revision.ISO_2_0) → float

Suggest the smallest z_scale that encodes data without overflow.

For integer data types, z_scale directly trades off range against resolution: this picks the smallest value (i.e. highest resolution) such that data / z_scale stays within the type’s representable range and avoids its invalid-point sentinel. Floating point data types already cover an enormous dynamic range at fixed relative precision, so 1.0 (no rescaling) is suggested for those.

Parameters:

  • data – Height values in metres; NaN marks non-measured or spurious points.

  • data_type – Target SDF data area storage type.

  • revision – SDF revision, see DataTypeInfo.invalid_value().

Returns:

A positive z_scale value, or 1.0 if data has no finite values or is not an integer data type.

sdfio.validate_trailer_tagged(revision: Revision, trailer: str | bytes) → None

Reject a trailer that is not tagged, for a revision that requires it.

Only a write is checked; a file with another trailer is read. An empty trailer is always accepted.

Raises:

SdfFormatError – If RevisionRules.requires_tagged_trailer is True and the trailer is not “Name = Value” lines, see parse_tagged_fields().

sdfio.write(path: str | PathLike[str] | IO[bytes], data: ArrayLike, *, x_scale: float, y_scale: float, z_scale: float | Literal['auto'] = 'auto', format: FileFormat = FileFormat.BINARY, metadata: Metadata | None = None, trailer: str | bytes = '') → None

Write a NumPy array to an SDF surface data file.

The file is built by SdfFile.from_array(), which takes the same arguments and returns it without writing, and saved with SdfFile.save().

Parameters:

  • path – Destination path, or an open binary file object.

  • data – (num_profiles, num_points) array of height values in metres. NaN marks points that were not measured.

  • x_scale – Sampling interval along x, in metres.

  • y_scale – Sampling interval along y, in metres.

  • z_scale – Scale factor of the coded heights, or "auto" (the default) for the smallest that fits data into metadata.data_type, which is 1 for a float type, see sdfio.suggest_z_scale().

  • format – The encoding, ASCII or binary.

  • metadata – Header template, shared by files that have the same metadata, such as Metadata(manufacturer_id="acme"). The defaults are ISO-2.0, binary64 and the current time. A binary file holds 10 characters of the manufacturer ID.

  • trailer – Record 3, the trailer, as 7-bit ASCII.

Raises:

SdfFormatError – If data is not 2-D, z_scale is not positive, the manufacturer ID or the trailer is not 7-bit ASCII, a trailer is not tagged “Name = Value” where the revision requires it, or data does not fit metadata.data_type.

path also accepts an open binary stream, e.g. to avoid touching disk:

>>> import io
>>> import numpy as np
>>> import sdfio
>>> buf = io.BytesIO()
>>> sdfio.write(buf, np.zeros((2, 3)), x_scale=1e-6, y_scale=1e-6)
>>> sdf = sdfio.read(io.BytesIO(buf.getvalue()))
>>> sdf.data.shape
(2, 3)