Read SDMX file formats

SDMX-ML

sdmx.reader.xml supports the several types of SDMX-ML messages.

Implementation details:

• The collections of StructureMessage (e.g. StructureMessage.codelist) are implemented by sdmx as DictLike, with str keys, for convenience; the standard would imply these could be other collections, such as a simple list. The format of the keys in each collection depends on the content of the message parsed by reader.xml:

  • Simply {object.id} (IdentifiableArtefact.id) of the contained objects, if these are unique;

  • Otherwise {maintainer.id}:{object.id} (using the Agency id) if these are unique;

  • Otherwise {maintainer.id}:{object.id}({object.version}) (using the VersionableArtefact.version).

  This ensures that all objects in a parsed message are accessible.

exception sdmx.reader.xml.XMLParseError

xml.Reader is unable to parse a message.

SDMX-ML v2.1 reader.

class sdmx.reader.xml.v21.DispatchingReader(name, bases, dct)

Populate the parser, format, and model attributes of Reader.

exception sdmx.reader.xml.v21.NotReference

Raised when the elem passed to Reference is not a reference.

class sdmx.reader.xml.v21.Reader

SDMX-ML 2.1 reader.

class Reference(reader, elem, cls_hint=None)

Temporary class for references.

• cls, id, version, and agency_id are always for a MaintainableArtefact.

• If the reference target is a MaintainableArtefact (maintainable is True), target_cls and target_id are identical to cls and id, respectively.

• If the target is not maintainable, target_cls and target_id describe it.

cls_hint is an optional hint for when the object is instantiated, i.e. a more specific override for cls/target_cls.

annotable(cls, elem, **kwargs)

Create a AnnotableArtefact of cls from elem and kwargs.

Collects all parsed <com:Annotation>.

classmethod end(names: str, only: bool = True)

Decorator for a function that parses “end” events for XML elements.

format: ClassVar = <module 'sdmx.format.xml.v21' from '/home/docs/checkouts/readthedocs.org/user_builds/sdmx1/envs/latest/lib/python3.12/site-packages/sdmx/format/xml/v21.py'>

get_single(cls_or_name: Type | str, id: str | None = None, version: str | None = None, subclass: bool = False) → Any | None

Return a reference to an object while leaving it in its stack.

Always returns 1 object. Returns None if no matching object exists, or if 2 or more objects meet the conditions.

If id (and version) is/are given, only return an IdentifiableArtefact with the matching ID (and version).

If cls_or_name is a class and subclass is True; check all objects in the stack cls_or_name or any stack for a subclass of this class.

identifiable(cls, elem, **kwargs)

Create a IdentifiableArtefact of cls from elem and kwargs.

maintainable(cls, elem, **kwargs)

Create or retrieve a MaintainableArtefact of cls from elem and kwargs.

Following the SDMX-IM class hierarchy, maintainable() calls nameable(), which in turn calls identifiable(), etc. (Since no concrete class is versionable but not maintainable, no separate method is created, for better performance). For all of these methods:

• Already-parsed items are removed from the stack only if elem is not None.

• kwargs (e.g. ‘id’) take precedence over any values retrieved from attributes of elem.

If elem is None, maintainable() returns a MaintainableArtefact with the is_external_reference attribute set to True. Subsequent calls with the same object ID will return references to the same object.

model: ClassVar = <module 'sdmx.model.v21' from '/home/docs/checkouts/readthedocs.org/user_builds/sdmx1/envs/latest/lib/python3.12/site-packages/sdmx/model/v21.py'>

nameable(cls, elem, **kwargs)

Create a NameableArtefact of cls from elem and kwargs.

Collects all parsed InternationalString localizations of <com:Name> and <com:Description>.

peek(cls_or_name: Type | str)

Get the object at the top of stack cls_or_name without removing it.

pop_all(cls_or_name: Type | str, subclass=False) → Sequence

Pop all objects from stack cls_or_name and return.

If cls_or_name is a class and subclass is True; return all objects in the stack cls_or_name or any stack for a subclass of this class.

pop_resolved_ref(cls_or_name: Type | str)

Pop a reference to cls_or_name and resolve it.

pop_single(cls_or_name: Type | str)

Pop a single object from the stack for cls_or_name and return.

classmethod possible_reference(cls_hint: type | None = None, unstash: bool = False)

Decorator for a function where the elem parsed may be a Reference.

Before calling the decorated function, attempt to parse the elem as a Reference. If successful, return the reference instead of calling the function. If elem does not contain a reference, call the decorated function.

Parameters:

• cls_hint – Passed to Reference.

• unstash (bool, optional) – If True, call unstash() after successfully resolving a reference.

push(stack_or_obj, obj=None)

Push an object onto a stack.

resolve(ref)

Resolve the Reference instance ref, returning the referred object.

classmethod start(names: str, only: bool = True)

Decorator for a function that parses “start” events for XML elements.

stash(*stacks)

Temporarily hide all objects in the given stacks.

unstash()

Restore the objects hidden by the last stash() call to their stacks.

Calls to stash() and unstash() should be matched 1-to-1; if the latter outnumber the former, this will raise KeyError.

class sdmx.reader.xml.v21.Reference(reader, elem, cls_hint=None)

Temporary class for references.

• cls, id, version, and agency_id are always for a MaintainableArtefact.

• If the reference target is a MaintainableArtefact (maintainable is True), target_cls and target_id are identical to cls and id, respectively.

• If the target is not maintainable, target_cls and target_id describe it.

cls_hint is an optional hint for when the object is instantiated, i.e. a more specific override for cls/target_cls.

sdmx.reader.xml.v21.add_localizations(target: InternationalString, values: list) → None

Add localized strings from values to target.

sdmx.reader.xml.v21.add_mds_events(reader: Reader, mds: MetadataStructureDefinition)

Add parser events for structure-specific metadata.

sdmx.reader.xml.v21.end(names: str, only: bool = True)

Decorator for a function that parses “end” events for XML elements.

sdmx.reader.xml.v21.matching_class(cls)

Filter condition; see get_single() and pop_all().

sdmx.reader.xml.v21.possible_reference(cls_hint: type | None = None, unstash: bool = False)

Decorator for a function where the elem parsed may be a Reference.

Before calling the decorated function, attempt to parse the elem as a Reference. If successful, return the reference instead of calling the function. If elem does not contain a reference, call the decorated function.

Parameters:

• cls_hint – Passed to Reference.

• unstash (bool, optional) – If True, call unstash() after successfully resolving a reference.

sdmx.reader.xml.v21.start(names: str, only: bool = True)

Decorator for a function that parses “start” events for XML elements.

sdmx.reader.xml.v21.to_snake(value)

Convert value from lowerCamelCase to snake_case.

SDMX-ML 3.0.0 reader.

class sdmx.reader.xml.v30.Reader

class Reference(reader, elem, cls_hint=None)

Parse SDMX-ML 3.0 references.

format: ClassVar = <module 'sdmx.format.xml.v30' from '/home/docs/checkouts/readthedocs.org/user_builds/sdmx1/envs/latest/lib/python3.12/site-packages/sdmx/format/xml/v30.py'>

model: ClassVar = <module 'sdmx.model.v30' from '/home/docs/checkouts/readthedocs.org/user_builds/sdmx1/envs/latest/lib/python3.12/site-packages/sdmx/model/v30.py'>

class sdmx.reader.xml.v30.Reference(reader, elem, cls_hint=None)

Parse SDMX-ML 3.0 references.

SDMX-JSON

SDMX-JSON v2.1 reader

class sdmx.reader.json.Reader

Read SDMX-JSON and expose it as instances from sdmx.model.

classmethod detect(content)

Detect whether the reader can handle content.

Returns:

True if the reader can handle the content.

Return type:

bool

media_types: List[MediaType] = [application/vnd.sdmx.data+json; version=1.0.0, application/vnd.sdmx.structure+json; version=1.0.0, application/vnd.sdmx.draft-sdmx-json+json; version=1.0.0, draft-sdmx-json; version=1.0.0, text/json; version=1.0.0]

List of media types handled by the reader.

read_dataset(root, ds_key)

read_message(source, dsd=None)

Read message from source.

Parameters:

• source (file-like) – Message content.

• dsd (DataStructureDefinition, optional) – DSD for aid in reading source.

Returns:

An instance of a Message subclass.

Return type:

Message

read_obs(root, series_key=None, base_key=None)

suffixes: List[str] = ['.json']

List of file name suffixes handled by the reader.

SDMX-CSV

class sdmx.reader.csv.Reader

Stub (incomplete) implementation of a SDMX-CSV reader.

media_types: List[MediaType] = [application/vnd.sdmx.data+csv; version=1.0.0, application/vnd.sdmx.metadata+csv; version=2.0.0]

List of media types handled by the reader.

read_message(source, dsd=None)

Not implemented.

suffixes: List[str] = ['.csv']

List of file name suffixes handled by the reader.

Reader API

sdmx.reader.READERS = [<class 'sdmx.reader.json.Reader'>, <class 'sdmx.reader.xml.Reader'>]

Reader classes

sdmx.reader.detect_content_reader(content)

Return a reader class for content.

The BaseReader.detect() method for each class in READERS is called; if a reader signals that it is compatible with content, then that class is returned.

Raises:

ValueError – If no reader class matches.

sdmx.reader.get_reader_for_media_type(value)

Return a reader class for HTTP content/media type value.

Raises:

ValueError – If no reader class matches.

See also

BaseReader.media_type

sdmx.reader.get_reader_for_path(path)

Return a reader class for file path.

Raises:

ValueError – If no reader class matches.

See also

BaseReader.suffixes

sdmx.reader.read_sdmx(filename_or_obj, format=None, **kwargs)

Load a SDMX-ML or SDMX-JSON message from a file or file-like object.

Parameters:

• filename_or_obj (str or PathLike or file) –

• format ('XML' or 'JSON', optional) –

• dsd (DataStructureDefinition) – For “structure-specific” format`=``XML` messages only.

class sdmx.reader.base.BaseReader

classmethod detect(content: bytes) → bool

Detect whether the reader can handle content.

Returns:

True if the reader can handle the content.

Return type:

bool

classmethod handles_media_type(value: str) → bool

True if the reader can handle content/media type value.

media_types: List[MediaType] = []

List of media types handled by the reader.

abstract read_message(source, dsd=None)

Read message from source.

Parameters:

• source (file-like) – Message content.

• dsd (DataStructureDefinition, optional) – DSD for aid in reading source.

Returns:

An instance of a Message subclass.

Return type:

Message

suffixes: List[str] = []

List of file name suffixes handled by the reader.

classmethod supports_suffix(value: str) → bool

True if the reader can handle files with suffix value.