API reference#
Some parts of the API are described on separate pages:
sdmx.source
on the page Data sources.
See also the Implementation notes.
Top-level methods and classes#
|
Client for a SDMX REST web service. |
|
Enumeration of SDMX-REST API resources. |
|
Add a new data source. |
Return a sorted list of valid source IDs. |
|
Top-level logger. |
|
|
Load a SDMX-ML or SDMX-JSON message from a file or file-like object. |
|
Request a URL directly. |
|
Convert an SDMX obj to |
|
Convert an SDMX obj to SDMX-ML. |
- class sdmx.Client(source=None, log_level=None, **session_opts)[source]#
Client for a SDMX REST web service.
- Parameters:
source (str or source.Source) – Identifier of a data source. If a string, must be one of the known sources in
list_sources()
.log_level (int) –
Override the package-wide logger with one of the standard logging levels.
Deprecated since version 2.0: Will be removed in
sdmx
version 3.0.**session_opts – Additional keyword arguments are passed to
Session
.
- get(resource_type=None, resource_id=None, tofile=None, use_cache=False, dry_run=False, **kwargs)[source]#
Retrieve SDMX data or metadata.
(Meta)data is retrieved from the
source
of the current Client. The item(s) to retrieve can be specified in one of two ways:resource_type, resource_id: These give the type (see
Resource
) and, optionally, ID of the item(s). If the resource_id is not given, all items of the given type are retrieved.a resource object, i.e. a
MaintainableArtefact
: resource_type and resource_id are determined by the object’s class andid
attribute, respectively.
Data is retrieved with resource_type=’data’. In this case, the optional keyword argument key can be used to constrain the data that is retrieved. Examples of the formats for key:
{'GEO': ['EL', 'ES', 'IE']}
:dict
with dimension name(s) mapped to an iterable of allowable values.{'GEO': 'EL+ES+IE'}
:dict
with dimension name(s) mapped to strings joining allowable values with ‘+’, the logical ‘or’ operator for SDMX web services.'....EL+ES+IE'
:str
in which ordered dimension values (some empty,''
) are joined with'.'
. Using this form requires knowledge of the dimension order in the target data resource_id; in the example, dimension ‘GEO’ is the fifth of five dimensions:'.'.join(['', '', '', '', 'EL+ES+IE'])
.CubeRegion.to_query_string()
can also be used to create properly formatted strings.
For formats 1 and 2, but not 3, the key argument is validated against the relevant
DataStructureDefinition
, either given with the dsd keyword argument, or retrieved from the web service before the main query.For the optional param keyword argument, some useful parameters are:
‘startperiod’, ‘endperiod’: restrict the time range of data to retrieve.
‘references’: control which item(s) related to a metadata resource are retrieved, e.g. references=’parentsandsiblings’.
- Parameters:
resource_type (str or
Resource
, optional) – Type of resource to retrieve.resource_id (str, optional) – ID of the resource to retrieve.
tofile (str or
PathLike
or file-like object, optional) – File path or file-like to write SDMX data as it is received.use_cache (bool, optional) – If
True
, return a previously retrievedMessage
fromcache
, or update the cache with a newly-retrieved Message.dry_run (bool, optional) – If
True
, prepare and return arequests.Request
object, but do not execute the query. The prepared URL and headers can be examined by inspecting the returned object.**kwargs – Other, optional parameters (below).
dsd (
DataStructureDefinition
) – Existing object used to validate the key argument. If not provided, an additional query executed to retrieve a DSD in order to validate the key.force (bool) – If
True
, execute the query even if thesource
does not support queries for the given resource_type. Default:False
.headers (dict) – HTTP headers. Given headers will overwrite instance-wide headers passed to the constructor. Default:
None
to use the default headers of thesource
.key (str or dict) – For queries with resource_type=’data’.
str
values are not validated;dict
values are validated usingmake_constraint()
.params (dict) – Query parameters. The SDMX REST web service guidelines describe parameters and allowable values for different queries. params is not validated before the query is executed.
provider (str) –
ID of the agency providing the data or metadata. Default: ID of the
source
agency.An SDMX web service is a ‘data source’ operated by a specific, ‘source’ agency. A web service may host data or metadata originally published by one or more ‘provider’ agencies. Many sources are also providers. Other agencies—e.g. the SDMX Global Registry—simply aggregate (meta)data from other providers, but do not provide any (meta)data themselves.
resource (
MaintainableArtefact
subclass) – Object to retrieve. If given, resource_type and resource_id are ignored.version (str) –
version>
of a resource to retrieve. Default: the keyword ‘latest’.
- Returns:
The requested SDMX message or, if dry_run is
True
, the prepared request object.- Return type:
- Raises:
NotImplementedError – If the
source
does not support the given resource_type and force is notTrue
.
- preview_data(flow_id, key={})[source]#
Return a preview of data.
For the Dataflow flow_id, return all series keys matching key. Uses a feature supported by some data providers that returns
SeriesKeys
without the correspondingObservations
.To count the number of series:
keys = sdmx.Client('PROVIDER').preview_data('flow') len(keys)
To get a
pandas
object containing the key values:keys_df = sdmx.to_pandas(keys)
- Parameters:
- Return type:
list of
SeriesKey
- source = None#
source.Source
for requests sent from the instance.
- sdmx.Request(*args, **kwargs)[source]#
Compatibility function for
Client
.New in version 2.0.
Deprecated since version 2.0: Will be removed in
sdmx
version 3.0.
- class sdmx.Resource(value)[source]#
Enumeration of SDMX-REST API resources.
Enum
membersdmx.model
classactualconstraint
agencyscheme
allowedconstraint
attachementconstraint
categorisation
categoryscheme
codelist
conceptscheme
contentconstraint
data
dataflow
dataconsumerscheme
dataproviderscheme
datastructure
metadataflow
metadatastructure
organisationscheme
provisionagreement
structure
Mixed.
customtypescheme
Not implemented.
hierarchicalcodelist
Not implemented.
metadata
Not implemented.
namepersonalisationscheme
Not implemented.
organisationunitscheme
Not implemented.
process
Not implemented.
reportingtaxonomy
Not implemented.
rulesetscheme
Not implemented.
schema
Not implemented.
structureset
Not implemented.
transformationscheme
Not implemented.
userdefinedoperatorscheme
Not implemented.
vtlmappingscheme
Not implemented.
- classmethod class_name(value: Resource, default=None) str [source]#
Return the name of a
sdmx.model
class from an enum value.Values are returned in lower case.
- sdmx.add_source(info: Union[Dict, str], id: Optional[str] = None, override: bool = False, **kwargs) None [source]#
Add a new data source.
The info expected is in JSON format:
{ "id": "ESTAT", "documentation": "http://data.un.org/Host.aspx?Content=API", "url": "http://ec.europa.eu/eurostat/SDMX/diss-web/rest", "name": "Eurostat", "supports": {"codelist": false, "preview": true} }
…with unspecified values using the defaults; see
Source
.- Parameters:
info (dict-like) – String containing JSON information about a data source.
id (str) – Identifier for the new datasource. If
None
(default), then info[‘id’] is used.override (bool) – If
True
, replace any existing data source with id. Otherwise, raiseValueError
.**kwargs – Optional callbacks for handle_response and finish_message hooks.
- sdmx.list_sources()[source]#
Return a sorted list of valid source IDs.
These can be used to create
Client
instances.
- sdmx.read_sdmx(filename_or_obj, format=None, **kwargs)[source]#
Load a SDMX-ML or SDMX-JSON message from a file or file-like object.
- Parameters:
filename_or_obj (str or
PathLike
or file) –format ('XML' or 'JSON', optional) –
dsd (
DataStructureDefinition
) – For “structure-specific” format`=``XML` messages only.
- sdmx.to_pandas(obj, *args, **kwargs)[source]#
Convert an SDMX obj to
pandas
object(s).See sdmx.writer.pandas.
- sdmx.to_xml(obj, **kwargs)[source]#
Convert an SDMX obj to SDMX-ML.
- Parameters:
kwargs – Passed to
lxml.etree.to_string()
, e.g. pretty_print =True
.- Raises:
NotImplementedError – If writing specific objects to SDMX-ML has not been implemented in
sdmx
.
format
: SDMX file formats#
This information is used across other modules including sdmx.reader
,
sdmx.client
, and sdmx.writer
.
- class sdmx.format.Flag(value)#
Bases:
IntFlag
Flag values for information about
MediaType
:data
:True
if this format contains (meta)data.False
if it contains (meta)data structures.meta
:True
if this format contains metadata (or metadata structures).False
otherwise.ss
:True
if this format contains data that is structure-specific. This distinction is only relevant before SDMX 3.0.ts
:True
if this format contains time-series data. This distinction is only relevant before SDMX 3.0.
- data = 1#
- meta = 2#
- ss = 4#
- ts = 8#
- sdmx.format.MEDIA_TYPES = [application/vnd.sdmx.generic+xml; version=2.1, application/vnd.sdmx.genericdata+xml; version=2.1, application/vnd.sdmx.structurespecificdata+xml; version=2.1, application/vnd.sdmx.generictimeseriesdata+xml; version=2.1, application/vnd.sdmx.structurespecifictimeseriesdata+xml; version=2.1, application/vnd.sdmx.structure+xml; version=2.1, application/vnd.sdmx.data+xml; version=3.0.0, application/vnd.sdmx.structure+xml; version=3.0.0, application/vnd.sdmx.metadata+xml; version=2.0.0, application/vnd.sdmx.schema+xml; version=2.1, application/vnd.sdmx.genericmetadata+xml; version=2.1, application/vnd.sdmx.structurespecificmetadata+xml; version=2.1, application/xml; version=2.1, text/xml; version=2.1, application/vnd.sdmx.data+json; version=1.0.0, application/vnd.sdmx.data+json; version=2.0.0, application/vnd.sdmx.structure+json; version=1.0.0, application/vnd.sdmx.structure+json; version=2.0.0, application/vnd.sdmx.metadata+json; version=2.0.0, application/vnd.sdmx.draft-sdmx-json+json; version=1.0.0, draft-sdmx-json; version=1.0.0, text/json; version=1.0.0, application/vnd.sdmx.data+csv; version=1.0.0, application/vnd.sdmx.metadata+csv; version=2.0.0]#
SDMX formats. Each record is an instance of
Format
.
- class sdmx.format.MediaType(label: str, base: typing_extensions.Literal[csv, json, xml], version: typing_extensions.Literal[1.0.0, 2.0.0, 2.1, 3.0.0, unknown], flags: ~sdmx.format.Flag = Flag.None, full: ~typing.Optional[str] = None)[source]#
Bases:
object
Structure of elements in
MEDIA_TYPES
.The
str()
of a MediaType is generally of the form:application/vnd.sdmx.{label}+{base};version={version}
…unless
full
is provided, in which case label and base are ignored.- base: typing_extensions.Literal[csv, json, xml]#
The base media type or file format.
- version: typing_extensions.Literal[1.0.0, 2.0.0, 2.1, 3.0.0, unknown]#
Format version.
- sdmx.format.list_media_types(**filters) List[MediaType] [source]#
Return the string for each item in
MEDIA_TYPES
matching filters.
SDMX-JSON#
Information about the SDMX-JSON file format.
SDMX-ML#
Information about the SDMX-ML file format.
message
: SDMX messages#
Classes for SDMX messages.
Message
and related classes are not defined in the SDMX
information model, but in the SDMX-ML standard.
sdmx
also uses DataMessage
to encapsulate SDMX-JSON data returned by
data sources.
- class sdmx.message.DataMessage(*, header: ~sdmx.message.Header = <Header> source: test: False, footer: ~typing.Optional[~sdmx.message.Footer] = None, response: ~typing.Optional[~typing.Any] = None, data: ~typing.List[~sdmx.model.v21.DataSet] = [], dataflow: ~sdmx.model.v21.DataflowDefinition = <DataflowDefinition (missing id)>, observation_dimension: ~typing.Optional[~typing.Union[~sdmx.model.v21._AllDimensions, ~sdmx.model.v21.DimensionComponent, ~typing.List[~sdmx.model.v21.DimensionComponent]]] = None)[source]#
Bases:
Message
Data Message.
Note
A DataMessage may contain zero or more
DataSet
, sodata
is a list. To retrieve the first (and possibly only) data set in the message, access the first element of the list:msg.data[0]
.- compare(other, strict=True)[source]#
Return
True
if self is the same as other.Two DataMessages are the same if:
their
dataflow
andobservation_dimension
compare equal.they have the same number of
DataSets
, andcorresponding DataSets compare equal (see
DataSet.compare()
).
- dataflow: DataflowDefinition#
DataflowDefinition
that contains the data.
- observation_dimension: Optional[Union[_AllDimensions, DimensionComponent, List[DimensionComponent]]]#
The “dimension at observation level”.
- class sdmx.message.ErrorMessage(*, header: ~sdmx.message.Header = <Header> source: test: False, footer: ~typing.Optional[~sdmx.message.Footer] = None, response: ~typing.Optional[~typing.Any] = None)[source]#
Bases:
Message
(optional)
Footer
instance.
- response: Optional[Any]#
requests.Response
instance for the response to the HTTP request that returned the Message. This is not part of the SDMX standard.
Bases:
BaseModel
Footer of an SDMX-ML message.
SDMX-JSON messages do not have footers.
Return
True
if self is the same as other.Two Footers are the same if their
code
,severity
, andtext
are equal.
The body text of the Footer contains zero or more blocks of text.
- class sdmx.message.Header(*, error: ~typing.Optional[str] = None, extracted: ~typing.Optional[~datetime.datetime] = None, id: ~typing.Optional[str] = None, prepared: ~typing.Optional[~datetime.datetime] = None, reporting_begin: ~typing.Optional[~datetime.datetime] = None, reporting_end: ~typing.Optional[~datetime.datetime] = None, receiver: ~typing.Optional[~sdmx.model.common.Agency] = None, sender: ~typing.Optional[~sdmx.model.common.Agency] = None, source: ~sdmx.model.internationalstring.InternationalString =, test: bool = False)[source]#
Bases:
BaseModel
Header of an SDMX-ML message.
SDMX-JSON messages do not have headers.
- compare(other, strict=True)[source]#
Return
True
if self is the same as other.Two Headers are the same if their corresponding attributes are equal.
- receiver: Optional[Agency]#
Intended recipient of the message, e.g. the user’s name for an authenticated service.
- reporting_begin: Optional[datetime]#
Start of the time period covered by a
DataMessage
.
- reporting_end: Optional[datetime]#
End of the time period covered by a
DataMessage
.
- source: InternationalString#
- class sdmx.message.Message(*, header: ~sdmx.message.Header = <Header> source: test: False, footer: ~typing.Optional[~sdmx.message.Footer] = None, response: ~typing.Optional[~typing.Any] = None)[source]#
Bases:
BaseModel
- compare(other, strict=True)[source]#
Return
True
if self is the same as other.Two Messages are the same if their
header
andfooter
compare equal.
(optional)
Footer
instance.
- response: Optional[Any]#
requests.Response
instance for the response to the HTTP request that returned the Message. This is not part of the SDMX standard.
- class sdmx.message.StructureMessage(*, header: ~sdmx.message.Header = <Header> source: test: False, footer: ~typing.Optional[~sdmx.message.Footer] = None, response: ~typing.Optional[~typing.Any] = None, categorisation: ~sdmx.util.DictLike[str, ~sdmx.model.common.Categorisation] = None, category_scheme: ~sdmx.util.DictLike[str, ~sdmx.model.common.CategoryScheme] = None, codelist: ~sdmx.util.DictLike[str, ~sdmx.model.v21.Codelist] = None, concept_scheme: ~sdmx.util.DictLike[str, ~sdmx.model.common.ConceptScheme] = None, constraint: ~sdmx.util.DictLike[str, ~sdmx.model.v21.ContentConstraint] = None, dataflow: ~sdmx.util.DictLike[str, ~sdmx.model.v21.DataflowDefinition] = None, metadataflow: ~sdmx.util.DictLike[str, ~sdmx.model.v21.MetadataflowDefinition] = None, structure: ~sdmx.util.DictLike[str, ~sdmx.model.v21.DataStructureDefinition] = None, organisation_scheme: ~sdmx.util.DictLike[str, ~sdmx.model.common.AgencyScheme] = None, provisionagreement: ~sdmx.util.DictLike[str, ~sdmx.model.v21.ProvisionAgreement] = None)[source]#
Bases:
Message
- add(obj: IdentifiableArtefact)[source]#
Add obj to the StructureMessage.
- categorisation: DictLike[str, Categorisation]#
Collection of
Categorisation
.
- category_scheme: DictLike[str, CategoryScheme]#
Collection of
CategoryScheme
.
- compare(other, strict=True)[source]#
Return
True
if self is the same as other.Two StructureMessages compare equal if
DictLike.compare()
isTrue
for each of the object collection attributes.- Parameters:
strict (bool, optional) – Passed to
DictLike.compare()
.
- concept_scheme: DictLike[str, ConceptScheme]#
Collection of
ConceptScheme
.
- constraint: DictLike[str, ContentConstraint]#
Collection of
ContentConstraint
.
- dataflow: DictLike[str, DataflowDefinition]#
Collection of
DataflowDefinition
.
- get(obj_or_id: Union[str, IdentifiableArtefact]) Optional[IdentifiableArtefact] [source]#
Retrieve obj_or_id from the StructureMessage.
- Parameters:
obj_or_id (str or .IdentifiableArtefact) – If an IdentifiableArtefact, return an object of the same class and
id
; ifstr
, an object with this ID.- Returns:
.IdentifiableArtefact – with the given ID and possibly class.
None – if there is no match.
- Raises:
ValueError – if obj_or_id is a string and there are ≥2 objects (of different classes) with the same ID.
- metadataflow: DictLike[str, MetadataflowDefinition]#
Collection of
DataflowDefinition
.
- objects(cls)[source]#
Get a reference to the attribute for objects of type cls.
For example, if cls is the class
DataStructureDefinition
(not an instance), return a reference tostructure
.
- organisation_scheme: DictLike[str, AgencyScheme]#
Collection of
AgencyScheme
.
- provisionagreement: DictLike[str, ProvisionAgreement]#
Collection of
ProvisionAgreement
.
- structure: DictLike[str, DataStructureDefinition]#
Collection of
DataStructureDefinition
.
rest
: SDMX-REST standard#
Information related to the SDMX-REST web service standard.
- sdmx.rest.RESPONSE_CODE = {200: 'OK', 304: 'No changes', 400: 'Bad syntax', 401: 'Unauthorized', 403: 'Semantic error', 404: 'Not found', 406: 'Not acceptable', 413: 'Request entity too large', 414: 'URI too long', 500: 'Internal server error', 501: 'Not implemented', 503: 'Unavailable'}#
Response codes defined by the SDMX-REST standard.
- class sdmx.rest.URL(source: sdmx.source.Source, resource_type: Resource, resource_id: str, provider: Optional[str] = None, agencyID: Optional[str] = None, version: Optional[str] = None, key: Optional[str] = None)[source]#
Bases:
object
Utility class to build SDMX REST URLs.
See also
https
//github.com/sdmx-twg/sdmx-rest/blob/v1.5.0/v2_1/ws/rest/src/sdmx-rest.yaml
session
: Access SDMX REST web services#
- class sdmx.session.Session(timeout=30.1, **kwargs)[source]#
requests.Session
subclass with optional caching.If
requests_cache
is installed, this class inherits fromCachedSession
and caches responses.
- class sdmx.session.ResponseIO(response, tee=None)[source]#
Buffered wrapper for
requests.Response
with optional file output.ResponseIO
wraps arequests.Response
object’s ‘content’ attribute, providing a file-like object from which bytes can beread()
incrementally.- Parameters:
response (
requests.Response
) – HTTP response to wrap.tee (binary, writable
io.BufferedIOBase
, defaults to io.BytesIO()) – tee is exposed as self.tee and not closed explicitly.
urn
: Uniform Resource Names (URNs) for SDMX objects#
- sdmx.urn.URN = re.compile('urn:sdmx:org\\.sdmx\\.infomodel\\.(?P<package>[^\\.]*)\\.(?P<class>[^=]*)=((?P<agency>[^:]*):)?(?P<id>[^\\(]*)(\\((?P<version>[\\d\\.]*)\\))?(\\.(?P<item_id>.*))?')#
Regular expression for URNs.
- sdmx.urn.make(obj, maintainable_parent=None, strict=False)[source]#
Create an SDMX URN for obj.
If obj is not
MaintainableArtefact
, then maintainable_parent must be supplied in order to construct the URN.
util
: Utilities#
- class sdmx.util.DictLike(*args, **kwargs)[source]#
Bases:
dict
,MutableMapping
[KT
,VT
]Container with features of a dict & list, plus attribute access.
- sdmx.util.Field(default: Any = PydanticUndefined, *, default_factory: Optional[Callable[[], Any]] = None, alias: Optional[str] = None, title: Optional[str] = None, description: Optional[str] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny, Any]] = None, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny, Any]] = None, const: Optional[bool] = None, gt: Optional[float] = None, ge: Optional[float] = None, lt: Optional[float] = None, le: Optional[float] = None, multiple_of: Optional[float] = None, allow_inf_nan: Optional[bool] = None, max_digits: Optional[int] = None, decimal_places: Optional[int] = None, min_items: Optional[int] = None, max_items: Optional[int] = None, unique_items: Optional[bool] = None, min_length: Optional[int] = None, max_length: Optional[int] = None, allow_mutation: bool = True, regex: Optional[str] = None, discriminator: Optional[str] = None, repr: bool = True, **extra: Any) Any #
Used to provide extra information about a field, either for the model schema or complex validation. Some arguments apply only to number fields (
int
,float
,Decimal
) and some apply only tostr
.- Parameters:
default – since this is replacing the field’s default, its first argument is used to set the default, use ellipsis (
...
) to indicate the field is requireddefault_factory – callable that will be called when a default value is needed for this field If both default and default_factory are set, an error is raised.
alias – the public name of the field
title – can be any string, used in the schema
description – can be any string, used in the schema
exclude – exclude this field while dumping. Takes same values as the
include
andexclude
arguments on the.dict
method.include – include this field while dumping. Takes same values as the
include
andexclude
arguments on the.dict
method.const – this field is required and must take it’s default value
gt – only applies to numbers, requires the field to be “greater than”. The schema will have an
exclusiveMinimum
validation keywordge – only applies to numbers, requires the field to be “greater than or equal to”. The schema will have a
minimum
validation keywordlt – only applies to numbers, requires the field to be “less than”. The schema will have an
exclusiveMaximum
validation keywordle – only applies to numbers, requires the field to be “less than or equal to”. The schema will have a
maximum
validation keywordmultiple_of – only applies to numbers, requires the field to be “a multiple of”. The schema will have a
multipleOf
validation keywordallow_inf_nan – only applies to numbers, allows the field to be NaN or infinity (+inf or -inf), which is a valid Python float. Default True, set to False for compatibility with JSON.
max_digits – only applies to Decimals, requires the field to have a maximum number of digits within the decimal. It does not include a zero before the decimal point or trailing decimal zeroes.
decimal_places – only applies to Decimals, requires the field to have at most a number of decimal places allowed. It does not include trailing decimal zeroes.
min_items – only applies to lists, requires the field to have a minimum number of elements. The schema will have a
minItems
validation keywordmax_items – only applies to lists, requires the field to have a maximum number of elements. The schema will have a
maxItems
validation keywordunique_items – only applies to lists, requires the field not to have duplicated elements. The schema will have a
uniqueItems
validation keywordmin_length – only applies to strings, requires the field to have a minimum length. The schema will have a
minLength
validation keywordmax_length – only applies to strings, requires the field to have a maximum length. The schema will have a
maxLength
validation keywordallow_mutation – a boolean which defaults to True. When False, the field raises a TypeError if the field is assigned on an instance. The BaseModel Config must set validate_assignment to True
regex – only applies to strings, requires the field match against a regular expression pattern string. The schema will have a
pattern
validation keyworddiscriminator – only useful with a (discriminated a.k.a. tagged) Union of sub models with a common field. The discriminator is the name of this common field to shorten validation and improve generated schema
repr – show this field in the representation
**extra –
any additional keyword arguments will be added as is to the schema