************************** Persistent Data Instances ************************** .. module:: yangson.instance :synopsis: Persistent data instances. .. testsetup:: import json import os from yangson import DataModel from yangson.enumerations import ContentType os.chdir("examples/ex2") .. testcleanup:: os.chdir("../..") The *instance* module implements the following classes: * :class:`InstanceNode`: Abstract class for instance nodes. * :class:`RootNode`: Root of the data tree. * :class:`ObjectMember`: Instance node that is an object member. * :class:`ArrayEntry`: Instance node that is an array entry. * :class:`InstanceRoute`: Route into an instance value. Doctest__ snippets for this module use the data model and instance document from :ref:`sec-ex2`. __ http://www.sphinx-doc.org/en/stable/ext/doctest.html .. doctest:: >>> dm = DataModel.from_file('yang-library-ex2.json', ... [".", "../../../yang-modules/ietf"]) >>> with open('example-data.json') as infile: ... ri = json.load(infile) >>> inst = dm.from_raw(ri) .. class:: InstanceNode(key: InstanceKey, value: Value, \ parinst: Optional[InstanceNode], \ schema_node: DataNode, timestamp: datetime.datetime) The *key* argument is the key of the instance in the parent structure, i.e. either :term:`instance name` for an :class:`ObjectMember` or integer index for an :class:`ArrayEntry`. The key becomes the last component of the :attr:`path` attribute. Other constructor arguments contain values for instance attributes of the same name. This class and its subclasses implement the *zipper* interface for instance data along the lines of Gérard Huet's original paper [Hue97]_, only adapted for the specifics of JSON-like structures. An important property of the zipper interface is that it makes the underlying data structure persistent__: any changes to the data realized through the methods of the :class:`InstanceNode` class return an updated *copy* of the original instance without changing the latter. As much as possible, the data are shared between the original instance and the updated copy. __ https://en.wikipedia.org/wiki/Persistent_data_structure Whilst the zipper interface slightly complicates access to instance data, it provides the advantages of persistent structures that are known from functional programming languages: * The structures are thread-safe. * It is easy to edit the data and then return to the original version, for example if new version isn't valid according to the data model. * Staging datastores, such as *candidate* in NETCONF (sec. `8.3`_ in [RFC6241]_) can be implemented in a space-efficient way. .. rubric:: Instance Attributes .. attribute:: parinst Parent instance node, or ``None`` for the root node. .. attribute:: schema_node Data node in the schema corresponding to the instance node. .. attribute:: timestamp The date and time when the instance node was last modified. .. attribute:: value Scalar or structured value of the node, see module :mod:`.instvalue`. The arguments of the :class:`InstanceNode` constructor provide values for instance attributes of the same name. .. rubric:: Properties .. attribute:: name The :term:`instance name` of the receiver. For an :class:`ArrayEntry` instance it is by definition the same as the qualified name of the parent :class:`ObjectMember`. .. attribute:: namespace The :term:`namespace identifier` of the instance node. .. attribute:: path Path of the instance in the data tree: a tuple containing keys of the ancestor nodes and the instance itself. .. attribute:: qual_name The :term:`qualified name` of the receiver. For an :class:`ArrayEntry` instance it is by definition the same as the qualified name of the parent :class:`ObjectMember`. An :class:`InstanceNode` structure can be created from scratch, or read from JSON text using :meth:`.DataModel.from_raw` (see the doctest snippet above). The internal representation of :class:`InstanceNode` values is very similar to the JSON encoding of data modelled with YANG [RFC7951]_. In particular, member names have to be in the form specified in sec. `4`_ of that document: .. productionlist:: member-name: [identifier ":"] identifier where the first identifier is a module name and the second is a data node name. The longer (namespace-qualified) form is used if and only if the member is defined in a different YANG module than its parent. .. doctest:: >>> inst.value["example-2:bag"]["bar"] True A structured :class:`InstanceNode` value is represented as either :class:`~.instvalue.ObjectValue` (subclass of :class:`dict`) or :class:`~.instvalue.ArrayValue` (subclass of :class:`list`), see :mod:`.instvalue` module for details. The representation of a scalar value depends on its type (see :mod:`datatype` module). Structured values, and some scalar values as well, are *not* the same as the values provided by the generic JSON parsing functions :func:`json.load` and :func:`json.loads`. Therefore, values read from JSON text need some additional processing, or “cooking”. *Yangson* methods such as :meth:`.DataModel.from_raw` take care of this step. .. doctest:: >>> type(inst.value) Inside the larger structure of a data tree, an :class:`InstanceNode` represents “focus” on a particular node of the structure. The focus can be moved to a neighbour instance node (parent, child, sibling) and the value of an instance node can be created, deleted and updated by using the methods described below. Each of the methods returns a new :class:`InstanceNode` that shares, as much as possible, portions of the surrounding data tree with the original instance node. However, any modifications to the new instance node – if performed through the methods of the :class:`InstanceNode` class and its subclasses – leave other instance nodes intact. Most methods for moving the focus inside the zipper structure and updating the value of an instance node are defined in the :class:`InstanceNode`, additional methods that are specific to an :class:`ObjectMember` or :class:`ArrayEntry` are defined in the respective class. .. rubric:: Public Methods .. automethod:: __str__ If the receiver's value is a scalar, then the result is the :term:`canonical representation` of the value, if it is defined for the value's type (see sec. `9`_ in [RFC7950]_). Otherwise, the result is the value returned by Python standard function :class:`str`. .. method:: __getitem__(key: InstanceKey) -> InstanceNode This method allows for selecting receiver's member or entry using square brackets as it is usual for other Python sequence types. The argument *key* is * an integer index, if the receiver's value is an array (negative indices are also supported), or * an :term:`instance name`, if the receiver's value is an object. The value returned by this method is either an :class:`ObjectMember` or :class:`ArrayEntry`. This method raises :exc:`~.InstanceValueError` if receiver's value is not structured, and :exc:`~.NonexistentInstance` if the member or entry identified by *key* doesn't exist in the actual receiver's value. .. doctest:: >>> bag = inst['example-2:bag'] >>> foo = bag['foo'] >>> foo.path ('example-2:bag', 'foo') >>> bag['baz'] Traceback (most recent call last): ... yangson.exceptions.NonexistentInstance: {/example-2:bag} member 'baz' >>> foo6 = foo[0] >>> foo6.value['number'] 6 >>> fool = foo[-1] >>> fool.value['in-words'] 'eight' >>> foo[4] Traceback (most recent call last): ... yangson.exceptions.NonexistentInstance: {/example-2:bag/foo} entry 4 .. method:: __iter__() Return receiver's iterator. If the receiver's value is an object, then this method returns simply the value's iterator that can be used exactly as a Python dictionary iterator: .. doctest:: >>> sorted([m for m in bag]) ['bar', 'foo'] However, if the receiver's value is an array, the returned iterator yields successive :class:`ArrayEntry` instances: .. doctest:: >>> [e.json_pointer() for e in foo] ['/example-2:bag/foo/0', '/example-2:bag/foo/1', '/example-2:bag/foo/2', '/example-2:bag/foo/3'] An attempt to iterate over an :class:`InstanceNode` that has a scalar value raises :exc:`~.InstanceValueError`. .. automethod:: json_pointer() -> JSONPointer .. doctest:: >>> fool.json_pointer() '/example-2:bag/foo/3' .. automethod:: instance_route() -> InstanceRoute .. doctest:: >>> str(fool.instance_route()) '/example-2:bag/foo[number="8"]' .. method:: is_internal() -> bool Return ``True`` if the receiver is an instance of an internal schema node, i.e. its :attr:`schema_node` is an :class:`~.schemanode.InternalNode`. Otherwise return ``False``. .. doctest:: >>> inst.is_internal() True .. method:: put_member(name: InstanceName, value: Union[RawValue, \ Value], raw: bool = False) -> InstanceNode Return receiver's member *name* with a new value specified by the *value* argument. The *raw* flag has to be set to ``True`` if *value* is a :term:`raw value`. If member *name* doesn't exist in the receiver's value, it is created (provided that the schema permits it). This method raises :exc:`~.InstanceValueError` if the receiver's value is not an object, and :exc:`~.NonexistentSchemaNode` if the schema doesn't permit member *name*. .. doctest:: >>> nbar = bag.put_member('bar', False) >>> nbar.value False >>> bag.value['bar'] # bag is unchanged True >>> e2bag = bag.put_member('baz', 3.1415926).up() # baz is created >>> sorted(e2bag.value.keys()) ['bar', 'baz', 'foo'] >>> bag.put_member('quux', 0) Traceback (most recent call last): ... yangson.exceptions.NonexistentSchemaNode: quux under example-2:bag .. method:: delete_item(key: InstanceKey) -> InstanceNode Return a new instance node that is an exact copy of the receiver, except that item *key* is deleted from its value. This method raises :exc:`~.InstanceValueError` if the receiver's value is a scalar, and:exc:`~.NonexistentInstance` if the item isn't present in the actual receiver's value. .. doctest:: >>> xbag = e2bag.delete_item('baz') >>> sorted(xbag.value.keys()) ['bar', 'foo'] >>> sorted(e2bag.value.keys()) # e2bag is unvchanged ['bar', 'baz', 'foo'] >>> xfoo = foo.delete_item(0) >>> len(xfoo.value) 3 >>> len(foo.value) # foo is unchanged 4 .. method:: look_up(raw: bool = False, /, \ **keys: Dict[InstanceName, ScalarValue]) -> ArrayEntry Return an instance node corresponding to the receiver's entry with specified keys. The receiver must be a YANG list. The keys are passed to this method as a sequence of Python `keyword arguments`_ in the form ``key=value`` where ``key`` is the :term:`instance name` of a key, and ``value`` is the corresponding key value. .. doctest:: >>> foo8 = foo.look_up(number=8) >>> foo8.json_pointer() '/example-2:bag/foo/3' Keyword arguments won't work for keys with namespace-qualified names such as ``yangmod:index``. In this case, the keys and values have to be packed in a dictionary and passed to the method as follows:: mylist.look_up(**{'yangmod:index': 42}) By default, the values of all keys are expected to be passed as :term:`cooked value`\ s. If :term:`raw value`\ s are passed instead, the *raw* flag has to be set to ``True``. In this case, this flag must appear as the **first positional** argument so as to avoid interference with the remaining arguments that are specified in the ``key=value`` form: .. doctest:: >>> foo.look_up(True, number='8').json_pointer() '/example-2:bag/foo/3' Whilst this method is mainly intended for use with YANG list keys (as the *number* leaf in the example above) but, with a bit of caution, it can be used with any child nodes of the receiver: .. doctest:: >>> foo.look_up(prime=True)['number'].value 3 The first list entry that satisfies the look-up criteria is returned. .. note:: Default values of leaves are always ignored by the look-up procedure. Therefore, if you need to take defaults into account, populate the receiver first with default values by using the :meth:`add_defaults` method. This method raises :exc:`~.InstanceValueError` if the receiver is not a YANG list, and :exc:`~.NonexistentInstance` if no entry with matching keys exists. .. method:: up() -> InstanceNode Return an instance node corresponding to the receiver's parent. This method raises :exc:`~.NonexistentInstance` if the receiver is the root of the data tree and thus has no parent. .. doctest:: >>> foo.up().name 'example-2:bag' >>> inst.up() Traceback (most recent call last): ... yangson.exceptions.NonexistentInstance: {/} up of top .. automethod:: top() -> InstanceNode .. doctest:: >>> e2inst = e2bag.top() >>> e2inst.value['example-2:bag']['baz'] 3.1415926 .. method:: update(value: Union[RawValue, Value], raw: bool = \ False) -> InstanceNode Return a new instance node that is a copy of the receiver with a value specified by the *value* argument. The *raw* flag has to be set to ``True`` if *value* is a :term:`raw value`. .. doctest:: >>> ebar = bag['bar'].update(False) >>> ebar.value False In the following example, the string ``'2.7182818'`` is an acceptable :term:`raw value` for the *baz* leaf whose type is **decimal64** (see sec. `6.1`_ in [RFC7951]_). Since the *raw* flag is set, the :meth:`update` method “cooks” the raw value first into the Python's :class:`decimal.Decimal` type. .. doctest:: >>> e3baz = e2bag['baz'].update('2.7182818', raw=True) >>> e3baz.value Decimal('2.7182818') >>> e2bag['foo'][0]['in-words'].update(66, raw=True) Traceback (most recent call last): ... yangson.exceptions.RawTypeError: {/example-2:bag/foo/0/in-words} expected string value .. method:: merge(value: Union[RawValue, Value], raw: bool = \ False) -> InstanceNode Return a new instance node whose value is the receiver's value merged with the *value* argument, in the sense of the **merge** operation in [RFC8072]_. The *raw* flag has to be set to ``True`` if *value* is a :term:`raw value`. The method uses (recursively) the following rules for merging instance values, as specified in [RFC7950]_, sections `7.5.8`_, `7.6.7`_, `7.7.9`_ and `7.8.6`_. * If the receiver is a leaf or anydata instance, the result is the same as for :meth:`update`, i.e. the receiver's value is overwritten by *value*. * Otherwise, receiver's entries or members are matched against the entries or members in *value* using list keys, leaf-list values or member names. Matching entries/members are merged, and non-matching entries/members are copied from both sources into the result. * If the receiver is a list or leaf-list instance, then the non-matching entries from *value* are appended at the end. This is also true for lists/leaf-lists ordered by user (see section `7.7.7`_ in [RFC7950]_). .. doctest:: >>> mfoo = foo.merge([{'number': '8', 'in-words': 'acht'}, ... {'number': '9', 'in-words': 'nine'}, ... {'number': '6', 'in-words': 'sechs'}, ... {'number': '11', 'prime': True, 'in-words': 'eleven'}], ... raw = True) >>> [en['in-words'].value for en in mfoo] ['sechs', 'three', 'seven', 'acht', 'nine', 'eleven'] >>> [en['in-words'].value for en in foo] # original instance doesn't change ['six', 'three', 'seven', 'eight'] .. method:: goto(iroute: InstanceRoute) -> InstanceNode Return an :class:`InstanceNode` corresponding to a target instance arbitrarily deep inside the receiver's value. The argument *iroute* is an :class:`InstanceRoute` (relative to the receiver) that identifies the target instance. The easiest way for obtaining an :class:`InstanceRoute` is to parse it either from a :term:`resource identifier` or :term:`instance identifier` using methods :meth:`.DataModel.parse_resource_id` and :meth:`.DataModel.parse_instance_id`, respectively. .. doctest:: >>> irt = dm.parse_resource_id('/example-2:bag/foo=3/in-words') >>> irt2 = dm.parse_instance_id('/example-2:bag/baz') This method may raise the following exceptions: * :exc:`~.InstanceValueError` if *iroute* isn't compatible with the schema * :exc:`~.NonexistentInstance` if the target instance doesn't exist in the receiver's value * :exc:`~.NonDataNode` if the target instance represents an RPC operation, action or notification (*iroute* can come from a RESTCONF :term:`resource identifier`). .. doctest:: >>> inst.goto(irt).value 'three' >>> inst.goto(irt2) Traceback (most recent call last): ... yangson.exceptions.NonexistentInstance: {/example-2:bag} member 'baz' .. method:: peek(iroute: InstanceRoute) -> Optional[Value] Return the value of a target instance arbitrarily deep inside the receiver's value. The argument *iroute* is an :class:`InstanceRoute` (relative to the receiver) that identifies the target instance. ``None`` is returned if the target instance doesn't exist. .. doctest:: >>> inst.peek(irt) 'three' .. CAUTION:: This method doesn't create a new instance, so the access to the returned value should in general be read-only. Any modifications of the returned value also affect the receiver, as shown in the next example. This means that the persistence property for the receiver is lost. .. doctest:: >>> irt3 = dm.parse_resource_id('/example-2:bag/foo=3') >>> e2inst.peek(irt3)['in-words'] = 'tres' >>> e2inst.value['example-2:bag']['foo'][1]['in-words'] # changed! 'tres' .. method:: validate(scope: ValidationScope = ValidationScope.all, \ ctype: ContentType = ContentType.config) -> None Perform validation on the receiver's value. The *scope* argument determines the validation scope. The options are as follows: * ``ValidationScope.syntax`` – verifies schema constraints (taking into account **if-feature** and **when** statements, if present) and data types. * ``ValidationScope.semantics`` – verifies **must** constraints, uniqueness of list keys, **unique** constraints in list nodes, and integrity of **leafref** references. * ``ValidationScope.all`` – performs all checks from both items above. The value of the *ctype* argument belongs to the :class:`~.enumerations.ContentType` enumeration and specifies whether the receiver's value is to be validated as configuration (``Content.config``) or as both configuration and state data (``Content.all``). The method returns ``None`` if the validation succeeds, otherwise one of the following exceptions is raised: * :exc:`~.SchemaError` – if the value doesn't conform to the schema, * :exc:`~.SemanticError` – if the value violates a semantic constraint. * :exc:`~.YangTypeError` – if the value is a scalar of incorrect data type. .. doctest:: >>> inst.validate(ctype=ContentType.all) # no output means OK >>> badinst = bag.put_member('baz', 'ILLEGAL').top() >>> badinst.validate(ctype=ContentType.all) Traceback (most recent call last): ... yangson.exceptions.YangTypeError: {/example-2:bag/baz} invalid-type: expected decimal64 In the following example, member ``baz`` is not allowed because it is a conditional leaf and its **when** constraint evaluates to ``False``. .. doctest:: >>> e2foo6 = e2bag['foo'][0] >>> bad2 = e2foo6.update( ... {'number': '42', 'in-words': 'forty-two'}, raw=True).top() >>> bad2.validate(ctype=ContentType.all) Traceback (most recent call last): ... yangson.exceptions.SchemaError: {/example-2:bag} member-not-allowed: baz .. method:: add_defaults(ctype: ContentType = None) -> InstanceNode Return a new instance node that is a copy of the receiver extended with default values specified the data model. Only default values that are “in use” are added, see sections `7.6.1`_ and `7.7.2`_ in [RFC7950]_. The argument *ctype* restricts the content type of data nodes whose default values will be added. For example, setting it to ``ContentType.config`` means that only default values of configuration nodes will be added. If *ctype* is ``None`` (default), the content type of added defaults will be the same as the content type of the receiver. .. doctest:: >>> wd = inst.add_defaults() >>> wd.value['example-2:bag']['baz'] Decimal('0E-7') >>> wd.value['example-2:bag']['foo'][0]['prime'] False .. automethod:: raw_value() -> RawValue .. doctest:: >>> wd['example-2:bag']['baz'].raw_value() '0.0' .. autoclass:: RootNode(value: Value, schema_node: SchemaNode, timestamp: datetime.datetime) :show-inheritance: .. class:: ObjectMember(key: InstanceName, siblings: \ Dict[InstanceName, Value], value: Value, parinst: \ InstanceNode, schema_node: DataNode, timestamp: \ datetime.datetime) This class represents an instance node that is a member of an object. It is a subclass of :class:`InstanceNode`. The additional constructor arguments *name* and *siblings* provide values for instance variables of the same name. Other arguments of the constructor have the same meaning as in :class:`InstanceNode`. .. rubric:: Instance Attributes .. attribute:: siblings Dictionary of the receiver's siblings (other members of the parent object). .. rubric:: Public Methods .. method:: sibling(name: InstanceName) -> ObjectMember Return the instance node corresponding to sibling member *name*. This method raises :exc:`~.NonexistentSchemaNode` if member *name* is not permitted by the parent's schema, and :exc:`~.NonexistentInstance` if sibling member *name* doesn't exist. .. doctest:: >>> foo.sibling('bar').json_pointer() '/example-2:bag/bar' .. class:: ArrayEntry(key: int, before: List[Value], after: List[Value], value: \ Value, parinst: InstanceNode, schema_node: \ DataNode, timestamp: datetime.datetime) This class is a subclass of :class:`InstanceNode`, and represents an instance node that is an entry of an array, i.e. list or leaf-list. The additional constructor arguments *before* and *after* provide values for instance variables of the same name. Other arguments have the same meaning as in :class:`InstanceNode`. .. rubric:: Instance Attributes .. attribute:: before Entries of the parent array that precede the receiver. .. attribute:: after Entries of the parent array that follow the receiver. .. rubric:: Properties .. attribute:: index The receiver's index within the parent array. .. doctest:: >>> foo6.index 0 >>> foo6.name # inherited from parent 'foo' .. rubric:: Public Methods .. method:: previous() -> ArrayEntry Return an instance node corresponding to the previous entry in the parent array. This method raises :exc:`~.NonexistentInstance` if the receiver is the first entry of the parent array. .. doctest:: >>> foo8.previous().json_pointer() '/example-2:bag/foo/2' >>> foo6.previous() Traceback (most recent call last): ... yangson.exceptions.NonexistentInstance: {/example-2:bag/foo[number="6"]} previous of first .. method:: next() -> ArrayEntry Return an instance node corresponding to the next entry in the parent array. This method raises :exc:`~.NonexistentInstance` if the receiver is the last entry of the parent array. .. doctest:: >>> foo6.next().json_pointer() '/example-2:bag/foo/1' >>> foo8.next() Traceback (most recent call last): ... yangson.exceptions.NonexistentInstance: {/example-2:bag/foo[number="8"]} next of last .. method:: insert_before(value: Union[RawValue, Value], raw: bool \ = False) -> ArrayEntry Insert a new entry before the receiver and return an instance node corresponding to the new entry. The *value* argument specifies the value of the new entry, and the *raw* flag has to be set to ``True`` if *value* is a :term:`raw value`. .. doctest:: >>> foo4 = foo8.insert_before({'number': '4', 'in-words': 'four'}, raw=True) >>> [en['number'] for en in foo4.up().value] [6, 3, 7, 4, 8] .. method:: insert_after(value: Union[RawValue, Value], raw: bool \ = False) -> ArrayEntry Insert a new entry after the receiver and return an instance node corresponding to the new entry. The *value* argument specifies the value of the new entry, and the *raw* flag has to be set to ``True`` if *value* is a :term:`raw value`. .. doctest:: >>> foo5 = foo4.insert_after({'number': '5', 'in-words': 'five'}, raw=True) >>> [en['number'] for en in foo5.up().value] [6, 3, 7, 4, 5, 8] .. autoclass:: InstanceRoute :show-inheritance: Instances of this class can be conveniently created by using one of the methods :meth:`~.DataModel.parse_resource_id` and :meth:`~.DataModel.parse_instance_id` in the :class:`~.datamodel.DataModel` class. .. rubric:: Public Methods .. automethod:: __str__ .. doctest:: >>> str(irt) '/example-2:bag/foo[number="3"]/in-words' >>> str(irt2) '/example-2:bag/baz' .. _keyword arguments: https://docs.python.org/3/tutorial/controlflow.html#keyword-arguments .. _4: https://rfc-editor.org/rfc/rfc7951.html#section-4 .. _6.1: https://rfc-editor.org/rfc/rfc7951.html#section-6.1 .. _7.5.8: https://rfc-editor.org/rfc/rfc7950.html#section-7.5.8 .. _7.6.1: https://rfc-editor.org/rfc/rfc7950.html#section-7.6.1 .. _7.6.7: https://rfc-editor.org/rfc/rfc7950.html#section-7.6.7 .. _7.7.2: https://rfc-editor.org/rfc/rfc7950.html#section-7.7.2 .. _7.7.7: https://rfc-editor.org/rfc/rfc7950.html#section-7.7.7 .. _7.7.9: https://rfc-editor.org/rfc/rfc7950.html#section-7.7.9 .. _7.8.6: https://rfc-editor.org/rfc/rfc7950.html#section-7.8.6 .. _8.3: https://rfc-editor.org/rfc/rfc6241.html#section-8.3 .. _9: https://rfc-editor.org/rfc/rfc7950.html#section-9