Handle an optional symengine and sympy in qpy (#13917)

* Document that symengine is optional for qpy.load

This commit changes the API for qpy.load() to make it explicit that
symengine may not be installed in the future but is a requirement for
deserializing certain payloads that were generated using symengine.
We're expecting to drop symengine as a requirement with #13278 which
should improve Qiskit's installation story and also will facilitate
adding a C api for circuits. But the qpy v10, v11, and v13 have a hard
requirement on symengine (specifically 0.11.0 and 0.13.0) to be able to
deserialize `ParameterExpression` if the symbolic encoding was set to
use it. Since the function needs to support being able to deserialize
these payloads symengine will need to be installed. This commit adds the
document and release note to indicate this change in behavior.

* Raise QPY compatibility version to remove symengine and sympy dependency

This commit raises the qpy compatibility version to QPY format version
13. As the larger PR is planning for a world without symengine installed
by default this becomes a problem for the generation side too. This
becomes a blocker for using QPY < 13 in the future so this commit opts
to just raise the minimum verison to get ahead of any potential issues.

* Add HAS_SYMENGINE check on load function

* Correct logic for ucr*_dg gates name changing

* Add feature string to optional decorator

* Prepare for an optional sympy

In addition to making symengine optional #13278 should also enable us to
make sympy optional. It would only be potentially needed for loading QPY
payloads that were generated using sympy encoding, but also some
visualization functions. This commit lays the groundwork for removing it
as a dependency by outlining the API changes for the functions that will
optionally depend on sympy and raise an exception if it's not installed.
It won't be possible to trigger the exception with sympy in the
requirements list, but when we do remove it the API is prepared for it.

* Fix release note categorization

* Add missing optional on template substitution

qiskit/circuit/parameterexpression.py

@@ -26,6 +26,7 @@
 import symengine
 
 from qiskit.circuit.exceptions import CircuitError
+from qiskit.utils.optionals import HAS_SYMPY
 
 # This type is redefined at the bottom to insert the full reference to "ParameterExpression", so it
 # can safely be used by runtime type-checkers like Sphinx.  Mypy does not need this because it
@@ -117,6 +118,8 @@ def __init__(self, symbol_map: dict, expr, *, _qpy_replay=None):
 
         Not intended to be called directly, but to be instantiated via operations
         on other :class:`Parameter` or :class:`ParameterExpression` objects.
+        The constructor of this object is **not** a public interface and should not
+        ever be used directly.
 
         Args:
             symbol_map (Dict[Parameter, [ParameterExpression, float, or int]]):
@@ -672,6 +675,7 @@ def numeric(self) -> int | float | complex:
             return out.real if out.imag == 0.0 else out
         return float(real_expr)
 
+    @HAS_SYMPY.require_in_call
     def sympify(self):
         """Return symbolic expression as a raw Sympy or Symengine object.
 

qiskit/qpy/binary_io/circuits.py

@@ -40,7 +40,7 @@
 from qiskit.circuit.instruction import Instruction
 from qiskit.circuit.quantumcircuit import QuantumCircuit
 from qiskit.circuit.quantumregister import QuantumRegister, Qubit
-from qiskit.qpy import common, formats, type_keys, exceptions
+from qiskit.qpy import common, formats, type_keys
 from qiskit.qpy.binary_io import value, schedules
 from qiskit.quantum_info.operators import SparsePauliOp, Clifford
 from qiskit.synthesis import evolution as evo_synth
@@ -753,13 +753,15 @@ def _write_instruction(
         or isinstance(instruction.operation, library.BlueprintCircuit)
     ):
         gate_class_name = instruction.operation.name
-        if version >= 11:
-            # Assign a uuid to each instance of a custom operation
+        # Assign a uuid to each instance of a custom operation
+        if instruction.operation.name not in {"ucrx_dg", "ucry_dg", "ucrz_dg"}:
             gate_class_name = f"{gate_class_name}_{uuid.uuid4().hex}"
-        # ucr*_dg gates can have different numbers of parameters,
-        # the uuid is appended to avoid storing a single definition
-        # in circuits with multiple ucr*_dg gates.
-        elif instruction.operation.name in {"ucrx_dg", "ucry_dg", "ucrz_dg"}:
+        else:
+            # ucr*_dg gates can have different numbers of parameters,
+            # the uuid is appended to avoid storing a single definition
+            # in circuits with multiple ucr*_dg gates. For legacy reasons
+            # the uuid is stored in a different format as this was done
+            # prior to QPY 11.
             gate_class_name = f"{gate_class_name}_{uuid.uuid4()}"
 
         custom_operations[gate_class_name] = instruction.operation
@@ -1217,48 +1219,25 @@ def write_circuit(
     num_registers = num_qregs + num_cregs
 
     # Write circuit header
-    if version >= 12:
-        header_raw = formats.CIRCUIT_HEADER_V12(
-            name_size=len(circuit_name),
-            global_phase_type=global_phase_type,
-            global_phase_size=len(global_phase_data),
-            num_qubits=circuit.num_qubits,
-            num_clbits=circuit.num_clbits,
-            metadata_size=metadata_size,
-            num_registers=num_registers,
-            num_instructions=num_instructions,
-            num_vars=circuit.num_vars,
-        )
-        header = struct.pack(formats.CIRCUIT_HEADER_V12_PACK, *header_raw)
-        file_obj.write(header)
-        file_obj.write(circuit_name)
-        file_obj.write(global_phase_data)
-        file_obj.write(metadata_raw)
-        # Write header payload
-        file_obj.write(registers_raw)
-        standalone_var_indices = value.write_standalone_vars(file_obj, circuit, version)
-    else:
-        if circuit.num_vars:
-            raise exceptions.UnsupportedFeatureForVersion(
-                "circuits containing realtime variables", required=12, target=version
-            )
-        header_raw = formats.CIRCUIT_HEADER_V2(
-            name_size=len(circuit_name),
-            global_phase_type=global_phase_type,
-            global_phase_size=len(global_phase_data),
-            num_qubits=circuit.num_qubits,
-            num_clbits=circuit.num_clbits,
-            metadata_size=metadata_size,
-            num_registers=num_registers,
-            num_instructions=num_instructions,
-        )
-        header = struct.pack(formats.CIRCUIT_HEADER_V2_PACK, *header_raw)
-        file_obj.write(header)
-        file_obj.write(circuit_name)
-        file_obj.write(global_phase_data)
-        file_obj.write(metadata_raw)
-        file_obj.write(registers_raw)
-        standalone_var_indices = {}
+    header_raw = formats.CIRCUIT_HEADER_V12(
+        name_size=len(circuit_name),
+        global_phase_type=global_phase_type,
+        global_phase_size=len(global_phase_data),
+        num_qubits=circuit.num_qubits,
+        num_clbits=circuit.num_clbits,
+        metadata_size=metadata_size,
+        num_registers=num_registers,
+        num_instructions=num_instructions,
+        num_vars=circuit.num_vars,
+    )
+    header = struct.pack(formats.CIRCUIT_HEADER_V12_PACK, *header_raw)
+    file_obj.write(header)
+    file_obj.write(circuit_name)
+    file_obj.write(global_phase_data)
+    file_obj.write(metadata_raw)
+    # Write header payload
+    file_obj.write(registers_raw)
+    standalone_var_indices = value.write_standalone_vars(file_obj, circuit, version)
 
     instruction_buffer = io.BytesIO()
     custom_operations = {}

qiskit/qpy/binary_io/value.py

@@ -165,17 +165,9 @@ def _write_parameter_expression_v13(file_obj, obj, version):
 
 def _write_parameter_expression(file_obj, obj, use_symengine, *, version):
     extra_symbols = None
-    if version < 13:
-        if use_symengine:
-            expr_bytes = obj._symbol_expr.__reduce__()[1][0]
-        else:
-            from sympy import srepr, sympify
-
-            expr_bytes = srepr(sympify(obj._symbol_expr)).encode(common.ENCODE)
-    else:
-        with io.BytesIO() as buf:
-            extra_symbols = _write_parameter_expression_v13(buf, obj, version)
-            expr_bytes = buf.getvalue()
+    with io.BytesIO() as buf:
+        extra_symbols = _write_parameter_expression_v13(buf, obj, version)
+        expr_bytes = buf.getvalue()
     symbol_table_len = len(obj._parameter_symbols)
     if extra_symbols:
         symbol_table_len += 2 * len(extra_symbols)

qiskit/qpy/common.py

@@ -18,15 +18,12 @@
 import io
 import struct
 
-import symengine
-from symengine.lib.symengine_wrapper import (  # pylint: disable = no-name-in-module
-    load_basic,
-)
+from qiskit.utils.optionals import HAS_SYMENGINE
 
 from qiskit.qpy import formats, exceptions
 
 QPY_VERSION = 14
-QPY_COMPATIBILITY_VERSION = 10
+QPY_COMPATIBILITY_VERSION = 13
 ENCODE = "utf8"
 
 
@@ -311,13 +308,19 @@ def mapping_from_binary(binary_data, deserializer, **kwargs):
     return mapping
 
 
-def load_symengine_payload(payload: bytes) -> symengine.Expr:
+@HAS_SYMENGINE.require_in_call("QPY versions 10 through 12 with symengine parameter serialization")
+def load_symengine_payload(payload: bytes):
     """Load a symengine expression from it's serialized cereal payload."""
     # This is a horrible hack to workaround the symengine version checking
     # it's deserialization does. There were no changes to the serialization
     # format between 0.11 and 0.13 but the deserializer checks that it can't
     # load across a major or minor version boundary. This works around it
     # by just lying about the generating version.
+    import symengine
+    from symengine.lib.symengine_wrapper import (  # pylint: disable = no-name-in-module
+        load_basic,
+    )
+
     symengine_version = symengine.__version__.split(".")
     major = payload[2]
     minor = payload[3]

qiskit/qpy/interface.py

@@ -76,7 +76,7 @@ def dump(
     programs: Union[List[QPY_SUPPORTED_TYPES], QPY_SUPPORTED_TYPES],
     file_obj: BinaryIO,
     metadata_serializer: Optional[Type[JSONEncoder]] = None,
-    use_symengine: bool = True,
+    use_symengine: bool = False,
     version: int = common.QPY_VERSION,
 ):
     """Write QPY binary data to a file
@@ -125,11 +125,9 @@ def dump(
         metadata_serializer: An optional JSONEncoder class that
             will be passed the ``.metadata`` attribute for each program in ``programs`` and will be
             used as the ``cls`` kwarg on the `json.dump()`` call to JSON serialize that dictionary.
-        use_symengine: If True, all objects containing symbolic expressions will be serialized
-            using symengine's native mechanism. This is a faster serialization alternative,
-            but not supported in all platforms. This flag only has an effect if the emitted QPY format
-            version is 10, 11, or 12. For QPY format version >= 13 (which is the default starting in
-            Qiskit 1.3.0) this flag is no longer used.
+        use_symengine: This flag is no longer used by QPY versions supported by this function and
+            will have no impact on the generated QPY payload except to set a field in a QPY v13 file
+            header which is unused.
         version: The QPY format version to emit. By default this defaults to
             the latest supported format of :attr:`~.qpy.QPY_VERSION`, however for
             compatibility reasons if you need to load the generated QPY payload with an older
@@ -252,9 +250,13 @@ def load(
         A list is always returned, even if there is only 1 program in the QPY data.
 
     Raises:
-        QiskitError: if ``file_obj`` is not a valid QPY file.
+        QiskitError: if ``file_obj`` is not a valid QPY file
+        TypeError: When invalid data type is loaded.
+        MissingOptionalLibraryError: If the ``symengine`` engine library is
+            not installed when loading a QPY version 10, 11, or 12 payload
+            that is using symengine symbolic encoding and contains
+            :class:`.ParameterExpression` instances.
         QpyError: if known but unsupported data type is loaded.
-        TypeError: if invalid data type is loaded.
     """
 
     # identify file header version

qiskit/quantum_info/operators/operator.py

@@ -206,6 +206,8 @@ def draw(self, output=None, **drawer_args):
 
         Raises:
             ValueError: when an invalid output method is selected.
+            MissingOptionalLibrary: If SymPy isn't installed and ``'latex'`` or
+                ``'latex_source'`` is selected for ``output``.
 
         """
         # pylint: disable=cyclic-import

qiskit/transpiler/passes/optimization/template_matching/template_substitution.py

@@ -443,6 +443,7 @@ def run_dag_opt(self):
         self.dag_dep_optimized = dag_dep_opt
         self.dag_optimized = dagdependency_to_dag(dag_dep_opt)
 
+    @_optionals.HAS_SYMPY.require_in_call("Bind parameters in templates")
     def _attempt_bind(self, template_sublist, circuit_sublist):
         """
         Copies the template and attempts to bind any parameters,

qiskit/utils/optionals.py

@@ -177,6 +177,13 @@
     special methods from Symengine to accelerate its handling of
     :class:`~.circuit.Parameter`\\ s if available.
 
+.. py:data:: HAS_SYMPY
+
+    `SymPy <https://www.sympy.org/en/index.html>`__ is Python library for symbolic mathematics.
+    Sympy was historically used for the implementation of the :class:`.ParameterExpression`
+    class but isn't any longer. However it is needed for some legacy functionality that uses
+    :meth:`.ParameterExpression.sympify`. It is also used in some visualization functions.
+
 .. py:data:: HAS_TESTTOOLS
 
     Qiskit's test suite has more advanced functionality available if the optional
@@ -322,7 +329,8 @@
     install="pip install scikit-quant",
 )
 HAS_SQSNOBFIT = _LazyImportTester("SQSnobFit", install="pip install SQSnobFit")
-HAS_SYMENGINE = _LazyImportTester("symengine", install="pip install symengine")
+HAS_SYMENGINE = _LazyImportTester("symengine", install="pip install symengine<0.14")
+HAS_SYMPY = _LazyImportTester("sympy", install="pip install sympy")
 HAS_TESTTOOLS = _LazyImportTester("testtools", install="pip install testtools")
 HAS_TWEEDLEDUM = _LazyImportTester("tweedledum", install="pip install tweedledum")
 HAS_Z3 = _LazyImportTester("z3", install="pip install z3-solver")

qiskit/utils/units.py

@@ -13,9 +13,12 @@
 """SI unit utilities"""
 from __future__ import annotations
 
+import typing
+
 import numpy as np
 
-from qiskit.circuit.parameterexpression import ParameterExpression
+if typing.TYPE_CHECKING:
+    from qiskit.circuit.parameterexpression import ParameterExpression
 
 
 def apply_prefix(value: float | ParameterExpression, unit: str) -> float | ParameterExpression:

qiskit/visualization/array.py

@@ -16,8 +16,10 @@
 import numpy as np
 
 from qiskit.exceptions import MissingOptionalLibraryError
+from qiskit.utils.optionals import HAS_SYMPY
 
 
+@HAS_SYMPY.require_in_call("Create a latex representation of a ket expression")
 def _num_to_latex(raw_value, decimals=15, first_term=True, coefficient=False):
     """Convert a complex number to latex code suitable for a ket expression
 
@@ -83,6 +85,7 @@ def _matrix_to_latex(matrix, decimals=10, prefix="", max_size=(8, 8)):
 
     Raises:
         ValueError: If minimum value in max_size < 3
+        MissingOptionalLibraryError: If sympy is not installed
     """
     if min(max_size) < 3:
         raise ValueError("""Smallest value in max_size must be greater than or equal to 3""")
@@ -171,7 +174,7 @@ def array_to_latex(array, precision=10, prefix="", source=False, max_size=8):
         TypeError: If array can not be interpreted as a numerical numpy array.
         ValueError: If the dimension of array is not 1 or 2.
         MissingOptionalLibraryError: If ``source`` is ``False`` and ``IPython.display.Latex`` cannot be
-                     imported.
+                     imported. Or if sympy is not installed.
     """
     try:
         array = np.asarray(array)

qiskit/visualization/state_visualization.py

@@ -1266,6 +1266,9 @@ def state_to_latex(
 
     Returns:
         Latex representation of the state
+        MissingOptionalLibrary: If SymPy isn't installed and ``'latex'`` or
+            ``'latex_source'`` is selected for ``output``.
+
     """
     if dims is None:  # show dims if state is not only qubits
         if set(state.dims()) == {2}:
@@ -1438,6 +1441,9 @@ def state_drawer(state, output=None, **drawer_args):
 
     Raises:
         MissingOptionalLibraryError: when `output` is `latex` and IPython is not installed.
+            or if SymPy isn't installed and ``'latex'`` or ``'latex_source'`` is selected for
+            ``output``.
+
         ValueError: when `output` is not a valid selection.
     """
     config = user_config.get_config()

releasenotes/notes/qpy-missing-symengine-ee8265348c992ef3.yaml

@@ -0,0 +1,47 @@
+---
+upgrade_qpy:
+  - |
+    The :func:`.qpy.load` function can now raise a
+    :class:`.MissingOptionalLibrary` exception if a QPY v10, v11, or v12
+    payload is passed in that is using symengine symbolic expressions
+    and symengine is not installed. Or if sympy is not installed for any
+    other QPY payload < v13. In the Qiskit 1.x releases symengine and sympy
+    were always guaranteed to be installed, but starting in 2.x this is no
+    longer a hard requirement and may only be needed if you're deserializing a QPY
+    file that was generated using symengine. Parsing these QPY payloads
+    requires symengine (0.11.0 or 0.13.0) as it's usage is baked into the
+    format specification for QPY v10, v11, and v12 so if the payload requires
+    it there is no option but to install a compatible version of symengine.
+    Similarly, sympy was was used for :class:`.ParameterExpression` encoding
+    for all QPY versions from 1 through 12.
+  - |
+    The minimum QPY compatibility version, :attr:`.QPY_COMPATIBILITY_VERSION`,
+    has been raised to 13 from 10 in the 1.x release. This version controls
+    the minimum version of QPY that can be emitted by the :func:`.qpy.dump`
+    function. This means :func:`.qpy.dump` can only emit QPY v13 and v14
+    in this release. QPY v13 is still compatible with Qiskit 1.3.x and 1.4.x
+    which means payloads can be generated in Qiskit 2.x that can be loaded
+    with the Qiskit 1.x release series still.
+
+    This change was necessary as QPY versions 10 through 12 requires either
+    the sympy and symengine libraries to generate a serialization for
+    :class:`.ParameterExpression` objects, but in Qiskit 2.x neither library
+    is required for the :class:`.ParameterExpression` object.
+upgrade_circuits:
+  - |
+    The :meth:`.ParameterExpression.sympify` method can now raise a
+    :class:`.MissingOptionalLibrary` exception if ``sympy`` is not installed.
+    In the Qiskit 1.x releases sympy was always guaranteed to be installed,
+    but starting in 2.x this is no longer a hard requirement and may only be
+    needed if you are using this method. As this functionality explicitly
+    requires ``sympy`` you will need to ensure you have ``sympy`` installed
+    to use the method.
+upgrade_visualization:
+  - |
+    The :func:`.array_to_latex` and :meth:`.Operator.draw` methods can now
+    raise a :class:`.MissingOptionalLibrary` exception if the ``sympy``
+    library is not installed. In the Qiskit 1.x releases symengine and sympy
+    were always guaranteed to be installed, but starting in 2.x this is no
+    longer a hard requirement. The latex visualization for a matrix relies
+    on the sympy library, so if you're using this functionality you should
+    ensure that you have sympy installed.