SoftSignalBackend wrapping arbitrary callables

docs/conf.py

@@ -150,6 +150,8 @@ def setup(app: application.Sphinx):
     "ophyd_async.core._command.MockExecuteCallback",
     "ophyd_async.core._protocol.C",
     "ophyd_async.core._signal_backend.SignalDatatypeV",
+    "ophyd_async.core._soft_signal_backend.Getter",
+    "ophyd_async.core._soft_signal_backend.Setter",
     "ophyd_async.core._status.AsyncStatusBase",
     "ophyd_async.core._utils.P",
     "ophyd_async.core._utils.P.args",

docs/explanations/decisions/0019-soft-signals-backed-by-arbitrary-callables.md

@@ -0,0 +1,38 @@
+# 19. SoftSignalBackend Wrapping Arbitrary Callables
+
+## **Status**
+Accepted
+
+
+## **Context**
+Users working outside of EPICS/Tango ecosystems such as those wrapping third-party Python APIs, calling analysis scripts, or interfacing with devices with their own Python drivers, currently have no supported path to create signals without writing a full custom `SignalBackend`. This was identified as a friction point for smaller lab-based groups during the Bluesky community workshop.
+
+To address this, `SoftSignalBackend` was extended to support arbitrary callables for getting, setting, and polling values. This allows users to integrate external systems without implementing a full backend, reducing boilerplate and improving usability.
+
+## **Decision**
+
+### **Extend `SoftSignalBackend` with Callable Support**
+`SoftSignalBackend` was augmented with three new **keyword-only** parameters:
+1. **`getter`**: A callable (`Callable[[], T | Awaitable[T]]`) invoked during `get_value()` and `get_reading()` to fetch the current value from an external source. If `poll_period` is set, the `getter` is called periodically while a subscription is active.
+2. **`setter`**: A callable (`Callable[[SignalDatatypeT], SignalDatatypeT | None | Awaitable[SignalDatatypeT | None]]`) invoked during `put()`. It may return a `SignalDatatypeT`; if it returns `None`, the `getter` (if configured) is called immediately to refresh the cache.
+3. **`poll_period`**: A float representing the interval (in seconds) at which the `getter` is polled while a subscription is active. Requires `getter` to be set.
+
+### **Design Choices**
+- All three parameters are **optional**. If none are provided, behavior remains identical to the existing `SoftSignalBackend`.
+- The internal `self._reading` store remains the **single source of truth**. The `getter` updates this store rather than bypassing it, preserving coherence for subscriptions and cached reads.
+- The `put` method accepts `SignalDatatypeT` (the same type as the signal's stored value) to maintain type safety and consistency.
+- Polling tasks are used for subscriptions, starting in `set_callback` and canceling when subscriptions end.
+- `get_setpoint()` **does not invoke the `getter`**; it returns the last value written to the `setter` or the initial value of.
+
+### **Factory Function Updates**
+The convenience functions `soft_signal_rw` and `soft_signal_r_and_setter` were updated to accept `getter`, and `poll_period` arguments. `soft_signal_rw` additionally accepts a `setter` argument. These additional arguments are passed to `SoftSignalBackend`.
+
+## **Consequences**
+**Improved Usability**:
+   Non-EPICS/Tango users can now easily integrate external systems (e.g., Python APIs, scripts) without writing full backends.
+
+**No Breaking Changes**:
+   Existing code continues to work unchanged since all new parameters are optional.
+
+**Consistent Behavior**:
+   Polling, caching, and subscriptions align with EPICS/Tango backend patterns.

docs/explanations/decisions/0020-how-to-use-soft-signals.md

@@ -0,0 +1,73 @@
+# 20. How to use soft signals
+
+The introduction of callable-backed `SoftSignalBackend` enables users to integrate non-EPICS/Tango systems (e.g., Python APIs, scripts, or custom hardware drivers) into ophyd-async without writing full `SignalBackend` implementations. Below are idiomatic patterns for common scenarios, balancing simplicity and type safety.
+
+## Case A: Single-Value Read/Write with Matching Types
+
+**Use Case**: A callable with a single argument where the input and output types match the signal's `SignalDatatypeT` (e.g., a motor position setter/getter).
+
+**Approach**:
+```python
+def read_position() -> float:
+    # returns current position
+    ...
+def move_to(position: float) -> float:
+    # Move hardware and return actual position
+    ...
+motor_position = soft_signal_rw(
+    float,
+    setter=move_to,
+    getter=read_position,
+)
+```
+**Rationale**:
+- Directly wrap the callable in a `SoftSignalBackend`-backed signal.
+- Avoids the need for separate `Command` + `Signal` pairs when types align.
+- Preserves type hints and integrates seamlessly with scans.
+
+## Case B: Mismatched setter and getter types or multiple input types
+
+**Use Case**: A callable where the input type differs from the output (e.g., sending a config object but receiving a string status).
+
+```python
+status = soft_signal_rw(str)
+def configure_subsystem(*args, **kwargs) -> None:
+    # Apply config...
+    await status.set("configured")
+config_cmd = soft_command(configure_subsystem)
+await config_cmd.execute(...)
+current_status = await status.read()
+```
+**Rationale**:
+- Use a **`Command`** to handle the mismatched input/output types.
+- Store the result in a separate `Signal` (here, `status`) for readability in plans.
+- Ensures type safety: `Command` input (`MotorConfig`) and `Signal` output (`float`) remain distinct.
+
+## Case C: Complex returns or multiple outputs
+
+**Use Case**: A callable returning structured data (e.g., a diagnostic function yielding many metrics).
+
+**Approach**:
+```python
+# Split outputs into individual signals
+temp_signal = soft_signal_rw(float, getter=lambda: run_diagnostics()["temperature"])
+pressure_signal = soft_signal_rw(float, getter=lambda: run_diagnostics()["pressure"])
+async def run_diagnostics() -> None:
+    temp, pressure = _diagnostics()
+    await temp_signal.set(temp)
+    await pressure_signal.set(pressure)
+diagnostics_cmd = soft_command(run_diagnostics)
+result = await diagnostics_cmd.execute()
+temp = await temp_signal.read()
+pressure = await pressure_signal.read()
+```
+**Rationale**:
+- **Prefer splitting outputs** into discrete `Signal`s if they're independently useful.
+- For ad-hoc use, a **`Command`** suffices, with manual extraction of results.
+- Maintains separation of concerns: signals represent *state*, commands represent *actions*.
+
+**Key Takeaways**:
+1. **Prioritize `SoftSignalBackend` with callables** for simple, type-aligned read/write operations (Case A).
+2. **Combine `Command` + `Signal`** when types diverge or actions yield secondary results (Cases B/C).
+3. **Avoid overloading signals**: If a callable performs an action *and* returns data, model the action as a `Command` and the data as one or more `Signal`s.
+4. **Polling**: Use `poll_period` in `SoftSignalBackend` for live updates (e.g., sensor readings), but ensure `getter` is lightweight.

src/ophyd_async/core/_signal.py

@@ -25,7 +25,7 @@
 from ._mock_signal_backend import MockSignalBackend
 from ._protocol import AsyncReadable, AsyncStageable
 from ._signal_backend import SignalBackend, SignalDatatypeT, SignalDatatypeV
-from ._soft_signal_backend import SoftSignalBackend
+from ._soft_signal_backend import Getter, Setter, SoftSignalBackend
 from ._status import AsyncStatus
 from ._utils import (
     CALCULATE_TIMEOUT,
@@ -353,6 +353,10 @@ def soft_signal_rw(
     name: str = "",
     units: str | None = None,
     precision: int | None = None,
+    *,
+    getter: Getter[SignalDatatypeT] | None = None,
+    setter: Setter[Any] | None = None,
+    poll_period: float | None = None,
 ) -> SignalRW[SignalDatatypeT]:
     """Create a read-writable Signal with a [](#SoftSignalBackend).
 
@@ -363,8 +367,26 @@ def soft_signal_rw(
     :param name: The name of the signal.
     :param units: The units of the signal.
     :param precision: The precision of the signal.
+    :param getter:
+        Optional callable returning the current device value, called on
+        get_value/get_reading and periodically if poll_period is set.
+    :param setter:
+        Optional callable performing the set action. May return the settled
+        value; if it returns None and a getter is configured, the getter is
+        called to refresh the cache.
+    :param poll_period:
+        How often (seconds) to call the getter while a subscription is active.
+        Requires getter to be set.
     """
-    backend = SoftSignalBackend(datatype, initial_value, units, precision)
+    backend = SoftSignalBackend(
+        datatype,
+        initial_value,
+        units,
+        precision,
+        getter=getter,
+        setter=setter,
+        poll_period=poll_period,
+    )
     signal = SignalRW(backend=backend, name=name)
     return signal
 
@@ -375,6 +397,9 @@ def soft_signal_r_and_setter(
     name: str = "",
     units: str | None = None,
     precision: int | None = None,
+    *,
+    getter: Getter[SignalDatatypeT] | None = None,
+    poll_period: float | None = None,
 ) -> tuple[SignalR[SignalDatatypeT], Callable[[SignalDatatypeT], None]]:
     """Create a read-only Signal with a [](#SoftSignalBackend).
 
@@ -386,9 +411,22 @@ def soft_signal_r_and_setter(
     :param name: The name of the signal.
     :param units: The units of the signal.
     :param precision: The precision of the signal.
+    :param getter:
+        Optional callable returning the current device value, called on
+        get_value/get_reading and periodically if poll_period is set.
+    :param poll_period:
+        How often (seconds) to call the getter while a subscription is active.
+        Requires getter to be set.
     :return: A tuple of the created SignalR and a callable to set its value.
     """
-    backend = SoftSignalBackend(datatype, initial_value, units, precision)
+    backend = SoftSignalBackend(
+        datatype,
+        initial_value,
+        units,
+        precision,
+        getter=getter,
+        poll_period=poll_period,
+    )
     signal = SignalR(backend=backend, name=name)
     return (signal, backend.set_value)
 

src/ophyd_async/core/_soft_signal_backend.py

@@ -1,15 +1,17 @@
 from __future__ import annotations
 
+import asyncio
 import time
 import typing
 from abc import abstractmethod
-from collections.abc import Sequence
+from collections.abc import Awaitable, Callable, Sequence
 from dataclasses import dataclass
 from functools import lru_cache
 from typing import Any, Generic, get_args
 
 import numpy as np
 from bluesky.protocols import Reading
+from bluesky.utils import maybe_await
 from event_model import DataKey
 
 from ._datatypes import Table
@@ -30,8 +32,6 @@
 
 class SoftConverter(Generic[SignalDatatypeT]):
     # This is Any -> SignalDatatypeT because we support coercing
-    # value types to SignalDatatype to allow people to do things like
-    # SignalRW[Enum].set("enum value")
     @abstractmethod
     def write_value(self, value: Any) -> SignalDatatypeT: ...
 
@@ -114,6 +114,14 @@ def make_converter(datatype: type[SignalDatatype]) -> SoftConverter:
     raise TypeError(f"Can't make converter for {datatype}")
 
 
+Setter = (
+    Callable[[SignalDatatypeT | None], SignalDatatypeT | None]
+    | Callable[[SignalDatatypeT | None], Awaitable[SignalDatatypeT | None]]
+    | None
+)
+Getter = Callable[[], SignalDatatypeT | Awaitable[SignalDatatypeT]]
+
+
 class SoftSignalBackend(SignalBackend[SignalDatatypeT]):
     """An backend to a soft Signal, for test signals see [](#MockSignalBackend).
 
@@ -124,6 +132,16 @@ class SoftSignalBackend(SignalBackend[SignalDatatypeT]):
     :param units: The units for numeric datatypes.
     :param precision:
         The number of digits after the decimal place to display for a float datatype.
+    :param getter:
+        Optional callable returning the current device value, called on
+        get_value/get_reading and periodically if poll_period is set.
+    :param setter:
+        Optional callable performing the set action. May return the settled
+        value; if it returns None and a getter is configured, the getter is
+        called to refresh the cache.
+    :param poll_period:
+        How often (seconds) to call the getter while a subscription is active.
+        Requires getter to be set.
     """
 
     def __init__(
@@ -132,22 +150,48 @@ def __init__(
         initial_value: SignalDatatypeT | None = None,
         units: str | None = None,
         precision: int | None = None,
+        *,
+        getter: Getter[SignalDatatypeT] | None = None,
+        setter: Setter[SignalDatatypeT] | None = None,
+        poll_period: float | None = None,
     ):
-        # Create the right converter for the datatype
+        if poll_period is not None and getter is None:
+            raise ValueError("poll_period requires a getter to be set")
         self.converter = make_converter(datatype or float)
-        # Add the extra static metadata to the dictionary
         self.metadata = make_metadata(datatype, units, precision)
-        # Create and set the initial value
         self.initial_value = self.converter.write_value(initial_value)
         self.reading: Reading[SignalDatatypeT]
         self.callback: Callback[Reading[SignalDatatypeT]] | None = None
+        self._getter = getter
+        self._setter = setter
+        self._poll_period = poll_period
+        self._poll_task: asyncio.Task | None = None
         self.set_value(self.initial_value)
+        self._setpoint = self.initial_value
         super().__init__(datatype)
 
+    async def _update_value_from_getter(self) -> SignalDatatypeT | None:
+        if self._getter is None:
+            return
+        result = await maybe_await(self._getter())
+        self.set_value(result)
+
+    async def _poll(self) -> None:
+        if self._poll_period is None:
+            raise RuntimeError("No poll_period configured")
+        while True:
+            await asyncio.sleep(self._poll_period)
+            try:
+                await self._update_value_from_getter()
+            except Exception:
+                continue
+
     def set_value(self, value: SignalDatatypeT):
         """Set the current value, alarm and timestamp."""
+        setp = self.converter.write_value(value)
+        self._setpoint = setp
         self.reading = Reading(
-            value=self.converter.write_value(value),
+            value=setp,
             timestamp=time.time(),
             alarm_severity=0,
         )
@@ -160,28 +204,44 @@ def source(self, name: str, read: bool) -> str:
     async def connect(self, timeout: float):
         pass
 
-    async def put(self, value: SignalDatatypeT | None) -> None:
+    async def put(self, value: Any) -> None:
         write_value = self.initial_value if value is None else value
-        self.set_value(write_value)
+        if self._setter is not None:
+            returned_by_setter = await maybe_await(self._setter(write_value))
+            if returned_by_setter is not None:
+                self.set_value(returned_by_setter)
+            elif self._getter is not None:
+                await self._update_value_from_getter()
+            else:
+                self.set_value(write_value)
+        else:
+            self.set_value(write_value)
 
     async def get_datakey(self, source: str) -> DataKey:
         return make_datakey(
             self.datatype or float, self.reading["value"], source, self.metadata
         )
 
     async def get_reading(self) -> Reading[SignalDatatypeT]:
+        await self._update_value_from_getter()
         return self.reading
 
     async def get_value(self) -> SignalDatatypeT:
+        await self._update_value_from_getter()
         return self.reading["value"]
 
     async def get_setpoint(self) -> SignalDatatypeT:
-        # For a soft signal, the setpoint and readback values are the same.
-        return self.reading["value"]
+        return self._setpoint
 
     def set_callback(self, callback: Callback[Reading[SignalDatatypeT]] | None) -> None:
         if callback and self.callback:
             raise RuntimeError("Cannot set a callback when one is already set")
         if callback:
             callback(self.reading)
+            if self._poll_period is not None:
+                self._poll_task = asyncio.create_task(self._poll())
+        else:
+            if self._poll_task is not None:
+                self._poll_task.cancel()
+                self._poll_task = None
         self.callback = callback

tests/system_tests_tango/test_tango_command.py

@@ -89,7 +89,7 @@ class TangoEverythingOphydDeviceTriggerableAnnotation(TangoDevice, StandardReada
 
 @pytest.fixture()
 async def everything_device(everything_device_trl):
-    return TangoEverythingOphydDevice(everything_device_trl)
+    return TangoEverythingOphydDevice(everything_device_trl, name="everything_device")
 
 
 @pytest.mark.asyncio