AVT (Allied Vision) camera: GenTL BusyException (-1022) and unrecoverable Harvester/ImageAcquirer state after camera power-cycle

[keale]

Summary

When using AVT (Allied Vision) GigE Vision cameras with Harvesters + the official AVT GenTL Producer (TL AVT), the system enters an unrecoverable “busy” state after the camera is power-cycled during acquisition.

Once this happens, all further calls to Harvester.create() or ImageAcquirer.start() fail with:

GenTL exception: Operation cannot be performed. Responsible module is busy. ID: -1022 (BusyException)

The only way to recover the GenTL producer is to restart the entire Python process.

This behavior does NOT occur with other camera brands (tested: Basler), and also not if another GenTL provider e.g. Balluf used. It seems specific to AVT’s TL library and how Harvesters interacts with it.

Environment

• Windows 11
• Harvesters versions: 1.4.3
• AVT Vimba VimbaGigETL.cti V6.0
• Python 3.11.9
• GigE Vision camera: MANTA G-series
• Application scenario:
  • Continuous acquisition loop using fetch()
  • Camera is physically power-cycled (power OFF → ON)
  • Alternative: disconnect network

Observed behavior

1. Power-cycle during acquisition leads to BusyException (-1022)

If the AVT camera is switched OFF while an acquisition loop is running:
The next fetch() call raises a Vimba-specific error, so fare so good.
After attempting to stop acquisition and destroy the ImageAcquirer:
Any attempt to create a new ImageAcquirer:

ia = harvester.create(device=info)

fails with:
GenTL exception: Operation cannot be performed. Responsible module is busy. ID: -1022
This “busy” state persists until the entire Python interpreter is restarted.
Calling:

ia.stop()
ia.destroy()
harvester.reset()

does not clear the busy state.

2. Harvester internal thread (run_as_thread=True) dies silently

When using:

ia.start(run_as_thread=True)

and then power-cycling the camera:
The internal Harvester acquisition thread terminates unexpectedly
Exceptions do not surface to the user
The producer remains in a locked / half-open state
All subsequent acquisition attempts fail

3. Auto-reconnect appears to work only when staying within a single ImageAcquirer instance

I discovered this in a simple Notebook script:
If I switch of the cam in standby state (no acquisition) - sometimes the AVT GenTL producer reconnects automatically inside the same ImageAcquirer instance.

Expected Behavior

• Power-cycling a camera should not require restarting the entire application
• After destroying the old ImageAcquirer, it should be possible to create a new one
• Harvester’s internal acquisition thread should surface its exceptions cleanly

Motivation

We are a large laser facility (Helmholtz Institute Jena) and are currently implementing an alternative to NI Vision based on Harvesters. Our control system is Tango Controls. Because experimental setups are highly flexible and devices are frequently repowered or disconnected during installation or due to EMC/EMI effects, the Tango Device Server must be robust against temporaly device losts.