AI-Deck Frame capture on external command / Determine drone position at time of frame capture

[pistoe]

I want to try taking a single frame from a specific position of the drone. Is there any example on triggering a capture via external command? Or has anyone tried doing this?

My end goal is synchronising a captured frame and the position at the time of capture, maybe there is also another way to go about this that I am missing, if anyone has any tips. In my current setup, there is too much movement between frame reception and position logging, and they are not well synchronized since I don't know exactly when the frame was captured and where the drone was in that moment.

So far, I have only found https://github.com/bitcraze/aideck-gap8-examples/tree/master/examples/other/test_functionalities/test_camera.

Currently, I am running modified versions of this AI-Deck ESP firmware https://github.com/larics/aideck-esp-firmware-udp/tree/main/main and this Gap8 task https://github.com/larics/aideck-gap8-examples/tree/master/examples/other/wifi-img-streamer.

Any ideas are much appreciated! :)

[ataffanel]

My understanding is that you already solved the single frame capture but you are triggering it remotely over Wifi/Crazyradio?

If so, the best would be to trigger the capture from the Crazyflie itself. This will greatly reduce the latency. You can run an APP on the Crazyflie that gets the position using the Log subsystem and sends it, together with Crazyflie timestamp, to the GAP8. This can be used as capture signal on the GAP8. At the end you will have (position, STM32 time, GAP8 time, picture) which should allow you to get quite good synchronization.

The relevant example is the STM32–GAP8 CPX application:

• Crazyflie STM32 application
• GAP8 application

This should allow you to get the synchronization down to a couple of milliseconds. Note that the STM32 clock drifts much more than the GAP8 one. You can achieve better synchronization by using the bidirectional STM-GAP link to implement clock synchronization. I would not start with that though, single direction may be enough for you.

Hope this helps :)