What an On-Orbit AI Demo Actually Proves
Three announcements this year put AI on small satellites and let it act without waiting for a ground operator. In headlines they read the same: AI in orbit, no human in the loop. The details differ a lot, and so does the amount of evidence each one offers.
If you are scoping onboard autonomy for a program, those differences decide what you can ask a vendor to show you.
Three demos, three different claims
In March, Syntiant and Novi Space announced that they had run computer vision models on Novi’s SP240 onboard computer, which is built around an AMD Versal chip. The models detected ground vehicles and ships directly in raw sensor imagery. The satellite sent down small packets with each object’s coordinates and type instead of full images, and the companies say a model can be retrained and redeployed in under 24 hours. The coverage gives no detection accuracy figures.
In August, Shield AI and Sedaro reported initial flight tests of Shield AI’s Hivemind autonomy software on a Novi satellite in low Earth orbit. Across consecutive 24-hour windows, Hivemind handled payload imaging, health diagnostics, battery state of charge, and pointing with no real-time operator intervention. The satellite executed 189 commands that Hivemind generated, and each one was validated by Sedaro’s SAFE safety framework. The work ran under Sedaro’s SBIR Phase II contract with the Air Force Research Laboratory and Space Systems Command. The one performance number in the announcement, an 8-point gain in Iridium contact efficiency, came from standalone evaluations, and the coverage doesn’t say whether it was measured in flight.
The most recent is from KAI, the Korean aerospace company. Its BEE-1012 CubeSat reached orbit on October 7 aboard the fifth Nuri launch. On October 20, according to Herald Business and Money Today, a ground station sent a simulated fault signal. KAI’s onboard AI module worked out the cause and scope of the fault, wrote a technical report with countermeasures, and sent it down. The module was trained on tens of thousands of fault scenarios, using methods KAI had developed for aircraft fault prediction and predictive maintenance. KAI published no quantitative results, and it plans a larger demonstration mission for December.
What a simulated fault shows
The following is our reading, not anything the companies state.
A fault injected from the ground tests the pipeline. A signal goes in, a diagnosis comes out, and a report goes back down. Showing that on flight hardware, with a neural processor that fits a CubeSat’s power budget, is a genuine engineering step. It doesn’t show how the model behaves on a fault that wasn’t in its training set, how often it is wrong, or what it does when two things fail at once. If the injected fault resembles the training scenarios, the test measures recall more than diagnosis, and the announcement doesn’t say how the fault was chosen.
The output is also a report sent to the ground. In this test, a person still stands between the diagnosis and any action.
The checker is the part that transfers
The Sedaro and Shield AI result gives a program the most to work with, because of SAFE. Hivemind proposes commands, and a separate framework validates each one before the satellite executes it. An arrangement like that can be specified and tested independently of whichever model does the proposing, which makes it easier for a program office to approve autonomy it can’t fully inspect.
The public material doesn’t describe how SAFE decides what counts as valid, and it doesn’t say how many proposed commands it rejected. The 189 are the ones that got through. A rejection count would say more about the planner than the executed count does.
What to ask for
For a program evaluating onboard autonomy, the public demos suggest a few requests that a vendor should be able to meet. Ask for miss and false-alarm rates on faults or targets held out from training. Ask for the rejection rate of whatever validates the autonomy’s output, along with the logic behind it. And ask how a model update gets re-validated, because a model that can be retrained and redeployed in under a day changes the satellite’s behavior every time it is, and the validation has to keep pace with that cadence.
KAI’s December mission is the next chance to see whether any of these numbers get published.