Designing SWaP-Optimized Compute for UAVs

The embedded architecture of unmanned aerial vehicles (UAVs) is changing as more intelligence moves from the remote operator onto the aircraft itself. In a conventional, first-person-view drone, a human analyzes the video feed, determines where the aircraft should go, and makes many of the decisions required to complete the mission.

But the architecture depends on reliable radio communications. When RF links are disrupted or GPS becomes unavailable, the aerial vehicle must perform navigation, perception, and other time-sensitive functions locally. This transition changes the computational requirement. Visual navigation, object detection, tracking, and other perception workloads need to continuously process high-bandwidth camera data alongside lower-level flight and navigation functions.

More processing capability has consequences for the available on-board space. Designers must still fit compute, sensors, and other supporting electronics within the same size, weight, and power (SWaP) budgets while managing the heat. SWaP optimization depends on consolidating perception, navigation, and mission software in an embedded computing platform.

Autonomy is driving more compute

Moving navigation and perception onboard changes how much computing a UAV requires and the type of processing it must perform.

Art Stout, director of product management for AI solutions at Teledyne FLIR OEM, said the company focuses on performing critical functions such as visual navigation, automatic target recognition, and terminal guidance 100% on-device. Cloud resources can still support functions such as fleet management or map distribution, but time-sensitive mission processing must remain available locally.

The increase in compute does not necessarily come from adding large numbers of different sensors. Cameras are the more significant source of data, according to Stout. For instance, a UAV might carry visible-light and thermal cameras and, in some designs, a downward-facing camera for visual navigation. Other devices such as magnetometers, altimeters, and inertial sensors produce small amounts of data and put less demand on the processor.

Visual navigation requires extracting features from sensor data, comparing them with stored map information, and combining the results with inertial measurements to estimate the motion and position. Perception also adds neural network inference, object detection, classification, and tracking.

This creates requirements for different processor classes. An inexpensive, remotely operated UAV can use relatively modest general-purpose compute because much of the higher-level interpretation remains with the operator. But once the vehicle must understand terrain, perform visual odometry, detect objects, and navigate, those functions become embedded workloads.

Consolidating real-time workloads within SWaP limits

Moving perception and navigation on the aerial vehicle raises another architectural question: whether each function should have its own processor or several workloads share a single embedded computing platform.

For small UAVs, adding a processor for every application runs counter to the SWaP goals that motivate highly integrated computing in the first place. Each additional compute board can introduce more printed-circuit-board area, connectors, power conversion circuitry, cabling, mounting hardware, and software to manage. Consolidating workloads onto a capable system-on-module can eliminate some of that duplicated hardware.

This is more relevant as autonomous platforms run several applications concurrently. According to Stout, these UAVs may combine automatic target recognition, collision avoidance, visual navigation, and, in some applications, seeker software for target detection, tracking, and terminal guidance. These software components may also come from different suppliers.

"If they don't want to have multiple processors running each of these stacks separately, which they don't want to do because that adds cost and management problems and complexity, it's ideal to run everything on these embedded processors," Stout said.

The difficulty is that the different processing blocks are not completely independent. They can share DRAM bandwidth, caches, on-chip interconnects, DMA resources, and the processor's overall power and thermal envelope. The more important requirement is whether all applications can meet their latency and throughput targets at the same time.

Two workloads that perform efficiently when benchmarked separately can compete for memory or processing resources after consolidation. For example, visual navigation depends on timestamped camera and inertial data, while tracking depends on new results from an object detector. In such situations, an application that achieves high average throughput can still be unsuitable if another time-sensitive function needs to wait for a shared resource.

Stout identified this resource allocation as one of the main integration challenges. "Each of these parts of the stack is going to be calling for resources, whether it's CPU cores or GPU cores," he said. "What's more important is how we orchestrate all of that so everything runs with low latency and reliably."

Workload placement can also affect the power required to achieve that performance. Stout said that in Teledyne FLIR's experience, Qualcomm-based hardware has used substantially less power than other platforms running the same software stack. He estimated about a 70% reduction in power for the configurations his team has tested.

The architectural point is that power efficiency depends heavily on assigning each workload to an appropriate hardware engine rather than treating the system-on-chip (SoC) as a single uniform pool of compute. This makes SWaP optimization a system-level problem and not simply a search for the processor with the lowest wattage.

Sensor selection adds another variable to the architecture. Higher-resolution cameras provide more pixels on a target, which can extend detection range or provide additional information for fine-grained classification. Stout described the difference between 640- and 1,280-class thermal imaging as providing the perception system with substantially more pixels with which to distinguish between object classes.

Stout explained that as the company has increased the pixel count for its cameras, it can either extend the detection capability to a greater range-4× as many pixels on target-or improve the ability to discriminate between targets at the same range.

But those additional pixels must also move through the embedded computing pipeline. The benefit of sensor resolution, therefore, must be weighed against the additional compute and thermal demands it imposes on the entire platform.

The SWaP tradeoff is how much useful concurrent processing can be consolidated onto the smallest practical hardware architecture while every navigation, perception, and mission workload continues to meet its latency and reliability requirements.

Making modular hardware practical

Consolidating compute does not eliminate the need for modular sensors and processing hardware.

For instance, a camera that plugs into a new processor is usable only after its interface, driver, operating system support, and image-processing pipeline are integrated. For cameras, Stout said MIPI and USB are the most common interfaces, as well as GMSL and Ethernet, depending on the physical architecture and the distance between the camera and processor.

In the case of a short connection between a camera and the compute board, the architecture may favor MIPI because of its compact, low-latency path into the imaging subsystem. But for longer or more complicated paths, including cameras mounted through gimbals or slip rings, serialized or networked interfaces can become more attractive.

There is also a significant difference between these sensor connections and higher-level communication frameworks. MAVLink and Robot Operating System (ROS) may both appear in a UAV architecture, but they do not perform the same function as a camera-to-processor link.

Stout described MAVLink as operating more at the system level, such as communicating vehicle information with an autopilot. For example, a camera could deliver images over MIPI, while software components exchange processed images, detections, or other information via ROS, and the vehicle exchanges commands and state via MAVLink.

Open-hardware standards exist at specific layers of UAV architecture, including standards for autopilots and payload interfaces. But these standards do not define a single, universally adopted sensor-to-AI-compute module interface that abstracts the camera and processor stack.

This is where the software side of modularity becomes more difficult. A raw camera sensor connected to an embedded Linux platform must still be recognized and configured by the operating system. The software stack must know how to control the sensor, establish its clocks and data lanes, receive the image stream, and connect that stream to the camera and image-processing pipeline.

From an OEM integration perspective, Stout emphasized that developing the Linux environment and drivers for the interfaces on a processor is "not an insignificant thing" and that embedded Linux and board support packages form an important part of the development effort.

Processor selection can also create a software integration tradeoff. Stout contrasted Nvidia's CUDA development ecosystem with the work his team encounters when integrating Qualcomm and other mobile-derived SoCs. In his experience, the latter can involve more non-recurring engineering effort around Linux, drivers, and platform integration, even when the processors offer advantages elsewhere in the design.

Qualcomm platforms provide support for Linux, ROS, computer vision, and digital-signal-processor development. The issue is how much of the specific combination of processor, camera, operating system, AI runtime, and application software has already been integrated and supported. For instance, a processor with attractive performance per watt can still have substantial development costs if a required camera lacks a mature driver or an existing software stack needs to be ported.

This makes the practical definition of modularity wider than the ability to unplug one board and install another. Electrical and mechanical compatibility determines if the module can physically integrate with the system. Drivers, board support packages, and APIs determine if the operating system can use it.

For SWaP-constrained UAVs, effective modularity means reducing the amount of reintegration required when a sensor or compute module changes. The most effective SWaP architecture will be the one that treats compute, sensors, thermal design, and software integration as a single co-design challenge rather than optimizing each module independently.

The post Designing SWaP-Optimized Compute for UAVs appeared first on Embedded.