ArmSoM
← 所有故事

How to Benchmark RK3588 and RK3576 NPU Performance Fairly

NPU 时序在描述可重复任务时最容易解释。单个推理测量省去了加载、数据转换和排队。单独报告这些成本以及测试模型达到的准确性。

ArmSoM Sige7s

ArmSoM Sige7s

定义测量边界

说明计时是从捕获的帧、准备好的张量还是推理调用开始。包括预热行为并在代表性数据集上重复运行。记录模型、精度、运行时间、输入大小和并发工作负载。保持延迟变化可见,而不是将观察到的最佳样本呈现为正常性能。

在多个边界处进行测量

仅推理时间对于诊断加速器很有用,而端到端时间包括捕获、数据转换、处理和输出。尽可能报告两者。吞吐量和延迟回答了不同的问题:批处理系统可以处理许多帧,但需要更长的时间才能返回任何一个结果。包括交互式应用程序中的尾部延迟和丢帧。

运行足够长的时间以揭示约束

测试预期的外壳、冷却和供电。观察温度、时钟变化、内存使用和服务稳定性。重复试验并保留原始日志。将 Sige5、Sige7s 和 Sige7 结果作为这些测试系统的测量结果,而不是每个图像和模型的永久排名。

ArmSoM Sige5

ArmSoM Sige5

根据您的要求进行选择

要求或情况

评估什么

决定详情

品质

转换后的任务准确性

拒绝无与伦比的输出。

时机

推理和完整管道

说明测量边界。

条件

散热、图像和电源

让重复成为可能。

示例设计

发布其他开发人员可以重复的基准

有用的 NPU 结果可识别电路板、图像、转换器、运行时间、模型、精度、输入形状和冷却。包括重复运行并将冷加载与稳态执行分开。如果测试不包括采集或后处理,请在结果旁边注明。报告准确性和延迟,因为未满足应用程序质量要求的更快转换模型并不是一种改进。保留原始测量值,而不仅仅是最佳图表值。

Keep the model and accuracy comparable

Record architecture, model checksum, input dimensions, batch size, precision and conversion settings. Include the output interpretation and thresholds. A smaller input or a different model can make a platform appear faster while solving a different problem.

Measure accuracy on the same representative evaluation set. Decide the minimum acceptable quality before comparing speed. For a detection application, inspect difficult scenes and class-specific behavior as well as an aggregate metric.

Rockchip's Model Zoo is useful for finding supported reference workloads. Treat any upstream performance table as a result under its stated conditions, not a guarantee for your board and application.

Separate four timing measurements

Measurement

What it answers

Inference latency

How long does one inference take under the defined runtime conditions?

Throughput

How many completed inputs are processed per second?

End-to-end latency

How old is an input when its useful result becomes available?

Startup time

How long until the system can process its first valid input?

Do not invert average latency to report throughput unless the execution model makes that valid. As a simple arithmetic example, a strictly serial 20 ms operation has an ideal upper rate of 50 operations per second before other work. Concurrent queues and pipeline stages change that relationship.

Measure end-to-end delay with timestamps that have a known relationship. Camera exposure timestamps, network arrival and application timestamps may come from different clocks; define what your measurement actually includes.

Use repeatable timing boundaries

Warm up the system before a steady-state run, but report cold-start behavior separately when it matters. Wait for asynchronous work to complete before stopping an inference timer. Otherwise the measurement may capture only submission overhead.

Use a monotonic clock and collect a distribution rather than one sample. Report a central value and a tail measure such as the 95th percentile, with the number of samples and run duration. Keep dropped or failed inputs visible instead of silently excluding them.

Measure power at a stated boundary

If energy matters, state where power is measured: board input, complete device input or a particular rail. Include the workload duration and baseline behavior. An input measurement that includes an SSD and fan cannot be compared directly with a processor-only number.

For battery-powered equipment, consider the duty cycle. Startup, idle periods, communication and sleep behavior can matter more than the peak inference interval. Use the product's operating pattern to estimate energy use.

Additional references

硬件和文档

Sige5 · RK3576

RK3576,四个 Cortex-A72 和四个 Cortex-A53 内核,6 TOPS NPU,双千兆以太网和 M.2 PCIe 2.1 x1。

硬件文档 · 产品详情

Sige7s · RK3588S

RK3588S、6 TOPS NPU、千兆以太网和可拆卸 eMMC。尺寸:85 × 56 × 17 mm。更换 eMMC 之前请断开电源。

硬件文档 · 产品详情

Sige7 · RK3588

RK3588、6 TOPS NPU、双 2.5GbE 和 M.2 PCIe 3.0 x4。检查所选的eMMC和无线配置。

硬件文档 · 产品详情

常见问题

我应该只发布最好的成绩吗?

否。报告重复结果并解释差异。

供应商基准数据可以与我们自己的基准数据混合吗?

仅在来源和条件明确分离的情况下;它们不应该看起来是一项受控测试。

产品信息和支持

了解 Sige5

联系 ArmSoM

来源

由 ArmSoM 使用以下文档编写。示例设计是建议,而不是测量的客户安装。

相关阅读