When I first piloted a multi-camera inspection pilot on an assembly line, the bottleneck wasn't the models — it was how we moved video from cameras to inference. We were sending full streams to a central PC, queuing frames, and paying the price in latency and lost throughput. By streaming multi‑camera inference to an edge gateway (no cloud required) I was able to halve inspection cycle time on a production pilot, often without touching the model architecture itself. Below I walk through the practical steps, tradeoffs, and concrete tools I used so you can replicate the result on your line.
Why edge streaming reduces inspection time
Inspection time in a camera-based system is determined by two factors: the time to get a frame to an inference engine, and the time to run inference and return a decision. Centralized approaches add transport and queuing delays, especially when multiple high‑resolution cameras compete for a single inference machine. Moving inference to an edge gateway and streaming frames reduces network hops, enables parallel processing, and keeps end‑to‑end latency predictable.
Key gains come from:
Lower round‑trip latency — no cloud or distant server hop.Concurrent inference — multiple cameras handled by a multi‑core/GPU edge box.Efficient codecs and batching — stream encoded frames and decode on the gateway, or stream preprocessed frames to reduce data size.Target architecture I recommend
Here’s the simple, resilient architecture that worked for us:
| Cameras | RTSP/ONVIF cameras or industrial GigE with MJPEG/H.264 output |
| Edge gateway | NVIDIA Jetson Xavier/Nano or Intel NUC with GPU + OpenVINO/TensorRT |
| Local message bus | gRPC or lightweight broker (MQTT, NATS) for metadata and commands |
| Storage / logging | Local SSD + periodic sync to NAS for audit/case review |
| Orchestration | Docker on the gateway, systemd for service supervision |
The idea is simple: cameras stream encoded video to the gateway; the gateway decodes, optionally pre‑processes and batches frames, runs inference locally, and publishes pass/fail or bounding boxes to the MES/PLC via OPC UA or HTTP.
Practical steps to implement
Choose the right cameras and codecs: Use cameras that can stream in H.264/H.265 or MJPEG. H.264 gives lower bandwidth at the cost of slightly higher decode latency — usually a good trade. For ultra‑low latency choose MJPEG or uncompressed over GigE if the network can handle it.Use an edge gateway sized for parallel inference: For up to 4 1080p streams, a Jetson Xavier or Intel box with a discrete GPU provides headroom. For simple 640x480 defect detection, a Jetson Nano often suffices. I prefer NVIDIA hardware for TensorRT acceleration; on Intel platforms OpenVINO gives equally good results for optimized ONNX models.Stream, don’t transport whole files: Connect cameras directly to the gateway via RTSP. Use GStreamer or ffmpeg pipelines on the gateway to decode and feed frames to the inference process. This avoids file I/O and eliminates remote buffering delay.Implement frame selection at the source: Avoid naïvely inferring every frame. Align inference to process-relevant moments — triggers from sensors, encoder pulses, or motion detection at the camera. In our case a simple PLC trigger + camera I/O reduced redundant frames by 60%.Batch and pipeline intelligently: Batch inference when possible to exploit GPU throughput (e.g., batch size 4–8). At the same time, pipeline decode → preprocess → inference → postprocess across threads so the GPU is saturated while CPU handles I/O and transformations.Quantize and optimize models: Convert models to INT8 or FP16 using TensorRT or OpenVINO. Quantization reduced inference time by 2–4x with negligible accuracy loss in many inspection tasks. Use representative calibration datasets collected from the line.Use ONNX as the interchange format: Train in PyTorch or TensorFlow, export to ONNX, then optimize with TensorRT or OpenVINO. This keeps your ML pipeline vendor‑agnostic while enabling edge acceleration.Reduce input resolution & region of interest (ROI): Split scenes into ROIs on the gateway and run small models on cropped areas. Often a full frame is unnecessary; a 400×400 crop processed at 30 fps suffices.Monitor and failover: Run a lightweight health monitor that reports CPU/GPU use, frame lag, and errors to a local dashboard. If the gateway is overloaded, fall back to sampling mode (process every Nth frame) to keep decisions timely.Tools and components I used
GStreamer pipelines for RTSP ingestion and hardware‑accelerated decoding.NVIDIA DeepStream when using Jetson — excellent for multi‑camera inferencing and batching.TensorRT for model acceleration; OpenVINO for Intel gates.ONNX as the model exchange format.MQTT or OPC UA to publish results to the MES/PLC.Performance tuning checklist
Measure baseline: average latency from capture to decision, throughput (decisions/sec), GPU/CPU utilization.Try model FP16/INT8; compare accuracy and latency.Tune batch size: start with 1 and increase until GPU saturates without raising per‑frame latency too much.Enable hardware decode to free CPU cycles.Implement smart frame selection or hardware triggers.Profile memory copies — zero‑copy between decoder and inference if supported (e.g., NVIDIA DMV buffers).Common pitfalls and how to avoid them
Network saturation: Don’t stream raw high‑res video across an overloaded plant network. Use edge decode and avoid routing full streams through PLCs or central servers.Latency surprises: Encoding adds latency; measure end‑to‑end. If latency is critical, prefer MJPEG or raw GigE when network allows.Thermal throttling: Edge devices running GPUs under full load can thermally throttle. Ensure proper cooling and monitor frequency changes.Over‑batching: Large batches increase throughput but also add latency. Choose batch sizes aligned to your cycle time requirements.How I measured the “half time” improvement
On a pilot line with four cameras, initial centralized inference produced a median decision latency of ~420 ms and throughput of 2.4 inspections/sec. After moving to an edge gateway with H.264 RTSP ingestion, TensorRT FP16 models, and batch size 4, latency dropped to ~190 ms and throughput rose to 5.0 inspections/sec. The observed cycle time halved primarily because we eliminated transport queuing and fully utilized the GPU for parallel inference.
If you want, I can share a minimal GStreamer + TensorRT pipeline example and a small checklist tailored to your camera types and cycle time target. Tell me your camera model(s), expected cycle time per part, and whether you prefer NVIDIA or Intel edge hardware — I’ll draft concrete commands and configurations you can deploy in a day.
