Candera Guide to Using GStreamer in CGI Studio
Audience: Developers and integrators working with Candera CGI Studio who want to decode and display video streams (e.g., H.264) via GStreamer inside their applications.
Goal: Install and configure GStreamer, enable streaming in CGI Studio projects, add the VideoStream control, diagnose common issues, and follow best practices for reliable playback.
Overview
CGI Studio integrates a sample implementation for GStreamer to show how to use the VideoStreamer interface. The VideoStreamer sample implementation uses GStreamer to decode the specified video source and provides frames to the Video Stream behavior.
This guide walks you through setup, configuration, and common diagnostics to ensure a smooth video playback experience within your CGI Studio projects and shows how to customize or add your own implementation.
Supported Platforms
-
Windows, Android, and Linux platforms with an installed GStreamer environment are supported.
-
Android support is limited to aarch64 targets.
-
Default modules support H.264 Baseline profile.
Typical Workflow (Quick Start)
-
Install GStreamer and verify with
gst-inspect-1.0. -
Ensure
PATHincludes GStreamer. -
Run CMake for host build and enable/set necessary flags for streaming.
-
Build Player for Host (includes
SCHost.dllfor Scene Composer) and target. -
Add Video Stream control in the Scene Composer Solution and set
VideoSource. -
Enable Playback in the statemachine or (optional) hook behaviors:
-
Configure and Play Stream to start playback.
-
Check Video State to loop or pause.
-
-
Debug with
GST_DEBUGand pipeline DOT graphs.
Prerequisites
GStreamer Installation & Verification
GStreamer must be installed and available on your system before running CGI Studio projects that use streaming. The GStreamer version provided with the CGI Studio 3rd‑party software (3PSW) is 1.16.1.
Verify installation using:
gst-inspect-1.0 --version
gst-inspect-1.0
You should see a list of plugins and elements if GStreamer is set up correctly.
Windows
Add GStreamer’s bin directory to your PATH:
# Temporary (Current Session)
$env:Path = "C:\gstreamer\1.0\x86_64\bin;" + $env:Path
gst-inspect-1.0
# Persistent (User Environment Variable)
[Environment]::SetEnvironmentVariable("Path", "C:\gstreamer\1.0\x86_64\bin;" + [Environment]::GetEnvironmentVariable("Path","User"), "User")
Android Evaluation Package
Check "Install GStreamer" during setup. Even if already installed, run Repair to ensure all dependencies are met. If CGI Player reports missing DLLs, ensure GStreamer’s bin directory is in the PATH, or copy the DLLs next to the executable.
Target (Linux)
On Linux targets (e.g., Raspberry Pi OS, STM32 MPU OpenSTLinux, or Torizon), use the package manager to install GStreamer and its plugins.
-
For hardware acceleration, refer to the manufacturer’s manuals.
-
For YOCTO or Buildroot, ensure GStreamer is included in the build. Note that specific plugins like
aiurdemuxmay have restrictions but offer hardware acceleration.
For more information, visit: https://gstreamer.freedesktop.org/
How to Build the CGI Player with GStreamer
Setup for Microsoft Visual Studio
-
Close Microsoft Visual Studio.
-
Install GStreamer for Windows (Developer and Runtime):
-
Manual File Copy (if directories are missing in
cgi_studio_3psw):-
Include files: Copy
glib-2.0andgstreamer-1.0fromC:/gstreamer/1.0/msvc_x86/includetocgi_studio_3psw/src/GStreamer/Win32/include. -
Library folders: Copy
glib-2.0andgstreamer-1.0fromC:/gstreamer/1.0/msvc_x86/libtocgi_studio_3psw/lib/GStreamer/Win32/lib. -
Library files: Copy
gio-2.0.lib,glib-2.0.lib,gobject-2.0.lib, andgstreamer-1.0.libto the same destination folder.
-
CMake Flags
-
Open CMake GUI.
-
Set "Where is the source code:" to
/cmake/Candera/Player. -
Set "Where to build the binaries:" to your build folder.
-
Go to
File -> Delete Cacheto ensure a clean build. -
Click Configure, choose Win32 (or your target) for the generator.
-
Adjust the following CMake defines:
-
COURIER_PLATFORM: Select platform (e.g.,AndroidPlatform). -
CGIAPP_SAMPLE_STREAMING_ENABLED: ON (Links GStreamer libraries). -
CGIAPP_SAMPLE_MIXEDREALITY_ENABLED: ON (Enables AR-related behaviors/controls). -
CGIAPP_SAMPLE_VIDEO_ROOT_PATH: Set the base folder for video files.
-
-
Click Configure again, then click Generate.
-
Open
Player.slnin Visual Studio and selectBuild -> Build Solution.
VideoStream Control
The VideoStream control is part of the Augmented Reality Controls (AR Controls). It is utilized in the MixedReality sample to handle video streams.
Properties
| Property | Type | Description |
| TargetNode | Candera::AbstractNodePointer | Node where the video will be displayed. If empty, the anchor node is used. |
| VideoSource | FeatStd::String | URI to the video (local path, RTSP, or asset). |
| ScaleToVideoSource | bool | If enabled, resizes texture to match video dimensions. |
| FlipVertically | bool | Flips video vertically (2D only). |
| Timestamp | FeatStd::UInt32 | Bind video timestamp to external reference for sync. |
Using Video Stream in CGI Studio
-
Add Control: Insert the
VideoStreamcontrol into a 2D or 3D scene. -
Assign Target: Select the mesh or rectangle for the video texture.
-
Set VideoSource (URI Formats):
-
Assets:
asset://SampleKit_MixedReality##Resources#Videos#SampleVideo_Intro.mp4 -
Network (RTSP/HTTP):
http://www.cgistudio.at/mixedreality_sample.mp4 -
Local File: Provide the file name (ensure
CGIAPP_SAMPLE_VIDEO_ROOT_PATHis correct in CMake).
-
-
Error Handling: Set a fallback image on the
RenderNode(2D) orAppearance(3D) to be displayed if the stream fails. -
Start Playback:
-
State Machine: Add
OnEntry-> Action:Configure and Play Stream. -
Behaviors: Use
Action: Configure and Play StreamandCondition: Check Video State.
-
The VideoStreamer Interface
The interface header can be found at:
/cgi_studio_controls/src/Behaviors/Streaming/VideoStreamer.h
GStreamer Sample Implementation
The Player contains a sample implementation in:
/cgi_studio_player/src/Player/Features/Streamer/GStreamer.cpp
-
Protocol Selection: Automatically chooses
giostreamsrc(assets),udpsrc(UDP), oruridecodebin(HTTP/RTSP/Files). -
Android: Plugins are treated as static and must be added manually (included in the sample). Currently, prebuilt binaries are only provided for
aarch64.
Debugging GStreamer
Enable GStreamer Logs (GST_DEBUG)
Set GST_DEBUG (1-9) to see console output:
# Linux/macOS
export GST_DEBUG=3
./Player AssetFileCff.bin
# Windows (PowerShell)
$env:GST_DEBUG=3
.\Player.exe AssetFileCff.bin
| Level | Name | Description |
| 1 | ERROR | Fatal errors only. |
| 2 | WARNING | Non-fatal, but visible problems. |
| 3 | FIXME | Incomplete code paths triggered. |
| 4 | INFO | Informational messages (rare events). |
| 5 | DEBUG | General debug (object lifecycle, setup). |
| 6 | LOG | Steady-state streaming events. |
Dumping Pipeline Graphs
To visualize the pipeline, add the following to GStreamer.cpp in InitImpl():
static bool init = false;
if (!init) {
init = true;
g_setenv("GST_DEBUG_DUMP_DOT_DIR", "/tmp", TRUE);
}
And to trigger the dump (e.g., in StopImpl()):
gst_debug_bin_to_dot_file(GST_BIN(m_pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "gstreamerpipe");
Convert the .dot file to SVG using Graphviz:
dot -Tsvg /tmp/gstreamerpipe.dot -o /tmp/gstreamerpipe.svg
Troubleshooting & Best Practices
Known Issues
-
Missing DLLs: Ensure GStreamer is in
PATH. -
Android Target: Only
aarch64is supported out-of-the-box. -
iMX8
aiurdemux: May cause issues withgiostreamsrc. Try usingfilesrcor fragmented MP4.
Best Practices
-
Use H.264 Baseline profile for maximum compatibility.
-
Start with moderate resolutions (e.g., 720p) before scaling up.
-
Leverage Hardware Acceleration (G2D compositor, VP DSP) by customizing the pipeline in
GStreamer.cpp.
Version Compatibility
| Component | Version |
| GStreamer (3PSW) | 1.16.1 |
| GStreamer (CGI Studio 3.15.1) | 1.22.7 |
References: