Candera Guide to Using GStreamer in CGI Studio
Audience: Developers and integrators working with Candera CGI Studio who want to decode and display video streams (e.g., H.264) via GStreamer inside their applications.
Goal: Install and configure GStreamer, enable streaming in CGI Studio projects, add the VideoStream control, diagnose common issues, and follow best practices for reliable playback.
Overview
CGI Studio integrates a sample implementation for GStreamer to show how to use the VideoStreamer interface. The VideoStreamer sample implementation uses GStreamer to decode the specified video source and provides frames to the Video Stream behavior.
This guide walks you through setup, configuration, and common diagnostics to ensure a smooth video playback experience within your CGI Studio projects and shows how to customize or add your own implementation.
Supported Platforms
-
Windows, Android, and Linux platforms with an installed GStreamer environment are supported.
-
Android support is limited to aarch64 targets.
-
Default modules support H.264 Baseline profile.
Typical Workflow (Quick Start)
-
Install GStreamer and verify with
gst-inspect-1.0.0. -
Ensure
PATHincludes GStreamer. -
Run CMake for host build and enable/set necessary flags for streaming.
-
Build Player for Host (includes
SCHost.dllfor Scene Composer) and target. -
Add Video Stream control in the Scene Composer Solution and set
VideoSource.VideoSource. -
Enable Playback in the statemachine or (optional) hook behaviors:
-
Configure and Play Stream to start playback.
-
Check Video State to loop or pause.
-
-
Debug with
GST_DEBUGand pipeline DOT graphs.
Prerequisites
GStreamer Installation & Verification
GStreamer must be installed and available on your system before running CGI Studio projects that use streaming. The GStreamer version provided with the CGI Studio 3rd‑party software (3PSW) is 1.16.1.
Verify
Please installationuse using:the following command to verify the installation.
gst-inspect-1.0 --version
gst-inspect-1.0
You should see a list of plugins and elements if GStreamer is set up correctly.
Windows
Add GStreamer’s bin directory to your PATH:
PowerShell
# Temporary (Current Session)
$env:Path = "C:\gstreamer\1.0\x86_64\bin;" + $env:Path
gst-inspect-1.0
#
PowerShell
[Environment]::SetEnvironmentVariable("Path", "C:\gstreamer\1.0\x86_64\bin;" + [Environment]::GetEnvironmentVariable("Path","User"), "User")
Android Evaluation Package
Check "Install GStreamer" during setup. Even if already installed, run Repair to ensure all dependencies are met. If CGI Player reports missing DLLs, ensure GStreamer’s bin directory is in the PATH, or copy the DLLs next to the executable.
Target (Linux)
On Linux targets (e.g.,with Raspberryaccess Pito a package manager like RaspberryPi OS, STM32 MPU OpenSTLinux,OpenSTLinux Distribution or Torizon), useTorizon the package manager can be used to install GStreamer and itsit's plugins.
Foroptimal results and hardware acceleration,acceleration please refer to the manufacturer’manufacturers manuals for installing GStreamer and it's manuals.plugins.
For most platforms Gstreamer and it's plugins should be built during YOCTO or Buildroot,buildroot ensurebuilt. GStreamer is includedKeep in the build. Notemind, that specific plugins like aiurdemux may havesupport restrictionsless features or may be more restrictive but offer hardware acceleration.
For more general information, visit:please visit the official GStreamer website: https://gstreamer.freedesktop.org/.
How to Build the CGI Player with GStreamer
Setup
For forthe following steps we assume you successfully finished GStreamer Installation & Verification.
Close Microsoft Visual Studio.
Install GStreamer for Windows (Developerdeveloper and Runtime):runtime:
https://gstreamer.freedesktop.org/data/pkg/windows/X.XX.X/msvc/gstreamer-1.0-devel-msvc-x86-
MSVCX.XX.X.msi
If 64-bityou Developmentskipped GStreamer Installation & Verification also install:
https://gstreamer.freedesktop.org/data/pkg/windows/X.XX.X/msvc/gstreamer-1.0-msvc-x86-X.XX.X.msi
MSVCyou 64-bitdo Runtimenot have GStreamer directories in the folder cgi_studio_3psw/lib and cgi_studio_3psw/src you
need to copy the following files and folders from your GStreamer installation directory:
Manual File Copy (ifthe directoriesfollowing are missing in cgi_studio_3psw):
Include files: Copy glib-2.0 and folders from gstreamer-1.0your GStreamer installation folder (C:/gstreamer/1.0/msvc_x86/lib) to includefolder cgi_studio_3psw/src/lib/GStreamer/Win32/include.
Library folders: Copy the glib-2.0andfollowing library files from gstreamer-1.0your GStreamer installation folder (C:/gstreamer/1.0/msvc_x86/lib) to libfolder cgi_studio_3psw/lib/GStreamer/Win32/lib.
Library files:Copygio-2.0.lib,
glib-2.0.libgobject-gobjext-2.0.libgstreamer-1.0.libCMake Flags
Set the platform and enable/disable streaming and mixed reality features at build time:
Open CMake
GUI.- Input
Setthe location of your source code into the first input field "Where is the source code:"
(/cmake/Candera/Player).
/cmake/Candera/PlayerSetto into the input field "Where to build the binaries:" to your build folder.
To
Gobe tosure Fileyou -> Delete Cache to ensurehave a clean build.
Click Configure,the "Configure" button in the middle of the CMake window on the left side.
Wait
Adjustfor CMake to finish configuration.
- COURIER_PLATFORM
SelectsCOURIER_PLATFORM:theSelectpaltformplatformto(build for. e.g.,AndroidPlatform). - CGIAPP_SAMPLE_STREAMING_ENABLED
Enables
libraries from the project’s third‑party directory.CGIAPP_SAMPLE_STREAMING_ENABLED:streamingONand(Linkslinks GStreamerlibraries).
behaviors and controls. This is required to run the mixedCGIAPP_SAMPLE_MIXEDREALITY_ENABLED:ON(
EnablesAR-augmented‑reality relatedbehaviors/controls).
reality sample solution included with Scene Composer which also contains samples for video
streaming.- CGIAPP_SAMPLE_VIDEO_ROOT_PATH
Make sure this isCGIAPP_SAMPLE_VIDEO_ROOT_PATH: SetSets the base folder for video files.
correctly set depending on you making a host or a target build.
Click Configurethe again,"Configure" thenbutton click Generate.
Wait
Openfor CMake Player.slninto finish configuration.
Wait for the Microsoft Visual Studio
be generated.
Navigate to the Microsoft Visual Studio Solution file in the folder "Where to build the binaries".Open the Solution file "Player.sln" with your working Microsoft Visual Studio.
Wait for the project to load.
In the Microsoft Visual Studio menu press "Build -
Wait for the solution to be built.
Finally, you can start the Player application.
VideoStream Control
The VideoStream control isAs part of the Augmented Reality Controls (AR Controls). Itthe CGI Studio Player was enhanced with a sample implementation for video Streams. This is utilizedused in the MixedReality Sample which is avaliable for selected platforms.
The MixedReality sample tohas handlebeen modified and uses the Video Stream (Video Stream) functionalities.
Furthermore, the intro video streams.is played in a infinite loop instead of once, which is realized with a condition behavior (Check Video State) and action behavior (Configure and Play Stream).
Properties
| Property | Type | Description |
| TargetNode | Candera::AbstractNodePointer | Node where the video will be displayed. If empty, the anchor node is used. |
| VideoSource | FeatStd::String | URI to the video (local path, RTSP, or asset). |
| ScaleToVideoSource | bool | If enabled, resizes texture to match video dimensions. |
| FlipVertically | bool | Flips video vertically (2D only). |
| Timestamp | FeatStd::UInt32 | Bind video timestamp to external reference for sync. |
Using Video Stream in CGI Studio
Add the Control
- Open
The process is similar for both.AddyourControl:sceneInsertintheCGIStudio.VideoStreamcontrolThisintocan be a 2D or 3D scene. Assign Target:SelectAdd themeshVideoStreamorcontrol to the scene.
Add a video source. The video source property is a string. Opening the property editor for the video
Settake VideoSourceyou (to the "Choose Item" dialog. The language table selection opens instead where you can create strings for this property based on the selected locale. The tooltip says to provide the URI Formats):
- Video
which is theAssets:stream from raw resource in the asset:asset://SampleKit_MixedReality##Resources#Videos#SampleVideo_Intro.mp4
"Candera Name" property of the video resource prefixed withasset:// - Video
Networkstream(RTSP/HTTP):from RTSP: Use the URL e.g.http://www.cgistudio.at/mixedreality_sample.mp4 - Video
the CMake configuration and therefore hardcoded in the application. Make sure to set the CGIAPP_SAMPLE_VIDEO_ROOT_PATH correctly for both host builds and target builds.LocalstreamFile:fromProvidefile: As video source string provide just the filenamename.(ensureThe base folder e.g.isCGIAPP_SAMPLE_VIDEO_ROOT_PATH../../cgi_studio_content/html/video/correctset inCMake).
Error Handling: Set a fallbackan image onthat theis RenderNode (2D) or Appearance (3D) to be displayedshown if the stream fails.
For a 3D scene the image is set as texture of the Appearance assigned to the Mesh that is set as StreamTarget. Now
Startchoose Playback:
State
Stream" and set the VideoControl in your Scene as TargetNode.Machine:AddMachine -OnEntry>Action:Openthe state machine and add a new OnEntry on the state of your choice. Set
the Action to "Configure and PlayStream.- Action
Behaviors:andUseConditionBehaviours
- Action: Configure and Play Stream
Condition: Check Video StateSee Candera AR Controls documentation:
cgi_studio_docs\offlineDoc Page: Augmented Reality Controls
The VideoStreamer Interface
VideoStreamer provides an Interface towards the VideoSteamBehavior. The interface header can be found at:
here:
/cgi_studio_controls/src/Behaviors/Streaming/VideoStreamer.h
GStreamer Sample Implementation
The Player contains a non optimized sample implementation in:of the VideoStreamer for GStreamer. The implementation targets to be close to the GStreamer samples and easy to understand. There are no platform specific implementations like specialized pipelines for the iMX 8 series.
The video source string is analyzed and according to the protocol the right source implementation of gstreamer chosen. For video files from assetit is giostreamsrc, for udp it's udpsrc, uridecodebin for http and rtsp and for files uridecodebin is used after concatenating the CGIAPP_SAMPLE_VIDEO_ROOT_PATH set in CMake with the file name and converting the file name to an URI./cgi_studio_player/src/Player/Features/Streamer/GStreamer.cpp
For
AndroidProtocolall Selection:the AutomaticallyGStreamer chooses giostreamsrc (assets), udpsrc (UDP), or uridecodebin (HTTP/RTSP/Files).
Android: Pluginsplugins are treated as static plugins and must be added manuallymanually. (This is
already included in the sample).sample Currently,implementation. As for now only a prebuilt binariesGStreamer areshared only providedlibrary for
Android on aarch64. architectures is in the 3psw folder. For all other architectures the customer must build this
shared libray.
Debugging GStreamer
GStreamer supports setting the loglevel over environment variables or in code.
Enable GStreamer Logs (GST_DEBUG)
SetSetting GST_DEBUG (1-9) to seea consolevalue output:between 1 and 9 enables debug output.
or # Linux/macOS
export GST_DEBUG=3<level>GST_DEBUG=<level> ./Player AssetFileCff.bin
# Windows (PowerShell)
$env:GST_DEBUG=3
.\Player.exe AssetFileCff.bin
| Level | Name | Description |
| 0 |
the requested action. The application can still recover if programmed to handle the
conditions that triggered the error.
2 WARNING
to happen.
3 FIXME
incomplete has been triggered. It may work in most cases, but may cause problems in
specific instances.
4 INFO
only happen once, or are important and rare
5 DEBUG
only a limited number of times during anobject's lifetime; these include setup,
teardown, change of parameters etc.
6 LOG
an object's lifetime; these includestreaming and steady-state
for logmessages that happen on every buffer in an element for example.
7 TRACE Logs all trace messages. Those are message that happen very very often. This is for
example is each time the reference count of a GstMiniObject, such as a GstBuffer or
GstEvent, ismodified.
9 MEMDUMP Logs all memory dump messages. This is the heaviest logging and may include
dumping the content of blocks of memory.
See GStreamer Basic tutorial 11, 2025 for details.
Dumping Pipeline Graphs
ToSometimes visualizea themore pipeline,visual addrepresentation is very helpful to understand what is going on. GStreamer offers built
in support for that.
Add the following code to
GStreamer.cppyour initialization process.(e.g., within bool InitImpl(...) in
InitImpl()C++
static bool init = false;
if (!init) {
init = true;
g_setenv("GST_DEBUG_DUMP_DOT_DIR", "/tmp", TRUE);
}
Alternatively,
And
Add the following code at the point where you want to
(e.g., within bool StopImpl() in
StopImpl()C++
gst_debug_bin_to_dot_file(GST_BIN(m_pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "gstreamerpipe");
Convert
File
After dumping the .dot file, you can use Graphviz to view the file or convert it to SVG Bash
dot -Tsvg /tmp/gstreamerpipe.dot -o /tmp/gstreamerpipe.svg
Graph Structure and Legend
Examples of Pipeline Elements
Legend
Element States
Pad Activation
Pad Flags
Uppercase letters indicate that the flag is set.
Pad Task
Troubleshooting & Best Practices
Known
Android support is limited to aarch64 targets.
iMX8 aiurdemux causes problems with some video files when using giostreamsrc.
- The
MissingsameDLLs:videosEnsurework fine if filesrc is used.
PATHAndroidthe Target: Only aarch64 is supported out-of-the-box.
iMX8 aiurdemux: May cause issues with giostreamsrc.series. Try using fragmented MP4 for progressive playback which does not require seek.filesrc
/cgi_studio_player/src/Player/Features/Streamer/GStreamer.cpp for your needs or
Best Practices & Performance Tips
Use streams compatible with H.264
Baselineprofile for maximum compatibility.Baseline.Start with moderate
resolutions (e.g., 720p) before scaling up.resolutions.- For
optimal performance platform specific implementations are required.
- Leverage
HardwarehardwareAccelerationacceleration:(G2D compositor, videoconvert, VPDSP)DSP.byMostcustomizingSoCs of thepipelineiMXin6GStreamer.cpp.and 8 series have specialized hardware for decoding, color format conversion, scaling etc.
Version Compatibility
| Component | Version |
| GStreamer (3PSW) | 1.16.1 |
| GStreamer (CGI Studio 3.15.1) | 1.22.7 |
References: