Documentation / UI Guide

Feature

Description

Capture Screen

Capture and camera tabs can be re-ordered (cannot be saved); camera tabs are color coded - green = calibrated, red = not calibrated, yellow = calibrating. These tabs will also show notifications if a color or depth camera is disabled and if the camera has a tempearture out of the acceptable range.

Expand All/Collapse All

Expands and collapses every header section.

Camera Source

Microsoft Azure Kinect.

Refresh

Will show connected cameras that aren’t initially appearing.

Enable Hardware Sync

Should enable hardware sync:

  • Preview window should show the rendered mesh.

  • If delay from primary is set correctly in the color control section, user should not see light pulses from cameras in infrared view.

Preview

Will launch the preview window to the right. Recording will show up as a button next to stop after this is enabled. A recording timer will also be present when recording.

Load Recording

Load master manifest file which will take user to the playback screen.

View Stream

Type in URL to receive a stream from another Capture Suite.

Log

Launches a window where user can see success, warning, and failure prompts.

Volumization

Volumization Confidence

How confident do we have to be that a point is inside the volume. The default setting is .525.

Camera Facing Confidence Gain

How much do we boost confidence based on the surface facing directly to the camera. The default setting is 1.0.

Distance Falloff

How aggressively do we drop confidence away from depth samples. The default setting is .65.

Volumization Resolution

The volumization resolution used for sub-dividing the capture area box along its longest edge and maintains the aspect ratio. The default setting is 256. Note that increasing this value may potentially impact playback speed on client device. If increasing this value, you will have to increase the maximum vertices setting until your content reappears.

Maximum Vertices

When you’re processing a frame, we need to allocate memory in advance for the GPU. The default setting for 256 volumizaton resolution is 262144. If you elect to use a higher volumization resolution, you will have to increase this setting as your volumetric content may disappear.

Minimum Coverage

Will ensure geometry won’t be generated in areas where less than that number of cameras can see in their field of view. 3 is the default setting. If doing a room-scale capture, decrease this setting to 1.

Maximum Queued Frames

Decides how many frames you want to process ahead before you show one. 5 is the default setting. For raw and mesh export, use 3.

Bounding Box

These values, in meters, will allow you to crop out unwanted parts of the scene, such as the floor and ceiling. Shrinking the bounding box will improve the resolution on the subject. If you increase the bounding box too much, you're essentially spreading that volumization resolution over the whole bounding box - you halve the resolution in each dimension as you double the size.

Rebase to bottom

Allows you to rebase the origin to the bottom of the bounding box. Only works in preview mode.

Static Capture

Treats the depth input as one continuous exposure.

Cap at Bounding Box

Will cap the volumized mesh where it passes through the bounding box.

Calibrate Floor

Allows you to remove the floor from your scene. This setting either needs the preview window open or a raw capture to be imported. Clicking cull floor plane will calculate a plane at which points below it get culled - with this enabled, you will notice the floor in a camera depth view will turn a dark purple. If there is an error with floor calibration, try lowering the floor (increasing the min y) then trying again.

Vertex Solving

Type:

  • Point Cloud Reference

    • Max Iterations

    • Reference Confidence

  • Node Center

Point Cloud Reference - uses valid point cloud information from within the bounding box as a reference to produce better output vertices. This is the default setting.

Max Iterations - the number of smoothing iterations; the more iterations you have, the more smoothing you have, but also the higher the performance cost. The default value is 12.

Reference Confidence - this setting will take into account the areas that have point cloud information so the software will smoothen them less. The default value is 0.85.

Node Center - turns off vertex solving and simply chooses the mid point of each volumization cell.

Volumization Rendering

Fade Distance

Used to blend between color textures. The default setting is 0.011.

Color Camera Confidence

How much do we only use the most confident color camera for texturing. The default setting is 14.000.

Camera FOV

Widens or narrows the virtual camera view, but does not move the virtual camera; different from zooming. When preview is enabled, you can scroll the mouse wheel up and down to adjust this value.

Chroma Key:

  • Color

  • Gain

  • Bias

Chroma Key will allow you to choose a color and modify a couple of settings in order to remove green screen bleed. This setting can be set prior to recording compressed captures and raw recordings, but can also be modified in post if recording raw, and in post on the Unity end if recording compressed:

Color - select a color.

Gain - how severe it de-weights against the chroma distance; also controls de-saturation.

Bias - base level cut-off; how far is this color away from the selected chroma key.

Wireframe

Shows wireframe view.

Show Bounding Box:

  • Line Tightness

Modify the colors of your bounding box by way of the included color picker, as well as line tightness.

Show Cameras

Shows placement of cameras relative to the mesh.

Point Clouds:

  • Hide Mesh

  • Point Size

  • Connected cameras and colors

Can show the point clouds for each camera. Modify the point size. You can opt to hide the mesh while displaying the point clouds. This setting requires the preview window to be open or a raw capture to be imported.

Hot Keys:

Hide Mesh: F1

Hide Camera: F2 - F9

Calibration

Marker Width

Width of calibration cube. The default width for a Soar calibration cube is 216.1mm.

Top Marker Offset

This field is for measuring from the center of the face of the marker on a side, straight upwards to the top marker face, which is usually half the paper height + the height of whatever lid we have + the thickness of the paper. Essentially, half the height of the paper + the height of the lid. The default offset for a Soar calibration cube is 153.1mm.

Marker Width Units

Millimeters or inches.

Required Samples

Decides how many frames the calibration method uses. 20 is recommended for the highest accuracy, but will take slightly longer to calibrate compared to a lower number.

Adaptive Thresholding:

  • Bias

Allows you to calibrate in varying lighting conditions. Enabled by default.

Bias - A positive value makes it so more things come through as light. A negative value makes it so more things come through as black.

Only take depth from light areas

During calibration, this setting only takes depth from points that pass a light threshold - should improve results.

Manual Refinement:

  • Import

  • Export

  • Individual Camera

    • Relative

    • Absolute

    • Extrinsics Visual Editor (with Reset)

This can be done prior to recording or after loading a raw capture. In this panel, you can modify the translation and rotation of each camera, either by modifying the text box values or using the visual editor. Relative works by applying modifications to the absolute values. You can also directly edit the absolute values. These settings can then be saved via the profile section. You can also import and export different calibrations, as well as reset. There is also a button to undo a refinement. You can click into a camera tab to disable the specific camera for your export.

Calibrate All Cameras

Will calibrate all connected and turned on cameras.

Reset Calibration

Will reset the calibration state.

Audio

Capture Device

Selects capture device for audio input.

Enable Audio Capture

Enables or disables audio capture. No audio file or manifest file should be generated if disabled.

Enable 3D Audio Playback

Enables 3D sound spatialization on playback devices. Ensure the audio m3u8 file has #EXT-X-SPATIALIZE.

Input Gain

Sets a gain multiplier for recorded audio streams. Note that boosting this value may result in clipped audio.

Raw Capture Format

Formats include IEEE Float or PCM (32 bit or 64 bit).

Start Monitoring

Previews audio prior to recording.

Playback Device

Selects playback device for audio output.

Preview Gain

Acts as a volume slider, but does not modify volume of the file.

Import Raw Capture

  • Start Paused

Modes:

  • Display Only

    • Loop

    • Crop Start

    • Crop End

  • Process with Display

    • Frame Export Initially Disabled

    • Crop Start

    • Crop End

    • Include Audio

  • Load (with file path)

After Importing

  • Play/Pause

  • Step

  • Resubmit

  • Frame Time

  • Make Start

  • Make End

  • Camera Tabs (serial numbers) with color/depth/infrared views and color controls

  • Disable Camera

Start Paused - starts the loaded clip as paused.

Display Only - does not process the clip into another output format.

Loop - loops the loaded clip.

Crop Start - sets a different start point.

Crop End - set a different end point.

Include Audio - can spatialize audio, as well as modify audio offset (seconds) and audio gain; can load custom WAV files.

Process with Display - processes clip into another output format which is selected in the Output section.

Frame Export Initially Disabled - does not export frames.

Load - loads an srd file.

Play/Pause - plays or pauses clip.

Step - steps through frames.

Resubmit - resubmits a frame after changes are made to view the changes to the clip; hot key 'r'.

Make Start - sets a start point.

Make End - sets an end point.

Camera Tabs - show serial numbers and color/depth/infrared views.

Disable Camera - disables a camera for export.

Color Controls - modify brightness, contrast, saturation, and gain; can apply to all cameras.

Output

Capture Name

Name of capture.

Captures Path

Path to save capture.

Local Server

In order to stream on your local network to another Capture Suite or to another client side device, you must select Local Server and Compressed Capture. You can stream both VOD and live content.

If you want to stream to another Capture Suite instance on the same PC, you can use the loopback address (http://127.0.0.1) followed by a colon and the port; the default port is 8080, which usually open. Then append the filename. Ex: (http://127.0.0.1:8080/localstream1_01642615614_master.m3u8). You can also substitute the loopback address for the local IP address.

If you want to stream to another device, on that device you must use HTTP with the local IP address from the computer (found by going to network connections, adapter settings, IPv4 Address) followed by :port/filename. Ex: (http://192.168.8.67:8080/localstream1_01642615614_master.m3u8).

Raw Capture

Saves an srd file.

Mesh Output:

  • Mesh Format

    • Binary PLY (much faster, but no material link to texture)

    • ASCII PLY (good for interop with other software, but no material link to texture)

    • ASCII OBJ (better interop for games, modeling software, and VFX)

  • Unwrapped Textures:

    • Texture Width

    • Texture Height

    • Gutter

    • Max Stretch

  • Raw Color Feeds

  • MVE Output

  • Per-Camera Point Cloud PLY

  • Camera Model Output (Export Cameras)

Can output sequences of Binary PLY, ASCII PLY, and ASCII OBJ.

The associated JSON files will show the frame info and both Timebase 24 and Timebase 30. If run through Timebase 24, frames will snap to the closest frame as well as double. Ex: Your second and third frame may be closer to 4 than 3 or 5, which means the two frames will snap to 4, which is the closest timestamp-wise. The same will occur with Timebase 30.

Unwrapped Textures - will output textures for your meshes.

Texture Width/Texture Height - sets the resolution of the output texture; default value at 4096 x 4096 should result in no loss of deatail - for VFX work, you want as much texture detail and you can always downscale after the export.

Gutter - the border in pixels between different mesh segments in the texture for the purpose of mipmapping; lowering this value too far might cause issues when zoomed out on a model, whereas with a high res texture, you may want to increase this setting since you may be viewing from very far away and still want no visual issues. The default value is 16.

Max Stretch - how much a triangle can stretch and change shape when it gets UV unwrapped; if setting is lower, will preserve more uniform detail, but will result in longer texture solves and more seams. The default value is .250.

Raw Color Feeds - exports the JPEGs with your meshes.

MVE Output - a set of directories and files that describe the cameras, per-camera data, as well as 3D features and their relation to images.

Per-Camera Point Cloud PLY - exports a colored PLY that corresponds to each camera using the configured point cloud display color.

  • In order to export to MVE, ensure you do this when importing a raw capture and exporting to OBJ or PLY; geometry files are in the root folder and then all the MVE information is in the MVE folder.

Camera Model Output (Export Cameras) - export the camera models to show placement of cameras in virtual space; can either enable prior to processing with display or click export cameras.

Compressed Capture:

  • Transport:

    • HLS

  • Codec:

    • H264

  • Tuning:

    • High Quality

    • Low Latency

    • Ultra Low Latency

    • Lossless

  • Video Width

  • Video Height

  • Vertex Quantization

  • Rate Mode:

    • Variable Bitrate

    • Constant Quality

    • Constant Bitrate

  • Max Bitrate

  • Quality

Transport - only HLS.

Codec - only H264.

Tuning:

  • High Quality - highest quality.

  • Low Latency - in high bandwidth channel for bigger occasional frames.

  • Ultra Low Latency - in strictly bandwidth-constrained channel.

  • Lossless - preserving original footage.

Video Width/Video Height - set desired resolution; best to use a multiple of 16 and ideally a power of 2 while ensuring the width is larger than the height. The default setting is 2048 x 1024. At this setting, you will be able to run multiple instances on the client side device. There are a few resolutions where there may be a potential lockup on the decoder side - if modifying from the default, be sure to load the recording and confirm it plays.

Vertex Quantization - meant to control the Geometry Quality/Bandwidth Ratio. The default setting is 3.

Rate Mode:

  • Variable Bitrate - conforms to average bitrate over the long term while not exceeding max bitrate.

  • Constant Quality - consistent quality.

  • Constant Bitrate - average bitrate used as the target output bitrate.

    • Average Bitrate - average bitrate of content.

Max Bitrate - the highest bitrate the recording can be. For 8 cameras, this setting should be around 10000 (Kbps), which is the default setting.

Quality - determines the quality setting used for variable bitrate or constant quality modes. The default setting is 23.

Profile

Load:

  • Load Default Path

  • Load Settings

Save:

  • Save Default Path (Ctrl + S)

  • Save Settings

Restore:

  • Restore Defaults

Load:

  • Default Path - loads settings from AppData.

  • Other Path - loads settings from a different path.

  • When saving profile it saves to both default and currently loaded one.

Save:

  • Default Path - saves settings in AppData.

  • Other Path - save to a different path.

  • When saving profile it saves to both default and currently loaded one.

Restore:

  • Restore Soar default settings.

UI Settings

Show Tooltips

Enable or disable tooltips.

Pop-up log on all errors

Will pop the log up on errors.

Camera Screen

Friendly Camera Name/Serial Number

Can modify the name and save to profile. Will show camera serial number.

View: (Apply to All option)

  • Color

  • Depth

  • Infrared

Can switch between color, depth, and infrared views. Select apply to all to apply to all connected and turned on cameras.

Depth Type

Unfilled (raw depth map from the camera), Filled (post-processed hole filled depth map), and Confidence (shows the confidence map that correlates to the confidence facing gain setting). Confidence processing is not shown until preview has been switched on.

Calibrate Camera

Calibrates this camera.

Expand/Collapse All

Expands and collapses all sections.

Color:

  • Enabled

  • Resolution: (Apply to All option)

    • 720p

    • 1080p

    • 1440p

    • 1536p

    • 2160p

    • 3072p (15 FPS)

  • Hardware JPEG Decoding

  • Color Controls

    • Apply to All option

    • Exposure Time (Auto Exposure option)

    • Gain

    • White Balance (Auto White Balance option)

    • Brightness

    • Contrast

    • Saturation

    • Sharpness

    • Backlight Compensation

    • Powerline Frequency

      • 50Hz

      • 60Hz

    • Reset to Default Settings

    • Delay from Primary

Can enable and disable the color camera.

Can modify the resolution for this camera, as well as apply to all connected and turned on cameras.

Hardware JPEG Decoding - the Capture Suite will use more of the GPU to ensure higher performance. This is recommended for every camera.

Color Controls - can apply to this camera or to all cameras if apply to all is checked.

Exposure Time - can be manual; ets exposure time and gain based on lighting conditions of environment. There is an Auto option. When auto is enabled, the Exposure time and Gain values will be set for you. The unit of measurement is microseconds.

Gain - in combination with exposure time, gain will increase the exposure of the sensor color video. This setting is hidden when Auto Exposure is enabled.

White Balance - sets recommended color temperature based on lighting conditions of environment. There is an Auto option.

Brightness - adjusts overall brightness of sensor color video.

Contrast - adjusts overall contrast of sensor color video.

Saturation - adjusts overall saturation of sensor color video.

Sharpness - accentuates fine color detail represented in sensor color video.

Backlight Compensation - can be enabled if you are shooting in a low or inconsistently lit environment.

Powerline Frequency - parameter to prevent flickering or banding seen in video that is not compatible with the AC frequencies of the capture space; while 60Hz is most common in North America, most other countries have an AC frequency of 50Hz.

Reset to Default Settings - will reset all settings to reset the camera to default settings.

Delay from Primary - allows you to select the delay for each subordinate camera when using hardware sync. The unit of measure is microseconds. Hardware Sync requires each camera to have a delay offset which ensures no other infrared cameras are firing when that specific camera is taking a capture. We recommend a delay multiple of 160 (first camera is 0, second camera is 160, third camera is 320, etc. - this is dependent on camera configuration, depending on which camera is the primary). The primary camera, which has a sync cable in Sync Out, but no sync cable in Sync In, should be set to 0. This setting should be done prior to calibrating.

Depth

  • Enabled

  • Resolution: (Apply to All option)

    • 320 x 288

    • 640 x 576

    • 512 x 512

    • 1024 x 1024 (15 FPS)

  • Hole Filling: (can use Apply to All option)

    • None

    • Basic

      • Direction

        • Near Camera

        • Far Camera

Can enable and disable the depth camera.

Can modify the resolution for this camera, as well as apply to all connected and turned on cameras.

Hole Filling (Before selecting a hole filling option, you can click Apply To All to ensure your hole filling option is applied to all cameras.):

  • None - if your quality problem is more noise rather than surface fringe holes, turning hole filling off may be helpful.

  • Basic - this is the default hole filling option.

    • Direction

      • Near Camera - foreground will spread out.

      • Far Camera - background will spread out; this is the default setting.


Camera Geometry:

  • Near Plane

  • Far Plane

Near Plane - the minimum distance which the camera will capture; anything past this value will be discarded. The default value is 0.050.

Far Plane - the maximum distance which the camera will capture; anything past this value will be discarded. The default value is 10.000.

Device Statistics:

  • Camera frame orientation

  • Temperature

  • Accelerometer

  • Gyroscope

  • Sync Jack In status

  • Sync Jack Out status

  • Sync Status:

    • Primary, Subordinate, Subordinate (Last)

  • Sync State:

    • Active, Inactive

  • Calibration Status

Camera Frame Orientation - how the camera is rotated.

Temperature - temperature of the camera.

Accelerometer - acceleration of the camera.

Gyroscope - measurement of orientation and angular velocity of the camera.

Sync Jack In/Sync Jack Out - will show if the sync cable is connected or disconnected.

Sync Status:

  • Primary - only a sync out cable.

  • Subordinate - sync in and sync out cable.

  • Subordinate (Last) - only a sync in cable.

Sync State - tells whether hardware sync is active or inactive.

Calibration Status - tells whether the camera is calibrated.

Playback Screen

Duration/Seek Bar with Play/Pause/Stop

Duration of clip - can seek on this bar.

Text field (Ready, Decoding, etc.)

Shows state of clip.

Full Duration

Shows duration of clip.

Frame start time

Start time of frame.

Frame end time

End time of frame.

Triangles

How many triangles in this frame.

Make Import Start/Make Import End

Sets the start point and end point for the Import Raw Capture section.

Volumization Rendering:

  • Color Camera Confidence

  • Camera FOV

  • Reset View

  • Chroma Key Override

    • Color

    • Gain

    • Bias

  • Wireframe

  • Show Bounding Box

    • Front Top Color

    • Front Bottom Color

    • Back Top Color

    • Back Bottom Color

    • Line Tightness

  • Audio

    • Playback Device

    • Preview Gain

  • Log

  • Preview Window (check for mouse and keyboard inputs)

  • Full Screen

  • Auto-Rotate

  • Reset View

Color Camera Confidence - how much do we only use the most confident color camera for texturing. The default setting is 14.000.

Camera FOV - widens or narrows the virtual camera view, but does not move the virtual camera; different from zooming. When preview is enabled, you can scroll the mouse wheel up and down to adjust this value.

Reset View - resets the Camera FOV.

Chroma Key Override - the chroma key setting will use the chroma key value that was set prior to recording if there was one set. If not, you can override it, but this only applies to playback and won’t re-save to the format. You can modify this on the Unity end after importing the recording.

Wireframe - shows wireframe view.

Audio:

  • Playback Device - selects playback device for audio output.

  • Preview Gain - acts as a volume slider, but does not modify volume of the file.

Log - launches a window where user can see success, warning, and failure prompts.

Preview Window:

  • Mouse Inputs:

    • Left-Click on mouse and drag:

      • Left/Right - rotate along y-axis.

      • Up/Down - rotate along x-axis.

      Right-Click on mouse and drag:

      • Shifts content side to side along the y-axis.

      Scroll Mouse Wheel:

      • Adjusts virtual camera field of view.

      • Hold shit for zoom functionality.

  • Keyboard Inputs:

    • Keyboard Arrow Controls:

      • Hold or Click Up/Down - zoom in/out on content (hold Shift to increase speed).

      • Hold or Click Left/Right - rotate left/right around content (hold Shit to increase speed).

  • Full Screen - will enlarge the preview window to take up the whole application screen.

  • Auto-Rotate - will automatically rotate the view.

  • Reset View - will reset the camera FOV.

Preview Window

Mouse Inputs

Left-Click on mouse and drag:

  • Left/Right - rotate along y-axis.

  • Up/Down - rotate along x-axis.

Right-Click on mouse and drag:

  • Shifts content side to side along the y-axis.

Scroll Mouse Wheel:

  • Adjusts virtual camera field of view.

  • Hold shit for zoom functionality.

Keyboard Inputs

Keyboard Arrow Controls:

  • Hold or Click Up/Down - zoom in/out on content (hold Shift to increase speed).

  • Hold or Click Left/Right - rotate left/right around content (hold Shit to increase speed).

Full Screen

Full Screen - will enlarge the preview window to take up the whole application screen.

Auto-Rotate

Auto-Rotate - will enlarge the preview window to take up the whole application screen.

Reset View

Reset View - will reset the camera FOV.