You need the following Simdify® modules to complete all exercises involving the Raspberry Pi 3B+: Simdify® Free Edition, Simdify® Export Module
This exercise teaches you how to create a new Simdify Shader document, implement a GLSL shader module that performs image convolution, and then visualize the results with a fragment shader. The GPU on your Windows® development machine must support GLSL #version 120 in order to complete this exercise. This section assumes that you know how to access a folder on your Raspberry Pi® 3B+ device, and that you have terminal access to the device.
The application displays a splash screen and then the application desktop appears. The main menu is composed of three items: File, Desktop, and Help. These options provide access to commands relevant to the current context (which is an empty file). The application desktop changes when you create a new file or load a file from disk.
The software displays a wizard that allows you to specify the parameters of your new shader.
Pi3 Image Processing
When the document is saved, the file name will be Pi3 Image Processing.box.
The application creates a new shader document and the main menu options change. In the main application window you can see the hierarchy on the left, the rendered shader with geometry in the middle, and the property sheet on the right.
If the shader compiled successfully, you should see a red square in the center of the worksheet.
For future reference, the location where this file is saved can be obtained using File > Open Containing Folder, when nothing is selected.
The software displays a wizard that allows you to specify export parameters. We'll use the defaults.
The software exports the document and opens a folder to the location of the exported code.
The exported document is composed of a few different types of files. We'll discuss one file of each type below.
File | Description |
---|---|
Application.h | A C++ source code file containing a class declaration. |
Application.cpp | A C++ source code file containing a class definition/implementation. |
Makefile | A file containing instructions on how to build the application. |
main.cpp | The application source code where the main function is defined. |
Pi3 Image Processing.spa_binary_graph | A simplified binary version of the document you made in the Layout application. There is only one of these for each document you export. |
Program-E7274A5C-F421-4C62-9E3F-8E9F58E3E419.spa_binary_node | A simplified binary version of the <Program> node in the document. Other files ending in .spa_binary_node represent other document nodes. |
Note that each exported document will contain different .h, .cpp, and .spa_binary_node files depending on what's inside the document you export.
This creates a new folder and allows you to set the name of folder.
The files are now copied to your device.
cd Pi3-Image-Processing
make
The GCC compiler on your device starts to compile the application:
You receive control of the terminal again when the build is complete.
Please contact support@scenomics.com if you encounter any compiler or linker errors. In either case, it's likely that you haven't updated existing OpenGL packages on your device.
./Run.sh
On your Raspberry Pi® 3B+ desktop, you'll see a window that is completely filled with red.
Now you've exported, compiled, and executed a complete application. Let's create something a little more interesting.
This expands the graph so that you can see all the nodes.
NOTE: You can hover over each node icon, in the image below, for a description of the node and its function.
That covers the basic information about the graph.
Notice that you can see the GLSL version, profile, and source code locations. Many nodes, but not all of them, display useful information if you hover over them. We can see that the shader uses GLSL #version 120 as we requested when we created the document.
This displays a dialog that allows you to select GLSL shader source code (and any include files). The file path of the source item you select will be copied to the Windows® clipboard so you can open it in a text editor. Select pi3_image_processing_fragment_shader.glsl and click OK
The file pi3_image_processing_fragment_shader.glsl opens. Your fragment shader looks like this.
// #version 120
// The version number is automatically injected by the application.
// It is included above for reference purposes only.
#include <SPA_Version.glsl>
varying vec4 fs_position;
varying vec2 fs_texcoord;
varying vec3 fs_normal;
varying vec4 fs_color;
varying vec3 fs_normalViewspace;
varying vec3 fs_positionViewspace;
void main(void)
{
gl_FragColor = vec4( 1.0, 0.0, 0.0, 1.0 );
}
// #version 120
// The version number is automatically injected by the application.
// It is included above for reference purposes only.
#include <SPA_Version.glsl>
varying vec4 fs_position;
varying vec2 fs_texcoord;
varying vec3 fs_normal;
varying vec4 fs_color;
varying vec3 fs_normalViewspace;
varying vec3 fs_positionViewspace;
uniform mat3 convolution_kernel;
uniform int grayscale_enabled;
uniform float convolution_strength;
uniform float step_w;
uniform float step_h;
const int KERNEL_SIZE = int( 9 );
float kernel[KERNEL_SIZE];
vec2 offset[KERNEL_SIZE];
uniform sampler2D src_data;
void main(void)
{
vec4 convolution;
vec4 temp;
float sum = 1.0;
offset[0] = vec2( -step_w, -step_h ); offset[3] = vec2( -step_w, 0.0 ); offset[6] = vec2( -step_w, step_h );
offset[1] = vec2( 0.0, -step_h ); offset[4] = vec2( 0.0, 0.0 ); offset[7] = vec2( 0.0, step_h );
offset[2] = vec2( step_w, -step_h ); offset[5] = vec2( step_w, 0.0 ); offset[8] = vec2( step_w, step_h );
kernel[0] = convolution_kernel[0][0] * convolution_strength;
kernel[1] = convolution_kernel[0][1] * convolution_strength;
kernel[2] = convolution_kernel[0][2] * convolution_strength;
kernel[3] = convolution_kernel[1][0] * convolution_strength;
kernel[4] = convolution_kernel[1][1] * convolution_strength;
kernel[5] = convolution_kernel[1][2] * convolution_strength;
kernel[6] = convolution_kernel[2][0] * convolution_strength;
kernel[7] = convolution_kernel[2][1] * convolution_strength;
kernel[8] = convolution_kernel[2][2] * convolution_strength;
if( fs_texcoord.s < 0.5 )
{
for( int i = 0; i < KERNEL_SIZE; i++ )
{
temp = texture2D( src_data, fs_texcoord.st + offset[i] );
// If the grayscale flag is enabled, calculate the filter
// based on the intensity of the pixels rather than the color value.
// Otherwise process the rgb channels.
if( grayscale_enabled > 0 )
{
float intensity = ( ( 0.299 * temp.r ) + ( 0.587 * temp.g ) + ( 0.114 * temp.b ) );
convolution += vec4( intensity ) * kernel[i];
}
else
{
convolution += temp * kernel[i];
}
sum += kernel[i];
}
sum = max( sum, 1.0 );
convolution = clamp( convolution / sum, vec4( 0.0 ), vec4( 1.0 ) );
}
else
if( fs_texcoord.s > 0.502 )
{
convolution = texture2D( src_data, fs_texcoord.xy );
}
else
{
convolution = vec4( 1.0, 0.0, 0.0, 1.0 );
}
gl_FragColor = convolution;
}
Now we need to rebuild the document.
Your shader is going to turn black when you return to the application. Depending on the behavior of your graphics driver you may see other colors as well. This is because we have not yet set any values for any textures or uniforms. We're going to rebuild the document, which will add the uniforms we declared to the document so we can set their values.
The application rebuilds the document and displays build information in the output window.
--- <Building Project 'D:\Release6\Content\Library\Shader\Pi3 Image Processing\Pi3 Image Processing.box'> ---
Rebuilding Shader Resource Documents...
Rebuild succeeded: D:\users\dorbanhakatan\library\shader\pi3 image processing\120\pi3_image_processing_vertex_shader.glsl
Rebuild succeeded: D:\users\dorbanhakatan\library\shader\pi3 image processing\120\pi3_image_processing_fragment_shader.glsl
Updating document contents...
Adding to <UniformPaletteNode> 'Uniforms' <Float32MatrixNode> 'uniform mat3x3 convolution_kernel'
Adding to <UniformPaletteNode> 'Uniforms' <Int32Node> 'uniform int grayscale_enabled'
Adding to <UniformPaletteNode> 'Uniforms' <Float32Node> 'uniform float convolution_strength'
Adding to <UniformPaletteNode> 'Uniforms' <Float32Node> 'uniform float step_w'
Adding to <UniformPaletteNode> 'Uniforms' <Float32Node> 'uniform float step_h'
Adding to <SamplerPaletteNode> 'Samplers' <SamplerNode> 'uniform sampler2D src_data'
Build completed.
The graph looks like this. There are new uniforms that match the declarations in the shader source code. In the image below, the new nodes are highlighted red for illustrative purposes (but they aren't highlighted in your document).
The software displays a dialog that allows you to select a texture from the hard disk.
IPF_8888_ARGB
california_central_valley.png
The graph looks like this. A new <Texture> has been added to the document.
Next we'll build some local declarations.
It's near the top of the document.
The software displays a list of objects that can be created.
Int32Node
The software displays a dialog that lets you specify the name for the new variable.
image_width
The software adds a new <Int32Node> named image_width to the document.
This displays a dialog that allows you to create objects that capture data from nodes in the document. Notice that the last two options are grayed out. You have to select a <DataCapture> type before you can edit anything else.
Examine the dialog. It looks like the third option has been chosen. This isn't the case. This is just the first option in the list.
The software adds the <DataCapture> objects to the <Int32Node>. Now the value of the <Int32Node> will be set to the width of any <Texture> node connected to the <SamplerNode>.
You'll notice its value is 512. This means the <Texture> source is 512 pixels wide.
If your value does not match, right click on the <Int32Node> named int image_width and select Clear Data Capture from the listed options. The start this section over.
It's near the top of the document.
The software displays a list of objects that can be created.
Int32Node
The software displays a dialog that lets you specify the name for the new variable.
image_height
The software adds a new <Int32Node> named image_height to the document.
This displays a dialog that allows you to create objects that capture data from nodes in the document. Notice that the last two options are grayed out. You have to select a <DataCapture> type before you can edit anything else.
Examine the dialog. It looks like the third option has been chosen. This isn't the case. This is just the first option in the list.
The software adds the <DataCapture> objects to the <Int32Node>. Now the value of the <Int32Node> will be set to the height of any <Texture> node connected to the <SamplerNode>.
You'll notice its value is 512. This means the <Texture> source is 512 pixels high.
If your value does not match, right click on the <Int32Node> named int image_height and select Clear Data Capture from the listed options. The start this section over.
It's near the top of the document.
The software displays a list of objects that can be created.
Float32Node
The software displays a dialog that lets you specify the name for the new variable.
reciprocal_numerator
The software adds a new <Float32Node> named reciprocal_numerator to the document.
This displays a dialog that allows you to edit the value.
You'll notice its value is 1.0.
The software displays a menu that allows you to select various lambdas for the node.
Move the mouse over the <Float32MatrixNode> named uniform mat3x3 convolution_kernel and you can see the matrix values are no longer all zeros.
Repeat this step if your values do not match the values shown above.
This sets the value to 1.
Repeat this step if your value does not match the value shown above.
This displays a dialog that allows you to edit the value.
Repeat this step if your value does not match the value shown above.
This displays a dialog that allows you to create objects that capture data from nodes in the document.
This adds a small <DataCapture> hierarchy to the <Float32Node>. The left side of the division points at float reciprocal_numerator and the right side of the division points at int image_width. This produces the expression:
float reciprocal_numerator / int image_width
1.0 / 512
It's important not to do this division in the shader.
You'll notice its value is 0.001953. This is the result of 1.0 / 512.0.
If your value does not match, right click on the <Float32Node> named uniform float step_w and select Clear Data Capture from the listed options. The start this section over.
This displays a dialog that allows you to create objects that capture data from nodes in the document.
This adds a small <DataCapture> hierarchy to the <Float32Node>. The left side of the division points at float reciprocal_numerator and the right side of the division points at int image_width. This produces the expression:
float reciprocal_numerator / int image_width
1.0 / 512
It's important not to do this division in the shader.
You'll notice its value is 0.001953. This is the result of 1.0 / 512.0.
If your value does not match, right click on the <Float32Node> named uniform float step_h and select Clear Data Capture from the listed options. The start this section over.
This hides the docking panels.
This result looks good!
This shows the docking panels.
This displays an export dialog:
For now we'll use the default options.
The application displays a dialog that asks you to confirm file overwriting.
We did a test export at the start of this exercise. This means there are already files in the export folder. The application will ask you if you wish to delete these files.
The software exports the document and opens a folder to the location of the files.
The files are now copied to your device.
cd Pi3-Image-Processing
make
The GCC compiler on your device starts to compile the application:
You receive control of the terminal again when the build is complete.
Please contact support@scenomics.com if you encounter any compiler or linker errors. In either case, it's likely that you haven't updated existing OpenGL packages on your device.
./Run.sh
On your Raspberry Pi® 3B+ desktop, you'll see a window where the image processing shader is running.
Now you've exported, compiled, and executed a complete application. This is the process by which you can make shaders for your Raspberry Pi® 3B +.