For Engine Developers

Coding Style

This engine uses structural programming. The code is organized in modules. OOP methods are not used, classes are not defined, inheritance is not performed and so on.

The K&R style is used except for the fact that the opening bracket for a compound operator is placed on the same line, for example:

function foo_bar() {
    // ...
}

if (a > b) {
    // ...
}

4 spaces are used for indentation (no tabs allowed).

Examples

The underscore symbol is used in function and variable names:

var foo_bar = 123;  // correct
var fooBar = 123;   // wrong

All global variables begin with an underscore:

var _foo_bar = null;

The constants are written in capital letters and never begin with an underscore:

var FOO_BAR = 100;

The names of external API methods and properties are written after a point. To avoid obfuscation of fields they must be listed with the @cc_externs tag:

exports.FOO_BAR = 123;

exports.foo_bar = function() {

}

/**
 * Set properties.
 * @method module:properties.set_props
 * @param {Object} foo Foo object
 * @cc_externs props_1 props_2
 * @cc_externs props_3 props_4
 */
exports.set_props = function(foo) {

    var bar_1 = foo.props_1;
    var bar_2 = foo.props_2;
    var bar_3 = foo.props_3;
    var bar_4 = foo.props_4;

    ...
}

Commenting is in English only. Comment style - JSDoc.

Building the Engine

Before building please make sure that your system has all required dependencies installed (see the table).

To compile the engine and the applications included into the SDK please execute the following command (in the SDK root):

make compile

The full building that includes converting resources (textures, sounds and videos), compilation and converting the docs, can be performed with the following command:

make build

Building the archives with the distributions:

make dist

All above mentioned operations can be performed with a single command:

make all

Building the Add-on

Binary Blend4Web addon builds are available for the following platforms: Linux x32/64, macOS x64, Windows x32/64. At the same time users can compile the addon by themselves.

To do this Python 3.x (it’s better if it’s the same version as in Blender) and a C compiler are required. Under Linux it’s enough to install the python3-dev and build-essential packages.

Paths relative to the repository root:
  • build script: csrc/b4w_bin/build.py
  • Blend4Web addon: addons/blend4web/

The building process is started in the following way:

python3 ./csrc/b4w_bin/build.py

As a result of the building you’ll get a binary file called:

b4w_bin_[PLATFORM]_[ARCHITECTURE].[STANDARD_EXTENSION],

located in the same directory as the addon. Example: b4w_bin_Linux_64.so. After this the addon is ready to use under this platform.

Dependencies

All dependencies are listed in the table below in order of decreasing importance.

Name Ubuntu 16.04 package Purpose
Bash included by default script interpreter
Python 3 included by default script interpreter
NodeJS nodejs compiling shaders
Java default-jre compiling and obfuscating the engine modules
ImageMagick imagemagick converting textures
NVIDIA Texture Tools libnvtt-bin converting textures
Libav libav-tools converting media resources
NVIDIA Cg Toolkit nvidia-cg-toolkit debugging shaders
OptiPNG optipng optimizing PNG files
Emscripten from EMSDK source code building Uranium
Gnuplot gnuplot debugging
Graphviz graphviz debugging
xsel xsel debugging
Sphinx python3-sphinx building the manual
sphinx-intl installed with PIP v3 (pip3 install sphinx-intl) building the manual (internationalization)
TeX Live texlive texlive-latex-extra texlive-lang-cyrillic building the manual (PDF version)
JSDoc 3 installed with NPM (npm install -g jsdoc) building the API documentation
PEG.js from PEG.js source code building shader preprocessor

Naming Functions and Variables

When creating new functions and variables it is recommended to use the following prefixes and suffixes.

init_
create an abstract object
create_
create a certain object
update_
update the state of an existing object
attach_/detach_
add/remove a temporary object property
append_/remove_
add/remove a temporary property to the already existing properties of the same kind
insert_/pop_
add/remove an array element (accessed by index)
switch_
switch flag’s binary value
apply_/clear_
operation with flags, binary values or arbitrary parameters
set_/get_
set/get the property/variable value
_tmp
global variable - cache in the form of a simple object (array, vector)
_cache
global variable - cache in the form of a complex object

Debugging

Engine debugging is performed with the debug.js module methods.

The structure of the current render graph can be saved in the DOT format using the b4w.debug.scenegraph_to_dot() call, for example, in the browser console. After calling this method, save the console’s output into the file with the .gv extension. To get the graph in a visual form the graphviz utilities are required. Converting to the SVG format is performed using the command:

> dot -Tsvg graph.gv -o graph.svg

where graph.gv is the name of the file with the saved graph.

Shader Compilation

All shaders used in the engine are processed by a compiler. The compiler performs the following three main procedures:

  • validation of the shader code,
  • its obfuscation and
  • optimization.

In order to run the compiler, execute one of the following commands in the SDK root:

> make compile_shaders
> make verify_shaders
  • make compile_shaders - performs validation, obfuscation, optimization and finally, export of the compiled shaders,
  • make verify_shaders - performs only validation, obfuscation and optimization.

Syntax analysis (parsing) of the shader text is first performed during compilation. The corresponding parser is created automatically based on the grammar, using the PEG.js generator. Then the shaders are validated, obfuscated and optimized according to the parser data, and after that the shaders are exported in the form of an abstract syntax tree (AST) for direct loading in the engine.

The location of the main files in the repository:

  • initial grammar - glsl_utils/pegjs/glsl_parser.pegjs
  • parser generation script - glsl_utils/pegjs/gen_nodejs.sh
  • parser - glsl_utils/compiler/glsl_parser.js

Validation

The compiler performs the following procedures related to shader code validation:

  • reporting about unused variables and functions (dead code),
  • checking the syntax of shaders,
  • checking the conformance of shaders to the import/export mechanism,
  • removing odd or repetitive tokens: spaces, endlines and semicolons.

Obfuscation

Obfuscation minifies the GLSL code and makes it difficult to understand it. So far the following procedure is implemented:

  • replacing the user-defined identifiers with shorter single-symbol, two-symbol etc. names (with support of the import/export mechanism).

Optimization

Optimization constitutes the following procedures:

  • removing curly brackets which are not useful in any ways except creating local scopes (this functionality is used for processing node/lamp directives),
  • optimization inside functions - creating shared local variables to replace ones originally created by the programmer.

An example of removing unused curly brackets: replacing the following code

void function(){
    int a;
    {
        a = 1;
    }
}

with this code

void function(){
    int a;
    a = 1;
}

Low number of temporary local variables is achieved by repetitively using them in different contexts. For example, the following code

int function(){
    int a = 1;
    int b = a + 3;
    return b;
}

will be replaced with

int function(){
    int _int_tmp0 = 1;
    _int_tmp0 = _int_tmp0 + 3;
    return _int_tmp0;
}

Note

Local variables for structures and arrays are not optimized this way.

Import/Export Directives

import/export directives are used to organize, structure and increase the readability of the shader code in the include file. They are specified in the beginning of the file and should look approximately like this:

#import u_frame_factor u_quatsb u_quatsa u_transb u_transa a_influence
#import qrot

#export skin

The #import directive defines a set of ids which are declared outside the include file but can be accessed from inside it. There is a limitation though: such ids must necessarily be declared somewhere above the place where the include file is linked.

The #export directive defines a set of ids which can be accessed from outside this file. Such ids must necessarily be declared in this file.

Therefore, the shader which uses the include file must have all the declarations necessary for import before the place of linking, and can use the exported ids after it.

Ids can be both variable names and function names. If there are no import/export directives it’s considered by default that the include file does not use external declarations and does not allow the using of internal ones.

Recommendations and Limitations

Because of the following reasons: preprocessing, the need to process multiple shaders and include files and due to the compiler’s features - its possible to guarantee the work of the output code only if a number of rules and limitations are respected with regard to the shader source code:

  1. In order to describe constants which are defined by the engine at run, it’s necessary to use the #var special directive. For example:
#var AU_QUALIFIER uniform
AU_QUALIFIER float a;

The syntax here is similar to the #define directive. The point of the #var directive is that the value which it defines allows to parse the initial shader. It’s irrelevant what exactly it will be (e.g. ‘uniform’ or ‘attribute’ in the above example), because at this level it’s unknown anyway. Nevertheless, it’s better to specify a more or less suitable description and not something arbitrary.

Note

The #var directive is not necessary for constants used not in the shader code but in the preprocessor expressions.

  1. Using the import/export directives when needed.
  2. The built-in functions must not be overloaded - only the user ones.
  3. Variables should not be declared with names of the built-in functions, or main (even if it doesn’t lead to errors).
  4. The #var and #define directives must not be used for replacing single symbols in such operators as: “++”, “–”, “*=”, “/=”, “+=”, “-=”, “==”, “<=”, “>=”, ”!=”, “&&”, “||”, “^^”.

For example:

#var EQUAL =
...
a *EQUAL b;
...
  1. The usage of the #include directive should not lead to ambiguity during the obfuscation of the include file. This can happen when multiple shaders are included into the same file and the above defined directives (like #var or #define) can have influence on any of them. Also, it’s better not to use undeclared functions and variables in the include file.
  2. Multi-level includes or multiple inclusion of the same include into the same shader is not supported.
  3. The shader’s malfunction can also be caused by nontrivial using of preprocessing, for example, creating an invalid GLSL code:
#if TYPE
void function1() {
#else
void function1(int i) {
#endif
    ...
}
  1. Do not declare variables with such names as node_[NODE_NAME]_var_[IN_OUT_NODE], where NODE_NAME — name of some node, IN_OUT_NODE — name of an input or an output of the node.
  2. Repetitive use of #nodes_main, #nodes_global or #lamps_main directives is not permitted inside a single shader.
  3. The #nodes_main, #nodes_global and #lamps_main directives are recommended to use in the file, containing these shader nodes description, for example, in the same include-file. This is necessary for the correct shader validation.

WebGL Extensions

Compilation may depend on WebGL extensions being used if they somehow influence the shading language. At the moment the following extensions are supported by the compiler:

  • OES_standard_derivatives

Compilation Errors

In case of an error the compiler will output the corresponding message in the console.

Table of possible errors:

Error message Cause
Error! Ambiguous obfuscation in include file ‘FILE_NAME’. Ambiguous obfuscation in the ‘FILE_NAME’ include file.
Error! Extension NAME is unsupported in obfuscator. File: ‘FILE_NAME’. The NAME WebGL extension used in the FILE_NAME file is not supported by the obfuscator.
Error! Include ‘FILE_NAME’ not found. The FILE_NAME include file could not be found.
Error! Undeclared TYPE: ‘NAME’. File: ‘FILE_NAME’. Error in FILE_NAME file. Undeclared identifier NAME of type TYPE (variable, function, structure etc).
Error! Undeclared TYPE: ‘NAME’. Importing data missed. File: ‘FILE_NAME’. Undeclared identifier NAME of type TYPE (variable, function, structure etc). Declaration missing for the identifier required in the FILE_NAME include file according to the #import directive.
Error! Undeclared TYPE: ‘NAME’. Possibly exporting needed in include file ‘INCLUDE_NAME’. File: ‘FILE_NAME’. Error in FILE_NAME file. Undeclared identifier NAME of type TYPE (variable, function, structure etc). Possibly its export into the INCLUDE_NAME include file should be allowed.
Error! Undeclared TYPE: ‘NAME’. Possibly importing needed. File: ‘FILE_NAME’. Undeclared identifier NAME of type TYPE (variable, function, structure etc). Possibly it should be specified as imported in the FILE_NAME include file.
Error! Unused export token ‘NAME’ in include file ‘FILE_NAME’. Undeclared identifier NAME is allowed for export in the FILE_NAME include file.

Error! Using reserved word in TYPE ‘NAME’. File: ‘FILE_NAME’. Error in FILE_NAME file. A reserved id is used for declaring the identifier NAME of type TYPE (variable, function, structure etc).
Error! ‘all’ extension cannot have BEHAVIOR_TYPE behavior. File: ‘FILE_NAME’. The #extension directive specified for all WebGL extensions in the FILE_NAME file does not support the behavior BEHAVIOR_TYPE.
Syntax Error. ERROR_MESSAGE. File: FILE_NAME, line: LINE_NUMBER, column: COL_NUMBER. Syntax error in line LINE_NUMBER column COL_NUMBER during parsing the FILE_NAME shader. The initial error description is quoted in the ERROR_MESSAGE. The code listing taken from around the corresponding line is attached to the message (note the peculiarity of pegjs parser which specify the line which is a little bit after the actual error.
Warning! Function ‘NAME’ is declared in [include ]file FILE_NAME, but never used. An unused function NAME is declared in the FILE_NAME file.
Warning! Include file ‘FILE_NAME’ not used in any shader, would be omitted! The FILE_NAME include file is not used in any of the shaders and so it will be excluded from the obfuscated version.
Warning! Unused import token ‘NAME’ in include file ‘FILE_NAME’. An unused id NAME is imported in the FILE_NAME include file.
Warning! Variable ‘NAME’ is declared in include file FILE_NAME, but never used. An unused variable NAME is declared in the FILE_NAME file.

Updating Add-on Translations

If you need to update all existing .po files, run the script translator.py in the SDK/scripts directory without arguments:

> python3 translator.py

In order to update an existing .po file, run the script with a supported language code as an argument:

> python3 translator.py ru_RU

In order to view the list of supported languages, run the script as follows:

> python3 translator.py help

In any case, the file empty.po will be updated upon running the script.

After updates, the .po files can be edited/translated as usual.