Babylon.js: Creating a convincing world for your game with custom shaders, height maps and skyboxes - Eternal Coding - HTML5 / Windows / Kinect / 3D development - Site Home - MSDN Blogs

Babylon.js: Creating a convincing world for your game with custom shaders, height maps and skyboxes


 

Babylon.js: Creating a convincing world for your game with custom shaders, height maps and skyboxes

  • Comments 15

After talking about multi-materials in the previous post, I would like to share with you a more advanced sample. This sample will allow me to introduce you some really powerful features of Babylon.js:

  • Height maps
  • Skyboxes
  • Custom shaders

The result will look like that (using IE11 preview in this case):

Click here if you want a live demonstration (If your browser supports WebGL of course)

This world is composed of a sky, a ground and a reflective/refractive water. So let’s discuss about each of them Sourire

Preparing the web page

First of all we need a simple HTML 5 page with a full page canvas:

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>Using babylon.js - Test page</title>
    <script src="hand.minified-1.1.0.js"></script>
    <script src="babylon.js"></script>
    <script src="Water/waterMaterial.js"></script>
    <style>
        html, body {
            width: 100%;
            height: 100%;
            padding: 0;
            margin: 0;
            overflow: hidden;
        }

        #renderCanvas {
            width: 100%;
            height: 100%;
            touch-action: none;
        }
    </style>
</head>
<body>
    <canvas id="renderCanvas"></canvas>
</body>
</html>

The page just needs to reference babylon.js (you can find the latest version here) and hand.js for the touch support (latest version here).

Then you can create a script block right after the canvas element with the following code:

<script>
    if (BABYLON.Engine.isSupported()) {
        var canvas = document.getElementById("renderCanvas");
        var engine = new BABYLON.Engine(canvas, true);
        var scene = new BABYLON.Scene(engine);
        
        var camera = new BABYLON.ArcRotateCamera("Camera", 0, 0, 10, BABYLON.Vector3.Zero(), scene);
        var sun = new BABYLON.PointLight("Omni0", new BABYLON.Vector3(60, 100, 10), scene);

        camera.setPosition(new BABYLON.Vector3(-40, 40, 0));

        var beforeRenderFunction = function () {
            // Camera
            if (camera.beta < 0.1)
                camera.beta = 0.1;
            else if (camera.beta > (Math.PI / 2) * 0.9)
                camera.beta = (Math.PI / 2) * 0.9;

            if (camera.radius > 50)
                camera.radius = 50;

            if (camera.radius < 5)
                camera.radius = 5;
        };

        camera.attachControl(canvas);

        scene.registerBeforeRender(beforeRenderFunction);
        
        engine.runRenderLoop(function () {
            scene.render();
        });
    }
</script>

This code creates the engine, the main scene and add a camera and a light (the sun) to it.

The camera is an ArcRotateCamera so you can use your mouse/touch/keyboard to rotate around a central pivot. We just want to limit the amplitude of the camera with beforeRenderFunction function (because we don’t want to go under the ground or beyond the sky Sourire). This function is attached to the scene with registerBeforeRender so it will be called before every frame to guarantee our constraints.

Adding a skybox

A skybox is a box with a special material used to simulate the sky:

image

The material uses a special reflection texture. To create a skybox with Babylon.js you just have to use this code (because skyboxes are already supported by the StandardMaterial):

// Skybox
var skybox = BABYLON.Mesh.CreateBox("skyBox", 1000.0, scene);
var skyboxMaterial = new BABYLON.StandardMaterial("skyBox", scene);
skyboxMaterial.backFaceCulling = false;
skyboxMaterial.reflectionTexture = new BABYLON.CubeTexture("skybox/skybox", scene);
skyboxMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
skyboxMaterial.diffuseColor = new BABYLON.Color3(0, 0, 0);
skyboxMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
skybox.material = skyboxMaterial;

A skybox is just a box with a StandardMaterial Clignement d'œil. The key point is the CubeTexture used for the reflection channel. A cube texture is composed of 6 textures (one for each face of a cube). Babylon.js will choose the right one depending on the position of the viewer to simulate a continuous sky:

image

When you want to create a CubeTexture, you have to specify a file scheme based on the following; nx, ny, nz stand for negative (x, y, z) and px, py, pz stand for positive (x, y, z).

Adding the ground and the island

The ground is a simple plane textured with a repetitive bitmap:

image

To create it, the code is really simple:

var extraGround = BABYLON.Mesh.CreateGround("extraGround", 1000, 1000, 1, scene, false);
var extraGroundMaterial = new BABYLON.StandardMaterial("extraGround", scene);
extraGroundMaterial.diffuseTexture = new BABYLON.Texture("ground.jpg", scene);
extraGroundMaterial.diffuseTexture.uScale = 60;
extraGroundMaterial.diffuseTexture.vScale = 60;
extraGround.position.y = -2.05;
extraGround.material = extraGroundMaterial;
The diffuse texture is scaled by a ratio of 60 in order to repeat it along the ground.


The island also uses a plane but deforms it through an height map.

image

The height map is a simple map used to define the altitude of every vertex of the plane:

image

To use it, you have to create a plane with more subdivisions. The plane is then updated using the height map:

var ground = BABYLON.Mesh.CreateGroundFromHeightMap("ground", "heightMap.png", 100, 100, 100, 0, 10, scene, false);
var groundMaterial = new BABYLON.StandardMaterial("ground", scene);
groundMaterial.diffuseTexture = new BABYLON.Texture("ground.jpg", scene);
groundMaterial.diffuseTexture.uScale = 6;
groundMaterial.diffuseTexture.vScale = 6;
groundMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
ground.position.y = -2.0;
ground.material = groundMaterial;

The definition of the CreateGroundFromHeightMap is the following:

function (name, url, width, height, subdivisions, minHeight, maxHeight, scene, updatable)

As you can see the fifth parameter allows you to increase the complexity of your mesh in order to improve the visual quality of it:

image

The water

The water itself is just a simple plane created with the following code:

var water = BABYLON.Mesh.CreateGround("water", 1000, 1000, 1, scene, false);

image

The true water lies within the material used to display it. And this time we will not use the good old StandardMaterial.

Indeed, to create a convincing water surface, we need to take in account complex  phenomena like refraction and reflection that the StandardMaterial is not intended to handle. To reproduce them, we will use a very powerful feature of Babylon.js: the render target textures. This kind of textures allows you to render a scene into a texture in order to use it later in your own shaders.

Creating an empty custom material

So we need to create our first custom material! To do so, we first need to add a new JavaScript file to our site. This file named waterMaterial.js will contains the following empty anonymous function:

(function() {
})();

Do not forget to reference it in your HTML page:

<script src="Water/waterMaterial.js"></script>

Starting from our empty anonymous function, let’s add a new object called WaterMaterial:

(function() {
    WaterMaterial = function (name, scene, light) {
        this.name = name;
        this.id = name;
        this.light = light;

        this._scene = scene;
        scene.materials.push(this);
    };

    WaterMaterial.prototype = Object.create(BABYLON.Material.prototype);

    // Properties   
    WaterMaterial.prototype.needAlphaBlending = function () {
        return false;
    };

    WaterMaterial.prototype.needAlphaTesting = function () {
        return false;
    };

    // Methods  
    WaterMaterial.prototype.isReady = function (mesh) {
        return true;
    };

    WaterMaterial.prototype.bind = function (world, mesh) {
    };

    WaterMaterial.prototype.dispose = function () {
        this.baseDispose();
    };
})();

This is the minimal code to provide for a custom material:

  1. A constructor to register the material to the scene and get important information (in my case I need the current light to simulate the sun)
  2. Our material MUST retrieve and use  the BABYLON.Material prototype
  3. needAlphaBlending: The material must indicate to Babylon.js if it requires alpha blending (in our case we are not based on alpha blending but on render target textures)
  4. needAlphatesting: The material must indicate to Babylon.js if it requires alpha testing (same thing here, we do not need alpha testing)
  5. isReady: Babylon.js will call this function to know if the material is ready to be used
  6. bind: Babylon.js will call this function to activate the shader before rendering objects that use it
  7. dispose: This function allows you to release resources you may have created for your material

The vertex and fragment shaders

The next thing we need to prepare are the shaders themselves. Shaders define the code executed by the GPU to process the vertices sent by the meshes and the pixels produced by these vertices.

For more information about how a GPU works internally and how to create a 3D engine I suggest you to read the excellent series on how to create a 3D soft engine from scratch written by David Rousset:

http://blogs.msdn.com/b/davrous/archive/2013/06/13/tutorial-series-learning-how-to-write-a-3d-soft-engine-from-scratch-in-c-typescript-or-javascript.aspx

The goal of this blog is not to talk about shaders and GLSL but we need them, so here is the vertex shader that we will use for our material:


#ifdef GL_ES precision mediump float; #endif


// Attributes
attribute vec3 position; attribute vec3 normal; attribute vec2 uv; // Uniforms uniform vec2 waveData; uniform mat4 windMatrix; uniform mat4 world; uniform mat4 worldViewProjection; // Normal varying vec3 vPositionW; varying vec3 vNormalW; varying vec4 vUV; varying vec2 vBumpUV; void main(void) { vec4 outPosition = worldViewProjection * vec4(position, 1.0); gl_Position = outPosition; vPositionW = vec3(world * vec4(position, 1.0)); vNormalW = normalize(vec3(world * vec4(normal, 0.0))); vUV = outPosition; vec2 bumpTexCoord = vec2(windMatrix * vec4(uv, 0.0, 1.0)); vBumpUV = bumpTexCoord / waveData.x; }

And here is the fragment shader

#ifdef GL_ES
precision mediump float;
#endif

uniform vec3 vEyePosition;
uniform vec4 vLevels;
uniform vec3 waterColor;
uniform vec2 waveData;

// Lights
varying vec3 vPositionW;
varying vec3 vNormalW;
uniform vec3 vLightPosition;

// Refs
varying vec2 vBumpUV;
varying vec4 vUV;
uniform sampler2D refractionSampler;
uniform sampler2D reflectionSampler;
uniform sampler2D bumpSampler;

void main(void) {
    vec3 viewDirectionW = normalize(vEyePosition - vPositionW);

    // Light
    vec3 lightVectorW = normalize(vLightPosition - vPositionW);

    // Wave
    vec3 bumpNormal = 2.0 * texture2D(bumpSampler, vBumpUV).rgb - 1.0;
    vec2 perturbation = waveData.y * bumpNormal.rg;

    // diffuse
    float ndl = max(0., dot(vNormalW, lightVectorW));

    // Specular
    vec3 angleW = normalize(viewDirectionW + lightVectorW);
    float specComp = dot(normalize(vNormalW), angleW);
    specComp = pow(specComp, 256.);

    // Refraction
    vec2 texCoords;
    texCoords.x = vUV.x / vUV.w / 2.0 + 0.5;
    texCoords.y = vUV.y / vUV.w / 2.0 + 0.5;

    vec3 refractionColor = texture2D(refractionSampler, texCoords + perturbation).rgb;

    // Reflection
    vec3 reflectionColor = texture2D(reflectionSampler, texCoords + perturbation).rgb;

    // Fresnel
    float fresnelTerm = dot(viewDirectionW, vNormalW);
    fresnelTerm = clamp((1.0 - fresnelTerm) * vLevels.y, 0., 1.);

    // Water color

    vec3 finalColor = (waterColor * ndl) * vLevels.x + (1.0 - vLevels.x) * (reflectionColor * fresnelTerm * vLevels.z + 
(1.0 - fresnelTerm) * refractionColor * vLevels.w) + specComp; gl_FragColor = vec4(finalColor, 1.); }

These shaders use many external variables to achieve the rendering. External variables (defined with the uniform keyword) are used to communicate between your code and the GPU. The GPU will execute the shaders code using the values provided to the external variables by your code:

  • waveData: Defines height and amplitude of waves
  • windMatrix: Defines the direction of the water
  • world: World matrix (Matrix of the current mesh)
  • worldViewProjection: Combined transformation matrix (world x view x projection)
  • vEyePosition: Camera’s position
  • vLightPosition: Sun’s position
  • vLevels: Blending levels of reflection and refraction
  • waterColor: Water’s color
  • refractionSampler: Variable used to read the refraction texture
  • reflectionSampler: Variable used to read the reflection texture
  • bumpSampler: Variable used to read the bump texture (which is used to generate the waves)

To sum things up, we can decompose the shaders’ work through the following pipeline:

1. The mesh is transformed by the vertex shader to generate the triangle used to find the pixels that require to be painted:

image

2. Diffuse color is computed first (based on the sun position and on the water’s color)

image

3. Specular is then added (based on camera’s position):

image

4. The refraction texture is then used to simulate the transparency of the water:

image

5. The bump texture is then used to add perturbations:

image

6. The reflection texture is finally used to add the reflected objects (island and sky):

image

7. The cherry on the cake is added by using a Fresnel computation in order to prioritize reflection or refraction depending on the view angle of the camera:

As you can see we start for an almost full reflection to finish to full with a full refraction when we are perpendicular with the ground.

Linking shaders to the material

Now we have created our shaders, we need to link them with the material. We also need to prepare data for the external variables used by the shaders. To do so, let’s update the constructor:

WaterMaterial = function (name, scene, light) {
    this.name = name;
    this.id = name;
    this.light = light;

    this._scene = scene;
    scene.materials.push(this);

    this.bumpTexture = new BABYLON.Texture("Water/bump.png", scene);
    this.bumpTexture.uScale = 2;
    this.bumpTexture.vScale = 2;
    this.bumpTexture.wrapU = BABYLON.Texture.MIRROR_ADDRESSMODE;
    this.bumpTexture.wrapV = BABYLON.Texture.MIRROR_ADDRESSMODE;

    this.reflectionTexture = new BABYLON.MirrorTexture("reflection", 512, scene, true);
    this.refractionTexture = new BABYLON.RenderTargetTexture("refraction", 512, scene, true); 
    this.reflectionTexture.mirrorPlane = new BABYLON.Plane(0, -1, 0, 0);

    this.refractionTexture.onBeforeRender = function() {
        BABYLON.clipPlane = new BABYLON.Plane(0, 1, 0, 0);
    };

    this.refractionTexture.onAfterRender = function() {
        BABYLON.clipPlane = null;
    };

    this.waterColor = new BABYLON.Color3(0.0, 0.3, 0.1);
    this.waterColorLevel = 0.2;
    this.fresnelLevel = 1.0;
    this.reflectionLevel = 0.6;
    this.refractionLevel = 0.8;
    
    this.waveLength = 0.1;
    this.waveHeight = 0.15;

    this.waterDirection = new BABYLON.Vector2(0, 1.0);

    this._time = 0;
};

The bump texture is a standard texture:

image

The reflection texture is created from a BABYLON.MirrorTexture which is able to simulate a mirror (exactly what we need!).

The refraction texture is based on a BABYLON.RenderTargetTexture. This kind of texture can receive the rendering of a scene and can then be used as standard texture resource for a shader. With the onBeforeRender and onAfterRender functions you can configure and restore back the scene for your rendering (In this case, I’ve just activated a clipping plane to limit the refraction to the objects over the water).

Then we need to add a new function to the WaterMaterial in order to register these two special textures:

WaterMaterial.prototype.getRenderTargetTextures = function () {
    var results = [];

    results.push(this.reflectionTexture);
    results.push(this.refractionTexture);

    return results;
};

This function is mandatory for us, because we need to have our textures prepared before using our shaders.

The link with the shaders itself is done with the isReady function:

WaterMaterial.prototype.isReady = function (mesh) {
    var engine = this._scene.getEngine();
    
    if (this.bumpTexture && !this.bumpTexture.isReady) {
        return false;
    }

    this._effect = engine.createEffect("Water/water",
        ["position", "normal", "uv"],
        ["worldViewProjection", "world", "view", "vLightPosition", "vEyePosition", "waterColor", "vLevels", "waveData", "windMatrix"],
        ["reflectionSampler", "refractionSampler", "bumpSampler"],
        "");

    if (!this._effect.isReady()) {
        return false;
    }

    return true;
};

The engine object has a function called createEffect that you can use to compile/link your shaders into a simple object. This function has the following parameters:

  • An array of attributes describing the topology of your vertices
  • An array of uniforms (the external variables) defined by the shaders
  • An array of samplers (the objects used to read textures)
  • An optional define string

The createEffect has an internal cache in order to compile/link your shaders only once (the subsequent calls return directly the cached effect).

Once the effect is created we can use it to transfer values to the shaders within the bind function:

WaterMaterial.prototype.bind = function (world, mesh) {
    this._time += 0.0001 * this._scene.getAnimationRatio();

    this._effect.setMatrix("world", world);
    this._effect.setMatrix("worldViewProjection", world.multiply(this._scene.getTransformMatrix()));
    this._effect.setVector3("vEyePosition", this._scene.activeCamera.position);
    this._effect.setVector3("vLightPosition", this.light.position);
    this._effect.setColor3("waterColor", this.waterColor);
    this._effect.setFloat4("vLevels", this.waterColorLevel, this.fresnelLevel, this.reflectionLevel, this.refractionLevel);
    this._effect.setFloat2("waveData", this.waveLength, this.waveHeight);

    // Textures        
    this._effect.setMatrix("windMatrix", this.bumpTexture._computeTextureMatrix().multiply(
BABYLON.Matrix.Translation(
this.waterDirection.x * this._time, this.waterDirection.y * this._time, 0))); this._effect.setTexture("bumpSampler", this.bumpTexture); this._effect.setTexture("reflectionSampler", this.reflectionTexture); this._effect.setTexture("refractionSampler", this.refractionTexture); };

Please note that I use the animated _time variable to generate the windMatrix variable in order to simulate waves movement.

Finally do not forget to clean things up when leaving Sourire

WaterMaterial.prototype.dispose = function () {
    if (this.bumpTexture) {
        this.bumpTexture.dispose();
    }
    
    if (this.groundTexture) {
        this.groundTexture.dispose();
    }

    if (this.snowTexture) {
        this.snowTexture.dispose();
    }
    this.baseDispose();
};

Using our material

Now we have a specific material to simulate the water, we can go back to our HTML page. We have to attach the WaterMaterial to the water object and specify which meshes are used by the reflection and the refraction textures:

// Water
BABYLON.Engine.ShadersRepository = "";
var water = BABYLON.Mesh.CreateGround("water", 1000, 1000, 1, scene, false);
var waterMaterial = new WaterMaterial("water", scene, sun);
waterMaterial.refractionTexture.renderList.push(extraGround);
waterMaterial.refractionTexture.renderList.push(ground);

waterMaterial.reflectionTexture.renderList.push(ground);
waterMaterial.reflectionTexture.renderList.push(skybox);

water.material = waterMaterial;

One important thing to note here: The BABYLON.Engine.ShadersRepository is used to specify the path to shaders’ folder. For this demo I did not use one so we have to set it to empty string.

You are now ready to see your wonderful world with its so cute moving water Sourire

Going further

If you want to go more deeply into babylon.js, here are some useful links:

Leave a Comment
  • Please add 1 and 3 and type the answer here:
  • Post
  • Can you please link your full code ? Because in the HTML you're loading 2 js script and i don't realy know what to put in each to have the same texture result. Thanks for you awesome job ! ;)

  • Here they are:

    github.com/.../babylon.js

    https://handjs.codeplex.com/

  • No, i was talking about the sky pictures you link a screenshot of them not all the 6 pictures

  • You can grab them here:

    github.com/.../skybox

  • Nice if it worked but WaterMaterial.prototype.needAlphaBlending results in "Uncaught TypeError: Object [object Object] has no method 'needAlphaBlending'"

    Likely cause, it doesn't exist anymore and documentation is now out of date! :(

    So this whole tutorial is now moot and I can't duplicate the results.

  • Could you share the code you used?

  • I used the code above, found a couple errors in what I did because it wasn't so clear about the water.vertex.fx file and such, no where does it say to place it in such files, I found out using the live demo and even after downloading the live demo files including that one and also water.fragment.fx as well as your water.js and using it as you have it online, I get Uncaught Error: 0  in Babylon.js line 1, lol which tells me absolutely nothing and it doesn't show the water at all but shows the rest of the scene, so I am using identical code for the water with the water.js in the water folder with bump map and still no dice.

    http://pastebin.com/ABqyUiu1 => HTML File

    http://pastebin.com/48EYC3wf => water.js

  • All the code is There:

    github.com/.../WorldMonger

  • Sorry it didn't help me out, but thanks for trying, I'll figure it out, but Babylon should get a built in water system, most engines I have seen have them to some extent, but this isn't deterring me, I am sold, just am stumped on this water issue..

  • It could be related to .fx file not being served by your server. COuld you try by adding .fx MIME type tou your web server?

  • Am running it from my desktop while developing as local file using Chromes --allow-file-access-from-files so that cross domain warnings don't come up and file linking is allowed.

  • Could you just check that the .fx are well loaded? Using f12 tools

  • OI, they do not appear in Frames>ws.html anywhere, not in root or in scripts.. was hoping to avoid using a server to do this stuff, lol, looking like they aren't loading in so will have to go the server route..

  • Sorry about That:(

  • Success! Had to upload to my server, so you were right, wasn't loading in the fx files, cheers!

Page 1 of 1 (15 items)