Most of the stunning visual effects you see everyday on the web or in movies and games would not be possible without GPUs. These Graphical Processing Units with their thousands of cores running in parallel are incredibly fast, but can be tricky to program at first.
In this post we're going to look at how to set up Pixi.js, one of the most widely used Javascript rendering libraries, to run shaders on the web. It's built on top of WebGL and takes care of all the setup and cross-platform work. I won't be going into a lot of depth, but hopefully just enough to get you started.
So what is a shader?
A shader is just a piece of code that runs on the GPU. This piece of code takes as input a pixel on the screen, and returns a single color. Thanks to the GPU, the same piece of code can run on every pixel on the screen at the same time.
Here's an amusing video to show you what that would look like in real life:
Just to get an idea of how big a difference this parallelization makes: imagine that it takes the CPU 0.1 seconds to draw a single pixel. This means it would take 13 hours to draw an image with dimensions of 800 x 600. On the other hand, if it takes the GPU 0.1 seconds to draw a single pixel, it would take 0.1 seconds to draw the whole thing.
Of course these are just made up numbers, but that's essentially why GPUs are so much faster at tasks that can be run in parallel (such as graphics processing).
Setting up Pixi
Before we can write our first shader, we have to set up our rendering environment. Create a new html file and include Pixi.js:
<html>
<head>
<title>PixiJS Shaders</title>
</head>
<body>
<script type="text/javascript" src="https://cdn.jsdelivr.net/pixi.js/3.0.7/pixi.js"></script>
<script>
//Our code will go here
</script>
</body>
</html>Rendering an image
Now we'll just add some boilerplate code to set up the scene, load an image and render it. The code below is commented to explain what each line is doing. Add it to your html in the script tag we created:
// Get the screen width and height
var width = window.innerWidth;
var height = window.innerHeight;
// Chooses either WebGL if supported or falls back to Canvas rendering
var renderer = new PIXI.autoDetectRenderer(width, height);
// Add the render view object into the page
document.body.appendChild(renderer.view);
// The stage is the root container that will hold everything in our scene
var stage = new PIXI.Container();
// Load an image and create an object
var logo = PIXI.Sprite.fromImage("http://www.goodboydigital.com/pixijs/pixi_v3_github-pad.png");
// Set it at the center of the screen
logo.x = width / 2;
logo.y = height / 2;
// Make sure the center point of the image is at its center, instead of the default top left
logo.anchor.set(0.5);
// Add it to the screen
stage.addChild(logo);
function animate() {
// start the timer for the next animation loop
requestAnimationFrame(animate);
// this is the main render call that makes pixi draw your container and its children.
renderer.render(stage);
}
animate()You should see the image appear in the center of the screen when you open the file in your browser! You can try loading in any other image by putting in the url of the image you'd like in the PIXI.Sprite.fromImage() function.
Note: if you're working locally, you won't be able to load an image directly off your filesystem (Javascript prevents this for security reasons). You'll have to set up a local web server (the easiest way to do this is with Python ). If you don't want to bother with this, you can still load images as long as they're uploaded somewhere online.
Writing a shader
Shaders are written in GLSL (OpenGL Shading Language). This is a special language designed to run on the GPU. It looks and feels very much like C. Shaderific.com has a really nice reference page for the language.
We will need to write our GLSL code and pass it to Pixi as a string. We can do this in any number of ways. We can:
- Write it as a big string inside of our Javascript code
- Write it inside of a hidden HTML tag and grab the contents with our Javascript
- Write it in separate file and load the contents of that file with Javascript
Since Javascript doesn't support multiline strings, the first option isn't very convenient. We can write our GLSL code inside of a bogus script tag. Here's a simple example:
<script id="shader" type="shader">void main(){ gl_FragColor = vec4(1.0,1.0,1.0,1.0); }</script>
Because "shader" isn't a valid script type, the browser will just ignore this tag. We can then grab the code and apply it to our object in Pixi like this:
//Get shader code as a string
var shaderCode = document.getElementById("shader").innerHTML
//Create our Pixi filter using our custom shader code
var simpleShader = new PIXI.AbstractFilter('',shaderCode);
//Apply it to our object
logo.filters = [simpleShader]And you have a shader running! Here is a working CodePen of what we have so far.gl_FragColor is the color of the pixel to be returned. It's of type "vec4", which is as you'd expect, a vector of 4 numbers. (The first argument that we're leaving blank is a vertex shader, another type of shader that we won't go into in this article).
Grabbing data from the shader
Remember that all a shader needs to do is return a color. The color is returned as 4 numbers: red, green, blue and alpha. The numbers are capped between 0 and 1. In this case, we're returning white. (Just to see it for yourself, try changing it so it's red instead).Since this shader is running for every pixel, everything will be the same color. That's not very exciting. We have access to the pixel the shader is running on through the built in variable gl_FragCoord. These are the coordinates (x,y) of the pixel on screen. We can create a gradient effect by doing something like:
void main(){
gl_FragColor = vec4(gl_FragCoord.x/1000.0,0.0,0.0,1.0);
}A good way to understand how this works is to just pick out a few pixels on screen and think about what the output would be. A pixel on the left edge of the screen has a gl_FragCoord.x value of 0. So its color ends up being black. On the right edge, it has a value of whatever the width of your screen is, and assuming that's somewhere around 1000, it will be red.
Note: GLSL is strict about variable types. If you try to divide by 1000 instead of 1000.0 it will give you an error in the console that you can't divide a float by an int.
Another important thing to know is that there are two ways of accessing the components of a vector in GLSL. If you have a vector with 4 numbers:
vec3 someVector = vec3(0.0,0.5,1.0,2.0);Then you can access its components using x,y,z and w.
someVector.z; //This is 1.0
someVector.w; //This is 2.0This is the same thing as accessing them through r,g,b and a. So:
someVector.b; //Still 1.0
someVector.a; //Still 2.0 Both are equivalent, but the convention is to access using xyzw when the vector represents a point or a position, and rgba when it represents a color.
Sending custom data
We're not limited to just the pixel's coordinates as data. In fact you can send anything you want to your shader. You can do this using uniforms. A uniform variable is just a variable that you set from the CPU. (It's so called because its value is the same for every pixel, unlike something like gl_FragCoord).
Let's say we wanted to send a counter to keep track of time. We first create a uniforms object (in our original Javascript):
var uniforms = {}
uniforms.time = {
type:"f",
value:0
}Every uniform must have a type and a value. The type f translates to "float" in GLSL. v2 would translate to vec2 and so on.
Now in the line where we create our shader, we can pass the uniforms object as our third argument:
var simpleShader = new PIXI.AbstractFilter('',shaderCode,uniforms);And finally, we need to declare it in our actual shader. So it should look something like this:
//Define the precision of floating numbers so GLSL doesn't complain
precision mediump float;
uniform float time; //Declare that we're using this uniform
void main(){
gl_FragColor = vec4(sin(time),0.0,0.0,1.0);
//I'm using sin(time) to make it oscillate as time increases
}Now you won't see anything change, because the value of time is just always 0. Try incrementing it inside the animate function:
uniforms.time.value += 0.1;You should see the color changing over time! Here's a CodePen example sending time, mouse position as well as screen resolution.
Rendering an image
So far we've just been generating colors on the fly. In a lot of cases, you want to process an already existing image instead of creating one from scratch. Pixi passes some built in variables for us. These are vTextureCoord and uSampler. The former is a vec2 of the position of the current pixel, and the latter is the image data. We can use these to render the image:
precision mediump float;
varying vec2 vTextureCoord;//The coordinates of the current pixel
uniform sampler2D uSampler;//The image data
void main(void) {
gl_FragColor = texture2D(uSampler, vTextureCoord);
}Now this is exactly what we had originally. The function texture2D takes a point and an image, and returns the color of the pixel in the image at that point. If we managed to get the color, we don't have to stop there. We can add a line to remove all red from the image:
gl_FragColor = texture2D(uSampler, vTextureCoord);
gl_FragColor.r = 0.0;Once you have the image color, you can really do anything with it. You can set the blue value to 1 and give everything a blue tone, or swap out the green and the red, or blend two colors together. (In fact, this is how blend modes are made!)
Here is a working example of removing the red from the image.
Reverse engineering shaders
There's definitely a lot more to learn about shaders, but knowing how to set a pixel's color, render an image and send custom data covers all the basics you'd need to be able to pick apart any shader you see on the web.
ShaderToy is an awesome resource for shaders. A great way to learn would be to just try and break down a shader and understand what it's doing. There are minor differences between the code on ShaderToy and the conventional GLSL shader code.The main function, for example, is called mainImage on ShaderToy. gl_FragColor and gl_FragCoord are called fragColor and fragCoord respectively. ShaderToy also does not let you set our own uniforms, but they have a bunch provided under the "shader inputs" tab.
As a final example to get you started, here is the smoke shader that appears above:
There's a lot going on here, but the first thing I always do is find the line that sets the final color. In this case, it's:
gl_FragColor = vec4(c * cos(shift * gl_FragCoord.y / resolution.y), 1.0);
gl_FragColor.xyz *= 1.0-grad;One really easy thing to do is to remove the blue value, which we already know how to do. Doing that will give you a really snazzy looking fiery smoke.You can keep trying to play around with parameters and numbers to see how it works. Another fun thing to try is to change this factor:
float q = fbm(p - time * 0.1);What happens if you change 0.1 to 1.0? Or higher?I hope this article was a nice foray into the world of GPU programming!