Minko - Work with vertex attributes in the fragment shader

In this tutorial we will present how to pass values computed in the vertex shader to the fragment shader. This a very common operation and any non-trivial shader is more than likely to use it. The simplest use case is when one wants to use the vertex components - usually its texture coordinates or its color - in the fragment shader.

To understand how we can use vertex attributes in the fragment shader, we will take the example of per-vertex color and see how to create both the mesh and the shader to render the picture above.

Step 1: The triangle mesh

Here, we will work with a simple triangle that we will draw directly in clip space. Each vertex of the triangle will have two components:

A clip space 2D position
An RGB color

The "clip space" is the normalized 2D space of the screen: the x and y coordinates varies between -1 and 1. In 3D, we use the clip space mainly because it is normalized and as such every position is resolution independant. If our vertices were defined in screen space, we would have to change and upload our geometry everytime the resolution changes!

Now the format of our vertices is defined, we just have to create a VertexStream, fill it with the 3 vertices of our triangle and create a Mesh with it. It can be done in 3 simple steps:

Create the VertexFormat object that defines our vertices.
Create an empty VertexStream with the proper format to store our vertices.
Create and use a VertexIterator to make it easier to fill the VertexStream.

This is how it is done:


private function initializeTriangle() : IMesh
{
  var format : VertexFormat = new VertexFormat(VertexComponent.XY, VertexComponent.RGB);
  var vstream : VertexStream = new VertexStream(null, format);
  var vertices : VertexIterator	= new VertexIterator(vstream);

  vertices[0] = {x: 0.,	 y: .5,  r: 1., g: 0., b: 0.};
  vertices[1] = {x: -.5, y: -.5, r: 0., g: 1., b: 0.};
  vertices[2] = {x: .5,  y: -.5, r: 0., g: 0., b: 1.};

  return new Mesh(vstream);
}

There are multiple ways to create and fill a VertexStream. If you want to learn more vertex iterators, you can read the "Vertices and triangles iterators" technical article.

Step 2: The Vertex Shader

The vertex shader is supposed to output a clip space 2D position for each 3D vertex. In this case, our geometry is a simple triangle with each vertex directly defined in clip space. Therefore, there is no transformation to apply in our vertex shader: we just have to return the position stored in the vertex.

The getVertexAttribute() method returns the data read in the vertex. The only argument for this method is the VertexComponent object that represents the component we want to read.


private var _color : SValue = null;

override protected function getOutputPosition() : SValue
{
  // use float2() to make sure we read a value with size = 2
  var xy : SValue = float2(getVertexAttribute(VertexComponent.XY));

  // store the vertex RGB color in a private member to make it
  // available to the fragment shader
  _color = getVertexAttribute(VertexComponent.RGB);

  // return the position of the vertex unchanged, with dummy values for z and w
  return float4(xy, 0., 1.);
}

The vertex shaders read two vertex attributes: the (x, y) 2D clip space position and the RGB value of the vertex. The color is stored in a private member to make it available to the fragment shader and the 2D position is directly returned as the final clip space position.

The following figure shows the output of the vertex shader for our clip space triangle:

Step 3: The Fragment Shader

Here, we simply want to use the color of each vertex. But it is a bit more complicated. Indeed, we have per-vertex colors, but we are trying to compute/output a per-pixel value.

To do this, we will use the interpolate() method. This method takes any value available in the vertex shader - such as the vertex attributes - and makes it available in the fragment shader. But the per-vertex value is not provided as is: it is linearily interpolated to provide consistent per-pixel data.

Thus, we just have to call the interpolate() method on the RGB value stored by the vertex shader:


override protected function getOutputColor() : SValue
{
  return interpolate(_color);
}

Instead of getVertexAttribute(VertexComponent.RGB), you can use the vertexRGB protected property.

Any value coming from the vertex shader must be interpolated using the interpolate() method in order to be used in the fragment shader. Otherwise, the following error will be thrown:

Unable to compile shader: an allocation is missing. There might be a missing vertex to fragment shader interpolation.

Conclusion

Here is what the final program looks like:


public class Main extends Sprite
{
  private var _viewport : Viewport = new Viewport();
  private var _scene : IScene = null;

  public function Main()
  {
    _scene = initializeTriangle();

    _viewport.defaultEffect = new SinglePassRenderingEffect(new RGBShader());

    stage.addChild(_viewport);
    stage.addEventListener(Event.ENTER_FRAME, enterFrameHandler);
  }

  private function initializeTriangle() : IMesh
  {
    var format : VertexFormat = new VertexFormat(VertexComponent.XY, VertexComponent.RGB);
    var vstream : VertexStream = new VertexStream(null, format);
    var vertices : VertexIterator = new VertexIterator(vstream);

    vertices[0] = {x: 0.,  y: .5,  r: 1., g: 0., b: 0.};
    vertices[1] = {x: -.5, y: -.5, r: 0., g: 1., b: 0.};
    vertices[2] = {x: .5,  y: -.5, r: 0., g: 0., b: 1.};

    return new Mesh(vstream);
  }
  
  private function enterFrameHandler(event : Event) : void
  {
    _viewport.render(_scene);
  }
}

public class RGBShader extends ActionScriptShader
{
  private var _color : SValue = null;

  override protected function getOutputPosition() : SValue
  {
    var xy : SValue = float2(getVertexAttribute(VertexComponent.XY));

    _color = getVertexAttribute(VertexComponent.RGB);

    return float4(xy, 0., 1.);
  }

  override protected function getOutputColor() : SValue
  {
    return interpolate(_color);
  }
}

Work with vertex attributes in the fragment shader

Introduction

Step 1: The triangle mesh

Step 2: The Vertex Shader

Step 3: The Fragment Shader

Conclusion