WebKit, and Filters, and Shaders! Oh, my!

CSS Shaders are a new and relatively easy way to bring cinematic effects to the web.  You can find out all about them on the ADC, on the Adobe & HTML site or you can read the actual W3 draft proposal. While there are many articles out there showing how to use shaders, actually building your own custom shader is another story. That’s what we’re going to do here.

One caveat though – CSS filters and shaders are not completely mainstream yet.  Filters need a WebKit nightly or at least Chrome 18.0.976.0 to be seen.  While there is CSS shaders support available in recent versions of Chrome (disabled by default), the demonstrations below use the latest notations and restrictions (more on that later).  If you want to see the demos live, you’ll need to get Adobe’s WebKit prototype from  https://github.com/adobe/webkit/downloads. I’d recommend grabbing that so you can play around with it yourself.

Note – This page will auto-detect your browser’s capabilities and will show you a video if you don’t have support for the demo.  If your browser does support the necessary feature, you’ll be able to see the demo inline instead.

Filters First

Filter effects let you declare image effects on various part of your HTML.

In the following video we see that when I mouse over the image and text, a sepia filter is applied.

Mouse over the image below and it should have a sepia filter applied to it:

freefoto.com

The way we did that was with the following css style:

#img1:hover { -webkit-filter: sepia(100%);  }

When you hover over img1, the sepia filter is applied to the image. But filters can be applied to any HTML. We see below how easy it is to blur some text.

Mouse over the text below and it should blur slightly.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer nisi metus, sollicitudin non auctor ac, vehicula vitae libero. Quisque non mauris sapien, eget blandit eros. Donec id metus ac metus convallis sagittis. Donec nec est id erat sodales pellentesque. Ut vel lorem quis tellus rutrum dictum. Nunc tempor nisi at dolor ullamcorper in viverra risus fringilla. Integer tincidunt tincidunt vestibulum. Maecenas tincidunt, lorem id commodo ultricies, metus quam vulputate tortor, in porttitor dui mi pharetra orci. Nunc pellentesque massa sit amet lectus iaculis feugiat pretium odio venenatis.

That was done simply with the following css:

#text1:hover {-webkit-filter: blur(2px); }

It’s pretty straightforward. There are all kinds of filters available, including:

  • grayscale(amount)
  • sepia(amount)
  • saturate(amount)
  • hue-rotate(angle)
  • invert(amount)
  • opacity(amount)
  • brightness(amount)
  • contrast(amount)
  • blur(radius)
  • drop shadow(shadow)

Prefixes

If you’ll notice above, we used -webkit-filter, not simply filter.  I used the -webkit prefix because that is what is working right now.  Ultimately, for compatibility you should declare the filter multiple times, one with each vendor prefix, as in:

#myDiv {
    -webkit-filter: <Google Chrome & Safari filter definition>;
    -moz-filter: <Firefox filter definition>;
    -ms-filter: <Internet Explorer filter definition>; 
    -o-filter: <Opera filter definition>; 
}

All of the examples here just use the -webkit filter to keep the examples simple and to make it as straightforward as possible for you to get the examples working on your own.

Now what if we want to extend filters and create or reuse a custom filter?  That’s where shaders come in.

CSS Shaders

There are 2 types of shaders:

  • Fragment shaders - Also known as pixel shaders, operate on pixel color values.  A typically small parameterized program calculates the color value of each pixel of the declared content.
  • Vertex shaders - Operate on point coordinates (vertices).  The content is broken up into a grid of vertices based on the resolution declared in the CSS.  A similarly small parameterized program is used to transform each vertex in 3D space.  The result is then projected back onto a 2D plane and rendered in the browser.

Shaders are declared using the same -webkit-filter css style but use a custom filter, as in:

-webkit-filter: custom(<vertex shader> [<fragment shader>] 
    [, <vertex mesh>][, params...])

For example, you could have something like:

-webkit-filter: custom(url(shaders/myvshader.vs)
    url(shaders/myfshader.fs), 20 20, amt 1)

where 20 20 defines the mesh resolution for the Vertex shader and “amt 1″ is a parameter passed in to the shaders. If you don’t want one of the shaders, just specify none instead, or you can leave the fragment shader out altogether if you want:

-webkit-filter: custom(url(shaders/myvshader.vs), amt 1)

or

-webkit-filter: custom(none url(shaders/myfshader.fs))

Shader programs are written using the OpenGL ES shading language.  That’s right,  OpenGL.  WebGL has the same concept of shaders and also uses the OpenGL ES shading language. Ok, let’s have some fun here.  Remember, if you want to see the demos running inline, you need to be running the WebKit prototype.  If not, you’ll just see videos below:

Simple Fragment Shader

A Nice Tree

freefoto image

Image courtesy of freefoto.com

Notice when the mouse moves over the text and image, it slowly turns to grayscale. This is achieved with a combination fragment shader and css transition. In this case we have a div which contains some text and images.  The div is declared with a class attribute named “shader”.  The name itself doesn’t matter.  It’s just a way of identifying the div.  As with filters, the declaration lies in the css styling:

.shader{
    -webkit-filter: custom(none url(shaders/grayscale.fs), amount 0);
    -webkit-transition: -webkit-filter linear 1s;
 }

 .shader:hover{
     -webkit-filter: custom(none url(shaders/grayscale.fs), amount 1);
 }

In this case there is no vertex shader, but there is a fragment shader.

Note that we didn’t have to use a custom shader to achieve the grayscale effect.  We could have simply used a standard filter, with something like:

.shader { filter: grayscale(0); transition: filter 2s linear;}
.shader:hover {filter: grayscale(1); }

But I wanted to show you a simple example so we could better dissect it.  Let’s take a look at grayscale.fs:

precision mediump float;

// This uniform value is passed in using CSS.
uniform float amount;

void main()
{
 float oneMinusAmount = 1.0 - amount;
 css_ColorMatrix = mat4(
     (0.2126 + 0.7874 * oneMinusAmount),
     (0.7152 - 0.7152 * oneMinusAmount),
     (0.0722 - 0.0722 * oneMinusAmount),
     0.0,

     (0.2126 - 0.2126 * oneMinusAmount),
     (0.7152 + 0.2848 * oneMinusAmount),
     (0.0722 - 0.0722 * oneMinusAmount),
     0.0,

     (0.2126 - 0.2126 * oneMinusAmount),
     (0.7152 - 0.7152 * oneMinusAmount),
     (0.0722 + 0.9278 * oneMinusAmount),
     0.0,

     0.0, 0.0, 0.0, 1.0);
}

First we set the required precision qualifier.   This declares a minimum range and precision the underlying shader engine must use when storing variables.

precision mediump  float

In this case it means that floats should be stored with the minimum range and precision required for the fragment language.

uniform float amount;

Uniforms are variables passed in from the web browser to the shaders.  In the case of this grayscale fragment shader, the same amount will be passed in for every pixel color calculation.  Notice how we declared the amount in the css:

-webkit-filter: custom(none url(shaders/grayscale.fs), amount 1);

If we had declared a vertex shader, the amount uniform would have been passed into that as well.

The other variable type worth noting here is varying.  Any variables of type varying will be passed from one shader to another.  A variable set in a vertex shader can then be passed in to a fragment shader.  An example of this would be determining the 3D shape of  something in the vertex shader, and then using those coordinates to determine shading in the fragment shader.

Now, notice in grayscale.fs how we don’t explicitly calculate a new color.  Instead we calculate a color matrix – css_ColorMatrix.  The color matrix is actually pre-multiplied against each color value to calculate new colors, as in:

| R' |     | a00 a01 a02 a03 a04 |   | R |
| G' |     | a10 a11 a12 a13 a14 |   | G |
| B' |  =  | a20 a21 a22 a23 a24 | * | B |
| A' |     | a30 a31 a32 a33 a34 |   | A |
| 1  |     |  0   0   0   0   1  |   | 1 |

You can read more details on the color matrix from the current filter spec draft.

You may be thinking at this point, ‘Why don’t we simply calculate a new color?  The shader language even has the notion of gl_FragColor.  That seems to be a more straightforward and powerful option.”  Well, it turns out there are some potentially serious security concerns for shaders…

Shaders & Security

Shaders are intended to be reusable filter components.  It’s quite possible that there will be a situation where a shader might not be sourced from the same trusted domain as the content upon which it is operating.

Here’s a crazy thought – write a shader that, depending on the color of each pixel, takes more or less time to execute.  Then, write some nice little bit of JavaScript that measures the execution time of the shader and, here’s the really fun part, infers potentially protected data from that rendered content.  Is that even possible?

It turns out unfortunately that this is not such a far-fetched idea as one would hope.  This has been prototyped and proven to work in WebGL.  As a result, there is a lot discussion going on at present trying to determine the safest restrictions that can be imposed on shaders that still leave shaders as an extremely valuable tool in your design belt.  The current proposal is to restrict any access to the original content in the filter.   In the case of a fragment shader, instead of taking the current color and applying some calculation on it, we calculate a transform that the filter engine will use to calculate the colors on its own.  It’s still extremely useful; just a different way of thinking about things.

And now back to our previous shader development…

I have to admit, that last grayscale example was a little boring.  We just did in the custom filter what we easily could have done with a standard grayscale filter.  But what if we want to change the color matrix for every pixel rendered?  Aha!  Now things start to get interesting. Let’s look at this fragment shader:

precision mediump float;

uniform float amount;
uniform float resX;
uniform float resY;

void main( void ) {
    float dx = (resX / 2.0) - gl_FragCoord.x;
    float dy = (resY / 2.0) - gl_FragCoord.y;

    float k = (sin(amount * 0.4) + 1.0) * 0.5;
    float d = (dx * dx + dy * dy) * k;

    float r = (sin(amount + d * 0.029) + 1.0) * 0.5;
    float g = (sin(amount * 1.4 + d * 0.03) + 1.0) * 0.5;
    float b = (sin(amount * 10.0 + d * 0.03) + 1.0) * k;
    float a = (sin(amount * 5.0 + d * 0.03) + 1.0) * 0.5;

    css_ColorMatrix = mat4(
        vec4(r, 0.0, 0.0, 0.0),
        vec4(0.0, g, 0.0, 0.0),
        vec4(0.0, 0.0, b, 0.0),
        vec4(0.0, 0.0, 0.0, a)
    );
}

You can find the actual filter here.

This filter is an adaptation of this WebGL filter, found on the GLSL Sandbox.  I highly recommend checking out this site if you want to see some of the interesting things that can be done with shaders.

Like the grayscale filter, we’re still generating a css_ColorMatrix.  But, in this instance we perform a series of calculations based on gl_FragCoord.  gl_FragCoord is a vec2 available to all fragment shaders that contains the window relative x & y coordinates of the current color pixel being transformed.

The CSS which declares the shader is as follows:

@-webkit-keyframes myanim {
    from {
        -webkit-filter:
            custom(none url(shaders/shader2.fs), amount 400, resX 1300, resY 100);
    }

    to {
        -webkit-filter:
            custom(none url(shaders/shader2.fs), amount 405, resX 800, resY 1400);
    }
}

.shader2:hover {
    -webkit-animation: myanim 4s ease 0s infinite alternate;
}

Note how we use a CSS animation to animate the input parameters to the shader.  And here are the results:

As always, mouse over below to get the animation going…

Nice Tree

freefoto.com

That is some crazy stuff there, and we’ve only scratched the surface.

The next series of articles will focus more on vertex shaders and we’ll start to dig deeper into the shading language itself.  Please stay tuned – I hope to have some more fun articles shortly.

Tagged , , , , , . Bookmark the permalink.

8 Responses to WebKit, and Filters, and Shaders! Oh, my!

  1. Randall Bennett says:

    So if I wanted to just show an image straight, with no premultiplication, I’m going to use a fragment shader and some sort of color matrix action… but I’ve been testing and can’t find the right combo to get the straight version.

    Any guidance? When I’ve previously done this in Quartz composer, I believe I used a fifth value called a “bias” value. Is there some way to use this in your shader examples? (Forgive me, I’m a total graphics noob.)

    • agreenblatt says:

      If you want to just show the image straight with no premultiplication, you could either turn off the shader or use an identity matrix. Say for instance you only wanted the shader to take effect when the user’s mouse hovered over the image. You could then simply apply a style as follows:

      #myImg:hover {
      -webkit-filter:
      }

      By default there would be no shader and those no premultiplication. You could also just set the -webkit-filter property with some JavaScript so it only applies when you decide is an appropriate time.

      But, if you want to use a shader but as you say you want the image to look completely normal, you could use an identity matrix:

      1 0 0 0
      0 1 0 0
      0 0 1 0
      0 0 0 1

      You can create an identity matrix as follows:

      css_ColorMatrix = vec4(1.0);

      If a single scalar value is passed to a vec4 constructor, it is applied to all the components of the matrix’s diagonal, with all the other values being 0.
      You can take a look at how vec4′s are initialized here: http://www.khronos.org/files/opengles_shading_language.pdf
      Look in section 5.4.2. Vector and Matrix Constructors.

      If you’re testing out your shader, it’s also a good idea to disable your browser cache so your changes are always picked up.

      • Randall Bennett says:

        Sorry, to be clear…

        I don’t want it to look normal, I instead want it to look “ugly”. :)

        So I’m building something specifically targeted toward video, and I need to separate out the alpha channel (as luminance, which I’ve done successfully), and a “straight” version of the image, which has no alpha channel.

        Example: https://dl.dropbox.com/u/4031469/shadertest.html

        I’m so close, just trying to figure out the last shader.

        Here’s an example of what I’d expect to see. (This is from a Targa file)

        https://dl.dropbox.com/u/4031469/Randalls%20Mess.png – the fill (what I haven’t figured out)

        https://dl.dropbox.com/u/4031469/Randalls%20Mess%20Alpha.png – the key (aka alpha which I have figured out)

        https://dl.dropbox.com/u/4031469/Randalls%20Mess.tga – The final output targa.

        Does this make sense?

        • agreenblatt says:

          Unfortunately I don’t think is possible. I’ll try to post a more detailed answer shortly.

          • agreenblatt says:

            Currently we do not have a ‘bias’ or ‘translation’ that can be added to a computed color. Even though you say you have the luminance worked out, it’s not completely correct since your output is I believe:

            r' = a;
            g' = a;
            b' = a;
            a' = r + g + b + a;

            and I think you want something like:

            a' = 1;

            Strictly speaking, it should be the inverse of the matrix at:

            http://www.w3.org/TR/SVG/filters.html#feColorMatrixElement

            but that’s not possible since we only have a 4×4 matrix.

            I think what you’re looking to do is unmultiply the alpha from the color components and set alpha to 1. Unfortunately, that’s not possible with css_ColorMatrix.

          • Randall Bennett says:

            OK so… shaders aren’t the way to go then… but SVG filters are? Is it unlikely that bias / translations will ever be added because of the timing attack stuff?

            And if they are, is there a timeline for Adobe supporting their inclusion in Webkit? Last time I checked (admittedly six or more months ago) it was my understanding that they were only supported in Firefox. Feel free to correct me if I’m wrong. :)

            Oh and also: Thanks for being so helpful with these shaders! It’s a fun little experiment even if I fail.

  2. Pingback: Fun with 3D and CSS Shaders - Alan Greenblatt

  3. Pingback: CSS Custom Filters | Pranav's space

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>