Client-side image scaling on Ge.tt with HTML5

This week we introduced a new feature to Ge.tt, client side image scaling using new HTML5 technologies.

On Ge.tt we use several representations of an image. A small thumb in the albums overview, a larger thumb in the album itself, a scaled version in the file window, and finally the original.

We use the scaled versions so that we don't have to fetch the original image, which can be several megabytes large, every time and thereby degrade site responsiveness.

Scaling images is generally very CPU heavy. Not only that, but the user can't see the entire album until all images are uploaded, and then the thumbnails generated.

Newer browsers, like Firefox, Chrome and IE10, have introduced new JavaScript APIs which make building full scale web apps a lot easier. Some aren't standardized yet and others aren't compatible across browsers. A great number of our users use browsers which support the required technologies, so we decided that this feature was worth adding.

What follows is a description of the methods used to implement client side image scaling.

HTML5

We used a number of technologies, including canvas DOM elements, Blob objects and File objects. In our example we are going to use the following HTML snippet.

<canvas id = 'thumb-canvas'> 
<img id = 'thumb-image'> 
<input type = 'file' id = 'file-input'>

We receive a file object after the user has selected an image in the file dialog. Then a URL is created using the window.URL interface and the image is loaded with the Image object.

Now we can read the dimensions of the image, and scale it to the desired size while keeping the aspect ratio. By using the 2D canvas rendering context and its overloaded drawImage method we are able to draw the image on the canvas using the scaled dimensions.

As a last step we can obtain the image data with the toDataURL method of the canvas object and display it in an img tag. Here is the full example.

var fileInput = document.getElementById('file-input'); 
var thumbCanvas = document.getElementById('thumb-canvas'); 
var windowURL = window.URL || window.webkitURL;

fileInput.onchange = function() {
    // Get the first file object1
    var file = fileInput.files[0];

    if(!file) {
        return;
    }

    // Create url for file object. The lifespan of the url is tied to this document.
    var url = windowURL.createObjectURL(file);
    var context = thumbCanvas.getContext('2d');
    varimage = newImage();

    image.onload = function() {
        // Target dimensions as an object { width: x, height y }
        var dimensions = getDimensions();
        // Scale dimensions while keeping the aspect ratio. In this example the image is scaled
        // to be small as possible, while still filling out the entire target area.
        var scaledDimensions = scale(image, dimensions.width, dimensions.height);

        // Set the canvas to the desired size
        thumbCanvas.setAttribute('width', scaledDimensions.width);
        thumbCanvas.setAttribute('height', scaledDimensions.height);

        // Draw the image on the canvas
        context.drawImage(image,0,0, scaledDimensions.width, scaledDimensions.height);
        // Get the base64 encoded data url
        var data = thumbCanvas.toDataURL('image/png');

        document.getElementById('thumb-image').setAttribute('src', data);
    };

    // Load image so that we can access image.width and image.height
    image.src = url;
};

See the code in action here (inspect the source to see the full implementation). Note that the canvas is hidden and the scaled image is shown using an img tag with a black border.

This code makes up the basis for our client side scaling feature. What's missing is the part where we store the scaled image on the server. The toDataURL method returns the image data as an base64 encoded string, we could upload this to the server and decode it there, or we could create a Blob object and send the data as binary. The canvas object has a toBlob method, which returns the binary content of the canvas, but the method is not widely supported. Luckily there is polyfill for the method available at github (JavaScript-Canvas-to-Blob).

We can now initiate a XMLHttpRequest instance and send it to our server. The body of the request will contain the image.

thumbCanvas.toBlob(function(blob) {
    var xhr = newXMLHttpRequest();

    xhr.onload = function() {
        if(!/2\d\d/.test(xhr.status)) {
            alert('Non 2xx response code ' + xhr.status);
            return;
        }

        alert('OK');
    };

    xhr.open('POST','/path/to/server');
    xhr.send(blob);
});

One last thing that is missing is a check to see if the needed features are supported by the browser, this can done in a variety of ways. Our script checks if window.URL and canvas exists.

var supported = !!(window.URL || window.webkitURL) && (function() {
    var elem = document.createElement('canvas');
    return !!(elem.getContext && elem.getContext('2d'));
}());

If supported yields false, we fallback to server side scaling. The affected users will still have the same experience as before.

Conclusion

With this technique we achieve several benefits:

1. Senders get a better experience as they are able to see the images currently uploading on their end
2. Recipients get a much better experience because they can see all the images even before they are uploaded
3. We reduce loads to our image scaling servers

As more and more browsers support these HTML5 API's, this technique will help us scale even more.