Over the weekend, I attended HackOTT, a hackathon here in Ottawa that encouraged everyone to play around with some neat third-party APIs. It was a lot of fun seeing the awesome apps everyone came up with, and even though we didn’t get to demo, I’m happy with how much I learned.
My team was comprised of myself, my softball captain/ex-coworker @jyboudreau, and our fearless leader @davefp. The idea was that we would create an HTML5 application that allowed users to upload an image of a room, select a few colours from that image, and get back a list of products sold through Shopify that match the room. We were pretty excited, and so were some of the API guys we talked to.
Dave and JY grabbed the TinEye and Shopify APIs, so that left me with the UI. While we didn’t quite manage to get everything working in time to demo, we did make a lot of progress, and I thought I’d share part of my contribution, a Canvas-based app that lets the user pull colour swatches out of an image.
Let’s look at how it works!
Basic Setup
The layout is pretty simple. The empty blocks on either side are just divs that will hold our colour swatches, and that image in the middle is actually a canvas. In fact, it’s two canvases, overlayed on top of each other using some absolute positioning.
We used two canvases to make the drawing easier. The backmost canvas holds our image, and that’s it. The frontmost canvas, which is completely transparent, is where the swatch outlines are drawn. This makes it less expensive to redraw swatch outlines, because we don’t have to reload the image each time, and allows two swatch outlines overlap without having them affect one another’s colour.
Now, let’s have a look at the drawing code.
Loading the Image
Loading an image into a canvas is relatively straightforward. Check out the javascript file, and in particular note the addUserImage function. The mechanics of loading an image are simple:
- Create a new logical image:
var img = new Image();
- Make sure the image’s onload function ends by drawing the image:
context.drawImage(img,0,0,img.width,img.height);
- Trigger onload by setting the image’s src property:
img.src = TEST_IMG;
There are a couple of gotchas, though:
First, you may have noticed that in the image’s onload lamba, we’re adjusting the dimensions of both canvases and their container. This resizes our canvases and the surrounding layout to match the size of the image. We also set the canvases to display block because they are hidden by default (this avoids an ugly resizing-flash right after the page loads).
Second, the image src can’t be just any image. For canvas to load it properly, it must be an image contained within your own domain. This means you can’t just give it a url you found online, or even load it from a file using localhost. We deployed our app using App Engine, but any container should do the job just fine.
That’s all there is to loading an image, let’s move on to swatches.
Drawing the Swatch Outlines
There are three user events we care about for our canvas: mousedown, mousemove, and mouseup. To handle these events, there are three functions: handleCanvasClick, handleCanvasMouseMove, and handleCanvasMouseUp. Let’s look at these a little more in-depth.
First, you’ll notice that each function uses some simple math to get the coordinates of the mouse click:
var clickX = event.pageX - canvas.offsetLeft; var clickY = event.pageY - canvas.offsetTop;
We get the coordinates from the page via event.pageX, then subtract the top-left corner of the canvas so that we’re left with the distance of the click from the canvas’s top-left corner. Conveniently, the origin for canvas is located in the top-left corner, so we’re already in the right coordinate space and our x/y positions are ready to use.
Next let’s talk about getSwatchIndex(). This is a convenience function that parses the id of the currently-highlighted div to give us a numerical representation. Why is this important? Because we want to maintain an array that represents the current position of each swatch outline, and we use these numbers to index it.
By storing the positions of the swatch outlines in an array, we’re free to clear the swatch-outline canvas and redraw it completely on each pass. This might seem like overkill, but it’s necessary for situations where two swatch outlines overlap, and at a code level, it’s less work than repainting a transparent box over a swatch outline before the swatch outline is painted again it in its new position.
Once we’ve updated our array, it’s off to our redrawSwatches function to actually draw them. The algorithm here is what you would expect, we loop over the array of swatch outline positions, and draw each one with a semi-transparent background and a solid border. We’re also watching for the currently selected swatch index to come up, because we want to highlight that border with a brighter colour so that the user knows which swatch outline is active.
Handling Dragging
We wanted the user to be able to drag a swatch outline around to make sure it’s placed in exactly the right spot. This ended up being easier than we thought. You may have noticed the dragEnabled variable in our mouse event functions. This is a global boolean that we set on mousedown and clear on mouseup. That way, when mousemove fires, we can check it and redraw if a drag is occurring. Simple!
Extracting Colour Information
Let’s head back to handleCanvasMouseUp and look at the colour extraction (which should probably be in its own function).
The important step is this one:
var imageData = context.getImageData(
clickX,clickY,HIGHLIGHT_SIZE,HIGHLIGHT_SIZE).data;
Here we’re telling the canvas to give us the image data for a square positioned where the user clicked, and the size of our swatch outline (that I for some reason called a highlight this time — we were in a rush). That returns canvas’s own ImageData object, which probably does all kinds of neat things, but we just wanted the pixels, so we called .data to grab them.
Pixel data in canvas is stored as a giant rgba array. So if you have a 10×10 canvas, then the array will be of size 400 (10x10x4) and will be formatted as [r1, g1, b1, a1, r2, g2, b2, a2, etc]. We want the rgb values only (we skip straight over the alpha values in this case), so we sum up all of the reds, blues and greens individually.
Finally, we average out each colour by dividing it by the number of pixels, floor the totals to get integer values, and voilà! We have an average colour we can show in the swatch.
It was fun spending the day playing around with canvas. Hopefully next time we’ll get something we can demo!