At any rate, I need to render a regular 2D grid, and as it doesn’t support quads, either, I was forced to use triangles. In the interest of getting things running, I just provided a wasteful list of discrete triangles. This is wasteful because it references many more vertices than necessary – I ended up declaring $$6n^2$$ vertices when in reality there are only $$n^2 + n + 1$$ unique vertices. This worked fine, until I wanted to increase the resolution. It turns out, JavaScript doesn’t like large arrays.

That’s fair, because the implementation was pretty wasteful. A triangle strip was the best choice anyway. A triangle strip is a highly compact form of representing a mesh. For $$n$$ triangles, it requires only $$n + 2$$ vertices defined. Well, that’s roughly true. We’ll see another case in a minute. It’s useful when many triangles share vertices, and perhaps I’ll let Wikipedia’s explanation stand.

It wasn’t immediately obvious how to define a grid out of a single triangle strip and so I got out a pen and paper. I kept in mind a neat trick: if in a triangle strip, you need to skip the use of a vertex, a vertex can be introduced twice in a row. That is, if I need triangles (6, 3, 7) and (7, 11, 6) in that order, you can just make your strip with 6, 3, 7, 7, 11, 6. You can think of it as if there are two triangles created (3, 7, 7) and (7, 7, 11), but they have no area and a degenerate case – a line. Furthermore, these lines lie on triangles already defined.

Perhaps the obvious choice doesn’t yield any results, and in fact in this layout, it can’t be easily done (you have to have vertices appear three times in a row):

This is a bad idea for a topology if you want to use a single triangle strip.

To better convince yourself, try to come up with a good way to put this in a triangle strip. I’ll make the case that it is pretty difficult with a claim from graph theory. In order for a triangle mesh to be turned into a triangle strip, each consecutive triangle must share an edge. We can then think of the mesh as a connectivity graph (the dual of the mesh) and then the problem will emerge more clearly:

The dual graph of the bad idea.

To make the triangle strip “nice,” we ought to be able to visit each node once and exactly once. There’s good and bad news in this – it’s the same problem as finding a Hamlitonian path which is NP complete. The good news is that if we find a solution to our small problem, we’ve found a solution to all such grids (with arbitrarily many triangles). Note that we don’t need an Eulerian path.

If you stare long enough at the above connectivity graph, you’ll hopefully convince yourself that there’s no way to traverse it visiting each node once and exactly once. Go ahead and try – it’s pretty infuriating.

Looking at how we would traverse one strip (triangles a, b, c, d, e and f) actually gives us a clue. A triangular strip for that case would be 0, 4, 1, 5, 2, 6, 3, 7, and happiness ensues and we should move onto the next row. Unfortunately, in the context of this new row, we’re starting in a different place (topologically) than we started with the first strip. Vertex 0 has two connected neighbors in its row – 1 and 4. Vertex 7 has three in its row: 6, 10 and 11. It turns out we can change up the topology to remedy this simply:

A much better topology for drawing this with a single triangle strip.

We can also see that this is a much better solution by looking at this new graph’s dual:

The dual of the better topological choice.

You can probably easily find a Hamlitonian path in this case. But this still leaves us with how to determine the vertex orderings. We decided that the first row ought to be 0, 4, 1, 5, 2, 6, 3, 7, but moving on from there we need a bit of “glue” to move onto the next row. We insert 7 again, and then continue on from there: 7, 11, 6, 10, 5, 9, 4, 8. A bit more glue for the third row: 8, 12, 9, 13, 10, 14, 11, and 15:

An alternative representation of the vertex ordering

Looking at the indices from the first row, starting at 0, we can get the next index by alternately adding 4 and then subtracting 3. On the next row, we’ll continue to add 4, but then alternately subtract 5. The 4 is derived as being the number of vertices on a side (if there are $$n$$ divisions, then there are $$n+1$$ vertices), and the 3 and 5 are explained by the fact that we need to change columns in the mesh, by one step at a time.

An clean implementation is not trivial, but not extremely difficult. In terms of results, I can fit more than 4 times as many vertices into the mesh than with a per-triangle implementation. And to boot, it has cut the work of the vertex shader a great deal.

The big “get” is that it gives programmers access to hardware-accelerated graphics from directly within the browser. There’s a lot of interest in this arena for game development as it would obviate much of the need for separate distributions based on operating system. (Skip to the end for more of an opinion on this subject.)

As such, I’ve been working with WebGL as opposed to the Google-proposed O3D. (I have every intention to explore O3D, time permitting, as there are some jagged edges to the current specification.) The result of this recent toil is a budding WebGL implementation of my OpenGLot project. It’s still in early stages, but in the coming weeks, it should develop even further. To whet appetites, I have a few pictures.

A scalar field, my persistent test function.

A 3D surface, again one of my usual test functions.

I seriously doubt that WebGL will every match the performance of OpenGL. Even though JavaScript interpreters are getting faster at a somewhat alarming rate, they won’t match the speed of C or C++. That said, if one can appropriately offload work onto the GPU, it won’t matter as much, but there will always be that overhead.

It won’t so much be a matter of having the same performance, but enough performance. If a person can go to a single webpage and get 60 frames per second performance in a game or tool without having to install software, that’s tremendous. Currently I’ve been getting between 60 and 90 frames per second with WebGLot, and I’m sure I can keep that number up as more features are added.

My hope is that this will be a tool and library that has a wide-enough feature set by the time WebGL is widely adopted that becomes often-used. But that’s just ego. The purer motivation is that if you’re a math teacher, and you want to have interactive demonstrations of Newton’s method, or parametric surfaces, or even flow fields, you can write an application in 20 minutes that does all the heavy lifting of graphing it for you. As long as you can describe the mathematical primitives, you should be able to render it. Of course there will be a general-purpose grapher available for any calculus student who’s having trouble visualizing this or that, too. Or a resourceful PDE student who need to solve his homework (the GPU-based PDE solver will take a little bit more time, but it’s very nearly complete).

In short, the strength of WebGL is that is has respectable performance, and in a year’s time, half the browsers (well maybe not half) on computers will support it, giving the average internet-user access to a wealth of media.

OpenGLot3D from Dan Lecocq on Vimeo.

And they support time dependence.

It can plot virtually any function in x, y and t, and on my MacBook with its NVIDIA GeForce 9400M it has been getting 10k+ fps. I’m still a little surprised by this number, but it seems to be running at that speed.

Flow (vector) fields appear as advected dye. They're currently streamlines, but in the near future I hope to support streaklines and particle flow as well.

Scalar fields appear as a mapping of height onto color. If this function were to be plotted in 3D, it would like a sheet rippling, but sometimes it's more useful to see it in 2D.

On of the great thing about implementing this on the graphics card is that it doesn’t require much CPU time on the machine running it. Even at 10k frames per second, my MacBook never uses more than 30% of a single core’s time. A place where this particularly shines is on tiled displays – a bunch of HDTVs tiled together to run as if it were one large screen. In such setups, a computer will control 2-4 screens, and each computer’s graphics card has enough power to run the animation for its portion of the screen. There are still some bugs to be worked out, but I ran a proof-of-concept on one of the tiled displays at KAUST.

Running a demo of OpenGLot on a KAUST tiled display

Lately I’ve been working on getting the 3D analogs of the various 2D primitives working, again all with time dependence (it’s the support for animation that really makes this shine in my mind). So far it’s surfaces, parametric curves and surfaces and flow fields, but the flow fields have some work yet. It turns out that while modern hardware is definitely capable of handling 3D flow fields, it doesn’t actually make much sense when you see the result – it’s just too busy. To be able to easily visualize flow in 3D is very much an open problem.

3D streamlines end up just becoming confusing more than they are helpful.

In order to get some interesting shapes working, I had to add support for cylindrical and spherical coordinates which is actually providing an interesting challenge – how best to generate the shaders. The shader source code (that runs on the graphics card) is generated and compiled when you run OpenGLot, and I’ve not found an altogether easy and intuitive interface for adding simple coordinate transformations to it. Still, it works, but the programatic interface will likely change.

This is a torus of sorts, which I got as an example from Grapher.app

This is the same torus, just colored by using its surface normals as RGB values

In order to determine surface normals (which are something usually determined when one defines the geometry of an object), the vertex shader approximates various derivatives numerically. So far, the shading results have been pretty decent.

A trigonometric function, colored by mapping the surface normals to colors

The superimposition of two trigonometric functions, lit based on their surface normals and a texture to give visual clues about distortion

I’m still working on making video of this in action available, but so far a number of the tools I would usually use have come up short. I’ve been trying to integrate a video encoder into a utility library for OpenGLot so it can record video straight out of the box, but the framerate is still too low.

With graphics, the results are sometimes cool, sometimes horrifying and sometimes beautiful. I make a habit of documenting the results of broken code as much as I can, and after a recent one, I thought I’d share some of the more interesting ones.

I used a tool (pvm2raw) available as part of the V^3 library to convert the pvm files to raw, but VTK requires its own simple header. I actually found that this particular header didn’t work (perhaps a VTK versioning problem?) and so taking guidance from this, checked the header of one of the VTK-included volumes:

bash $> head VTKData/ironProt.vk


This header more or less included a little information on the grid size, spacing and representation of the data:

# vtk DataFile Version 1.0
<Name of File>

POINT_DATA <x * y * z>
SCALARS scalars <unsigned_char|unsigned_short>
LOOKUP_TABLE default
<remember to include a newline here>


Concatenating the header with the raw:

bash $> cat header CT-Head.raw > CT-Head.vtk


At that point, I was in business and was able to move on to generating pretty pictures. Granted, these datasets are pretty sparse, but still VTK did a pretty reasonable job. Update: a comment asked for a little bit more detail on this assignment, and so I’m including my report for the project.

I was amazed today that we can see inside of things… without taking them apart. What an age to live in. Especially the virtual autopsy table I read about recently. In 20 years, we’ll have Firefly-style real-time holographic body scans (ignore music, skip to 0:45):

A few days ago I got decent-looking contour lines working and today I got scalar fields implemented with a fragment shader. The programmer using the library can specify a function in the form of a string and have it plotted as a scalar field. Quickly. Really quickly.

Modern graphics cards support shaders, which are programs that get run on the graphics card, and in parallel. This is great for algorithms that can be run in isolation (one pixel doesn’t need to know what the others are doing), which is the case here. OpenGLot generates a fragment shader that colors a single pixel based on the value of the function. Each of the dozens or hundreds of cores on a GPU runs the same code in parallel for their particular pixel.

Contouring for a sinusoidal function with the isovalue 0.

Here's a first look at the results (taken moments after this first worked for me). It's the same sinusoidal function, and I will be improving upon the color mapping in the coming hours.

The graphing program I use, Grapher.app is rapidly showing its age. The results it gives for the same function (though using a much better coloring scheme) are either grainy or extremely slow (10 or more seconds). OpenGLot is generating these in less than 0.1 seconds. (Grapher.app implements its scalar fields on the CPU, so comparing times is a little like comparing apples and oranges. Still, responsiveness in this type of matter is important.)

The quick (and grainy) plot in Grapher.app

The slow (but smooth) plot in Grapher.app

He talks about the misconception that graphics lie. Of course some do, but his attitude encapsulates well what I think is great about visualization – good representations convey understanding. Graphics can be the most effective way to get a handle on data, or a trend, and they should reveal what underlies the numbers. But in a world of Excel and every insignificant and meaningless piece interrelationship being plotted in an impressive-looking format, it’s easy to forget this.

A quote I heard recently in my Scientific Visualization course (thanks, Thomas!) puts it well:

Visualize to inform, not to impress. If you really inform, you will impress. – Fred Brooks. SIGGRAPH 2003

Although a child can understand a time series, it wasn’t until a couple hundred years ago that they were actually used, as Tufte points out, but its power to convey is obvious. Similarly, just from glancing at a map like this one from the census bureau, one can almost instantly understand the distribution of income across the United States – literally tens of thousands of pieces of data.

A US Census Bureau graphic depicting the income of the 3000+ counties of the United States.

In this vein of conveying understanding, I remember several years ago now watching a TED Talk that immediately captivated me with visualization. Hans Rosling talks about how often when we see the rows about rows and tables upon tables of the massive amounts of census data, not only do our eyes glaze over but it becomes very difficult to keep it all in one’s head at any one time. Visualizing the data is thus a key tool for gaining the insight we seek.

The book is full of tremendous insight about how ink should be used as efficiently as possible (within reason – Tufte is quick to emphasize this point) and that the human eye has a great capacity for handling dense data sets if presented efficiently. It is an entirely necessary resource for anyone who intends to pursue any science, nay, anyone intends to pursue any discipline dealing with numbers.

He has several other books, all of which I intend to read as I was virtually unable to put down the book; I was constantly floored by the myriad examples of strong and weak graphics alike. He also published a book by his mother that I happened to encounter recently via Cool Tools.

I’ll close with a brief excerpt from his book with which I was taken:

Words and pictures belong together. Viewers need the help that words can provide. Words on graphics are data-ink, making effective use of the space freed up by erasing redundant and non-data-ink. It is nearly always helpful to write little messages on the plotting field to explain the data, to label outliers and interesting data points, to write equations and sometimes table son the graphic itself, and to integrate the caption and legend into the design so that the eye is not required to dart back and fort between textual material and the graphic.

An application for them I hadn’t seen until a few weeks ago was projection mapping. I’ve run across a couple of posts about them on MAKE. Where traditionally you project onto a flat screen or wall, you can equally project onto any other geometry. If you have a representation of that geometry, one can create all manner of optical illusions. Perhaps some videos will make this clear:

The amazing thing about these is that something so impressive is going on that you don’t even notice it. You might think that the same illusion can be accomplished on a flat screen, but making use of the fact that your brain uses context to build a picture means that this can be much more impressive.

Beau Lotto talks about this in a really interesting TED Talk I saw recently. What I have in mind particularly is when he talks about the tiles in light and shadow and how we perceive their color. The buildings the above videos project onto are the tiles in this analogy, and by projecting varying levels of light onto them, we perceive a different situation from what’s actually happening, and what’s so amazing to me is that it’s so compelling and convincing, you don’t it didn’t quite soak in at first just how impressive it is.

I hope to be able to have a project like one of these projection mappings at KAUST.

WebGL is the future of the internet. I’ve been hoping for something like this to come out for a while, and in Firefox’s latest nightly builds, it’s included. You should read more about it straight from the horse’s mouth, but this is going to be amazing.

I love OpenGL, and to have a framework from which to call these commands I know and love for the web will be fantastic. No need to worry about system requirements beyond whether or not WebKit runs on your computer… no need to download the latest versions of some piece of software…

Today sees the start of the co-located events (I’m going to NPAR – Non-Photorealistic Animation and Rendering), and I’m so psyched! If you’re at SIGGRAPH or in New Orleans and want to hang out and network, drop me an email / comment!

Woohoo! SIGGRAPH!

Long story short:

A render of the Stanford bunny with my raytracer without Gouraud shading.

A render of the Stanford bunny with smooth Gouraud shading.

Also, thanks to an improvement in my parallelization of the problem and a speedup in octtree traversal, I was able to render the Stanford dragon model (~1 million triangles):

The Stanford dragon model.

It contains a little under 70,000 triangles which makes brute-force ray tracing intractable. I mentioned octtrees earlier, and so having built octtrees into my ray tracer, I was able to render the Stanford Bunny in about 40 minutes on one core. Granted, that’s with only 1-pass anti-aliasing, but I feel pretty good about this. I don’t think I’ll have a chance to implement Gouraud shading (or normal interpolation for that matter), but as soon as I do, it will look a lot less blocky.

The Stanford Bunny rendered with my ray tracer.

We’ve got a picture of a model (in this case, the Stanford bunny model). It consists of thousands of tiny triangles that make a surface. Then, imagine a cube surrounding the entire model. If there are two many triangles in that cube, you cut the cube in to eight smaller cubes, and repeat. What this build is a tree where “busy” portions of the space get divided more.

And now for pretty pictures:

The bunny on its own. There are tricks to smooth it out, but I left it highly triangulated to better represent the idea.

Bunny with balanced wireframe octtree around it.

Profile of the bunny with a very deep octtree.

I’m finishing up the implementation, and then I’ll be using it as part of my octtree as a intersection speedup.

Texturing accomplishes by taking a primitive shape (like a sphere, triangle, surface of revolution, etc.) and wrapping an image onto and over it. Let’s consider a sphere in space:

A white sphere in space.

Now let’s say we mean it to be Earth. Then we can take a picture of Earth that’s flat:

Flattened map of Earth

and then map it onto a sphere to get a picture of what we all know Earth to look like:

Behold!

Two Mars globes where the left is what is seen with the eye, and the right is a topographic map.

Reflective spheres rendered at high resolution on a small cluster.

Another image rendered on the same cluster

For the benefit of those who are not computer scientists, this is what the input file looks like:

8192 8192
0 0 20
-1 -1 1
2 0 0
0 2 0
3 10 10 0.8
0.2
9
# These next few lines will define a triangle
T
# With one of the points at (1, 1, 1)
1 1 1
# and the next point here:
0.12321 0.12321 -1
# and the last point here:
-1 1 1
# and with this color setting
1 1 1 1 1 1 0 1 0.7
T
-1 1 1
-0.12321 0.12321 -1
0.12321 0.12321 -1
1 1 1 1 1 1 0 1 0
T
1 -1 1
0.12321 -0.12321 -1
-1 -1 1
1 1 1 1 1 1 0 1 0
T
-1 -1 1
-0.12321 -0.12321 -1
0.12321 -0.12321 -1
1 1 1 1 1 1 0 1 0
T
1 1 1
0.12321 0.12321 -1
1 -1 1
1 0 0 1 0 0 0 1 0
T
1 -1 1
0.12321 -0.12321 -1
0.12321 0.12321 -1
1 0 0 1 0 0 0 1 0
T
-1 1 1
-0.12321 0.12321 -1
-1 -1 1
1 1 0 1 1 0 0 1 0
T
-1 -1 1
-0.12321 -0.12321 -1
-0.12321 0.12321 -1
1 1 0 1 1 0 0 1 0
S
0 0 0 0.5
1 0 1 1 0 1 0 1 0.3


First, anti-aliasing. Here are a few pictures of the same image rendered with a different number of passes to change the smoothness.

Using one pass

Using 5 passes.

Using 50 passes.

And lastly, here’s a scene of several spheres, some of which are reflective.

A set of spheres, some of which are mirror-like.

A reflective sphere next to some triangles.

A mirrored sphere in the center of four other sphere.

More spheres!