1. Version control – svn, git, something! Anything, really! Just use something!
2. Unit testing – sure we talked about it, but its utility isn’t clear until you use it in earnest
3. Documentation – always felt more like a chore than a tool
4. Issue tracking – Joel on Software said it best: “I was born with only two bug-storing-slots in my brain.”
5. More best practices – Effective C++ (it’s worth it — get it and read it) alone was worth more than the course was.

I concede that these are things whose utility might be hard to convey to college students who may or may not be working on projects they really believe in. Still, very important.

One of the things I like about documentation is that you can refer to the documentation when uncertain about side-effects, input, etc. of functions and you don’t really feel like reading through code and trying to keep it in your brain-RAM. Not only that, but I usually refer to my own documentation to get an idea of the guarantee that a function was offering, to try to make sure that it’s making those same guarantees after I make changes. Not only this, but articulating functionality crystalizes it’s implications in your mind. When describing how something works to a friend, it often reminds you of issues you hadn’t considered. Writing documentation is the same in this respect.

When working with other people, while it’s nice to be able to talk face-to-face, the issues we think of that need changing go in one ear and out the other for me. Not that it wasn’t a good idea, but my brain-RAM is not infinite. Bugs, features, ideas, issues all get paged out of my head. Write them down instead, which is where issue tracking comes in. Design questions and decisions come out of issues all the time, and the trail of comments and discussion make a nice reference for documentation.

I find that when I’m fixing a bug, the functionality of that chunk of code gets reduced to that particular feature in my mind. So, if when I try using it, if that bug is fixed, then my job is done. But we’ve all experienced unintended side effects, where in fixing one thing, we break another. Having a list of features to check is a step up from trying to remember the things that need to work, but it’s much less than the power of unit testing. I’ve recently been using UnitTest++ for C++ applications, and QUnit from the jQuery people, and have been pleased to thrilled. It allows you to check functionality exhaustively and quickly on a whim. Even the task of writing the tests forces you to define the functional requirements of your code, and I find it helps to clarify a lot.

Version control lets you try crazy new approaches and track changes, isn’t that great? I was helping a friend last semester with some of his code, and he was trying to make code changes with copy, paste, and the undo buffer. In the end, he ended up introducing more bugs into the code than he was fixing. It’s intractable to keep these kinds of code changes in your brain-RAM. Use something, use anything. I have a qualm with svn because it was something I used so rarely, I could never remember how to set it up. Even a barrier as small as that was enough to prevent me from using version control at all. I love git because it’s so easy to set up, I use it for almost anything: from a report (if using LaTeX), to a weekend project, to my thesis. There are tons of tools out there, but just use something.

I remember when I took it, we were supposed to design a small device within certain (relatively arbitrary) constraints to collect a soil sample. The premise was unrelatable – that we might one day be responsible for a subsystem deployed to collect a soil sample on a distant planet, without thought about its return.

It’s become rather popular lately in certain communities to program and attach digital cameras to weather balloons and take pictures from as high up as 30km (about 20 miles) or so. The results speak for themselves:

What I found regrettable about my EPICS experience was that it was too far removed to really care about the project. It had its fun moments, and I was glad that for EPICS 2 I got to work on a project I really believed in with some friends (it was a considerable improvement). But this is an example of a project I contemplate and eye over longingly, even outside the context of a course. Similar projects can cost on the order of a couple hundred dollars and are relatively feasible, even for freshman engineers. To have something you built venture farther from the earth than any one of us likely ever will, bring back pictures and live to tell the tale would be extremely rewarding.

Had there been projects like this when I was in school, I know that my experience would have been that much more enriched.

And they support time dependence.

It can plot virtually any function in x, y and t, and on my MacBook with its NVIDIA GeForce 9400M it has been getting 10k+ fps. I’m still a little surprised by this number, but it seems to be running at that speed.

Flow (vector) fields appear as advected dye. They're currently streamlines, but in the near future I hope to support streaklines and particle flow as well.

Scalar fields appear as a mapping of height onto color. If this function were to be plotted in 3D, it would like a sheet rippling, but sometimes it's more useful to see it in 2D.

On of the great thing about implementing this on the graphics card is that it doesn’t require much CPU time on the machine running it. Even at 10k frames per second, my MacBook never uses more than 30% of a single core’s time. A place where this particularly shines is on tiled displays – a bunch of HDTVs tiled together to run as if it were one large screen. In such setups, a computer will control 2-4 screens, and each computer’s graphics card has enough power to run the animation for its portion of the screen. There are still some bugs to be worked out, but I ran a proof-of-concept on one of the tiled displays at KAUST.

Running a demo of OpenGLot on a KAUST tiled display

Lately I’ve been working on getting the 3D analogs of the various 2D primitives working, again all with time dependence (it’s the support for animation that really makes this shine in my mind). So far it’s surfaces, parametric curves and surfaces and flow fields, but the flow fields have some work yet. It turns out that while modern hardware is definitely capable of handling 3D flow fields, it doesn’t actually make much sense when you see the result – it’s just too busy. To be able to easily visualize flow in 3D is very much an open problem.

3D streamlines end up just becoming confusing more than they are helpful.

In order to get some interesting shapes working, I had to add support for cylindrical and spherical coordinates which is actually providing an interesting challenge – how best to generate the shaders. The shader source code (that runs on the graphics card) is generated and compiled when you run OpenGLot, and I’ve not found an altogether easy and intuitive interface for adding simple coordinate transformations to it. Still, it works, but the programatic interface will likely change.

This is a torus of sorts, which I got as an example from Grapher.app

This is the same torus, just colored by using its surface normals as RGB values

In order to determine surface normals (which are something usually determined when one defines the geometry of an object), the vertex shader approximates various derivatives numerically. So far, the shading results have been pretty decent.

A trigonometric function, colored by mapping the surface normals to colors

The superimposition of two trigonometric functions, lit based on their surface normals and a texture to give visual clues about distortion

I’m still working on making video of this in action available, but so far a number of the tools I would usually use have come up short. I’ve been trying to integrate a video encoder into a utility library for OpenGLot so it can record video straight out of the box, but the framerate is still too low.

I used a tool (pvm2raw) available as part of the V^3 library to convert the pvm files to raw, but VTK requires its own simple header. I actually found that this particular header didn’t work (perhaps a VTK versioning problem?) and so taking guidance from this, checked the header of one of the VTK-included volumes:

bash $> head VTKData/ironProt.vk


This header more or less included a little information on the grid size, spacing and representation of the data:

# vtk DataFile Version 1.0
<Name of File>

POINT_DATA <x * y * z>
SCALARS scalars <unsigned_char|unsigned_short>
LOOKUP_TABLE default
<remember to include a newline here>


Concatenating the header with the raw:

bash $> cat header CT-Head.raw > CT-Head.vtk


At that point, I was in business and was able to move on to generating pretty pictures. Granted, these datasets are pretty sparse, but still VTK did a pretty reasonable job. Update: a comment asked for a little bit more detail on this assignment, and so I’m including my report for the project.

I was amazed today that we can see inside of things… without taking them apart. What an age to live in. Especially the virtual autopsy table I read about recently. In 20 years, we’ll have Firefly-style real-time holographic body scans (ignore music, skip to 0:45):

He talks about the misconception that graphics lie. Of course some do, but his attitude encapsulates well what I think is great about visualization – good representations convey understanding. Graphics can be the most effective way to get a handle on data, or a trend, and they should reveal what underlies the numbers. But in a world of Excel and every insignificant and meaningless piece interrelationship being plotted in an impressive-looking format, it’s easy to forget this.

A quote I heard recently in my Scientific Visualization course (thanks, Thomas!) puts it well:

Visualize to inform, not to impress. If you really inform, you will impress. – Fred Brooks. SIGGRAPH 2003

Although a child can understand a time series, it wasn’t until a couple hundred years ago that they were actually used, as Tufte points out, but its power to convey is obvious. Similarly, just from glancing at a map like this one from the census bureau, one can almost instantly understand the distribution of income across the United States – literally tens of thousands of pieces of data.

A US Census Bureau graphic depicting the income of the 3000+ counties of the United States.

In this vein of conveying understanding, I remember several years ago now watching a TED Talk that immediately captivated me with visualization. Hans Rosling talks about how often when we see the rows about rows and tables upon tables of the massive amounts of census data, not only do our eyes glaze over but it becomes very difficult to keep it all in one’s head at any one time. Visualizing the data is thus a key tool for gaining the insight we seek.

The book is full of tremendous insight about how ink should be used as efficiently as possible (within reason – Tufte is quick to emphasize this point) and that the human eye has a great capacity for handling dense data sets if presented efficiently. It is an entirely necessary resource for anyone who intends to pursue any science, nay, anyone intends to pursue any discipline dealing with numbers.

He has several other books, all of which I intend to read as I was virtually unable to put down the book; I was constantly floored by the myriad examples of strong and weak graphics alike. He also published a book by his mother that I happened to encounter recently via Cool Tools.

I’ll close with a brief excerpt from his book with which I was taken:

Words and pictures belong together. Viewers need the help that words can provide. Words on graphics are data-ink, making effective use of the space freed up by erasing redundant and non-data-ink. It is nearly always helpful to write little messages on the plotting field to explain the data, to label outliers and interesting data points, to write equations and sometimes table son the graphic itself, and to integrate the caption and legend into the design so that the eye is not required to dart back and fort between textual material and the graphic.

For our first three weeks here, 350+ students were stowed in a hotel. Granted, the Intercontinental is a nice hotel but they told us that they’d be bringing us here early so that we might get settled – buy the things like blenders and speakers that we needed, and work the kinks out of our newly minted apartments. We did take care of things like applying for iqamas (our residency permits), but outside of that, we were by and large sitting around.

One event they put on for us that I know I enjoyed was they managed to take 100 students into the desert to go four-wheeling. That was actually pretty awesome.

Five days of orientation beginning the last days of August, largely filled with cultural sensitivity issues and massively behind schedule. There seems to be some cultural disagreement about punctuality, but with no exaggeration, one of these mornings the opening presentations scheduled for 8 am did not get under way until 10. And when you tack on an hour-and-a-half commute from the hotel to the campus, you get a bunch of tired students who would much rather be elsewhere. As a final `can’t believe it’s so’ moment, it was all scheduled during Ramadan – so students who were fasting and who would otherwise be sleeping had to be awake and active.

Getting to campus was another ordeal. We were all very excited to move in, and then largely disappointed to find everything from rodents to black mold, leaky pipes to bottled `leavings’ of construction workers. A few days later, ceilings began to collapse in some units.

At this point, school had started, of course. Like many others, for the first few days of class I was sleeping on a friend’s couch. When I and about a dozen other students were scheduled to move in (getting up at 7:30 to come to campus in time), we were told that it would be about an hour until we could be given keys and move in, but ended up waiting 7 hours to be told that there was no way we would be able to move in, and had to go back to Jeddah.

Books for most classes have still not arrived. We haven’t any clusters online for high-performance computing. In some disciplines, labs won’t be online until 2010. The second fastest supercomputer in academia sits dormant in the basement of one of our buildings because the data center is not clean.

I would like to make it clear that I understand that anything this big getting off the ground so quickly is bound to have problems. There were going to be wrinkles (if not kinks) to work out, and I can appreciate that. I hope to not descend into a rant as frankly I’m all ranted out.

There have been perks – the food on campus is free to students for an indeterminate amount of time; our books will be purchased for us this semester; there have been goodies abundant to be had (from t-shirts to thumb-drives). As much as I love a thumb drive, I would have preferred something else – a home.

That said, the students have mobilized. We’ve signed petitions, we’ve organized, and we’ve gotten interim student representation in place. We wanted to convey the idea that we don’t want to perpetually complain but would like to help out if that means us coming into a better situation. And despite some of our expectations, within two days, the administration had contacted us to schedule meetings and put us in contact with the contractors in charge of this and that (IT, housing, etc.). It’s actually been very inspiring to see the actions we take have tangible results. Had to talked to me two days ago, you would have found me a very tired and disenchanted man, but today I’m very hopeful.

In terms of academics, I am really happy with the faculty. I’ve secured a TA position, and I spend a couple hours a day with this prof in a constant stream of geeking out. And I spoke yesterday to a prof for two hours about the research he’s doing and how I might get involved on that front. Depending on how this goes, I’d be willing to consider staying on for a PhD!

I’ve made instant friends with a couple of students here, and three of us have gotten a license to host a local TED event: TEDxKAUST. We’ll replay some of our favorite talks, and those we think would be most inspiring to students here, and we hope to get a couple of professors to present as well. In our budget, we’ve been able to allocate funds to booths and demos and if all goes well (read: gets approved), we’ll have a Ruben’s Tube.

Long story short:

A render of the Stanford bunny with my raytracer without Gouraud shading.

A render of the Stanford bunny with smooth Gouraud shading.

Also, thanks to an improvement in my parallelization of the problem and a speedup in octtree traversal, I was able to render the Stanford dragon model (~1 million triangles):

The Stanford dragon model.

Tomorrow!

Reflective spheres rendered at high resolution on a small cluster.

Another image rendered on the same cluster

For the benefit of those who are not computer scientists, this is what the input file looks like:

8192 8192
0 0 20
-1 -1 1
2 0 0
0 2 0
3 10 10 0.8
0.2
9
# These next few lines will define a triangle
T
# With one of the points at (1, 1, 1)
1 1 1
# and the next point here:
0.12321 0.12321 -1
# and the last point here:
-1 1 1
# and with this color setting
1 1 1 1 1 1 0 1 0.7
T
-1 1 1
-0.12321 0.12321 -1
0.12321 0.12321 -1
1 1 1 1 1 1 0 1 0
T
1 -1 1
0.12321 -0.12321 -1
-1 -1 1
1 1 1 1 1 1 0 1 0
T
-1 -1 1
-0.12321 -0.12321 -1
0.12321 -0.12321 -1
1 1 1 1 1 1 0 1 0
T
1 1 1
0.12321 0.12321 -1
1 -1 1
1 0 0 1 0 0 0 1 0
T
1 -1 1
0.12321 -0.12321 -1
0.12321 0.12321 -1
1 0 0 1 0 0 0 1 0
T
-1 1 1
-0.12321 0.12321 -1
-1 -1 1
1 1 0 1 1 0 0 1 0
T
-1 -1 1
-0.12321 -0.12321 -1
-0.12321 0.12321 -1
1 1 0 1 1 0 0 1 0
S
0 0 0 0.5
1 0 1 1 0 1 0 1 0.3


Tonight was also the first time that a student in the class scheduled an appointment to get help with her homework. I hadn’t looked at the upcoming homework as they just finished the first assignment, but I was very pleased when I was able to jump right in and know exactly the problem. It’s a great feeling to know the entirety of a class without needing a refresher, and being able to take any section and just go.

In short, so far so good.

I did write a text-based interface for playing against it, but it’s tedious to actually use, and so, having done well in graphics, I decided to write a simple GUI in OpenGL. You can click-and-drag your pieces to where you want them, and then wait for the computer to play. Here’s a screenshot of a game’s conclusion:

A game's conclusion

I’d like to give the piece icons transparency on the border so that you can see the underlying square, but that will be a job for tomorrow. Even as it is, I’m pleased with it.

Yesterday he found yet another gem. Apparently there’s a mutex (mutual exclusion) on sex and studying engineering:

It's my future! I want to be an engineer... sex can wait!

At any rate, it really solidifies the KAUSTiness, and is oddly gratifying. I’m very excited for this.

This is all because one of my few regrets about KAUST is that I wouldn’t get a chance to T.A. or teach there as a student, so, I’m pursuing that here.

So, if you’ve still got Sci. Comp. left to take, take it next semester. I hear they’re going to have a sweet T.A.!

For those of you who aren’t computer scientists, here’s a quick explanation – we need space to store variables and information. So, one way to do this is ask the system for some space, and then the system gives back an address. Imagine mailboxes at the post office – you don’t have the stuff stored in your mailbox with you at all times, but rather the way (the key) to access it. One problem that arises when you tell the post office you don’t need the mailbox anymore, but you don’t give back the key. Someone else is assigned this mailbox, and his mail starts arriving in there, and when you go looking for yours, it’s not what you expect (this often causes a segmentation fault). The other problem is when one patron keeps requesting mailboxes, and you’re only allowed to hold one key at a time, so you discard the old ones. If you request too many, no mailboxes are left (this is a memory leak)!

At any rate, I don’t have much experience tracking down memory leaks, and so when I came across this problem, I asked an open question to everyone in the lab of the best way to do this. Someone suggested I put soapy water on my program and look for bubbles.

I ended up looking through my code for places where this might be happening, and I happened to find it pretty quickly. That’s pretty lucky.

A memory leak gets out of control

We get our memory back when the program is terminated.

I got gdb to seg fault!

The project is dealing with Voronoi diagrams, and so the first part was to generate a random set of points, and then the associated Voronoi diagram:

A Voronoi diagram

The next step was to actually create a mosaic. The idea is that the whole Voronoi region should be the color of the point on the picture that corresponds to the associated Voronoi point. At any rate, what it comes down to is pretty pictures. I’ve included the original and mosaic version for some of the pictures we were provided.

Some Columbines

A mosaic version of the same columbines

Guggenheim Hall

A mosaic of Guggenheim Hall

A picture of a couple of squirrels

A mosaic of two squirrels

I’ve got others, but they are less than impressive. If you’ve got a square picture that you’d like to see made into a mosaic in this way, and I like you, send it along. Or, use any number of image editors.

The first part was to generate a torus and texture it with a tile-able pattern (we were given a few wood textures and one brick). Same trackball interface applies.

A torus made of bricks

Next up was to add a way to render a model that’s seemingly made out of mirror, reflecting a virtual environment, provided to us by a teacher. This one seems to be the inside of a building near a window.

Bunny of mirrors

Lastly, given a 3D texture (marble), we were to “carve” a model out of that material. See Isis and the shark made of marble.

At any rate, I nit-picked an essay (they wanted an essay of 50 lines or less detailing academic achievement, international experience, statement of purpose, etc., so it’s bound to be a paced a certain way), and got letters of recommendation from three of my professors. Specifically, Graeme Fairweather, the department head, Mike Colagrosso, a recently tenured professor who just left us for the private sector (you’ll be missed, Mike), and Yong Bakos, a first year professor. All in all, I felt pretty good about my application.

Well, earlier this week I got an email saying that I had been selected as a finalist. Of the 2300 who applied, they whittled it down to 672 students (of whom they’re saying they’ll accept 500); 99 of these are from the US or Canada. Making it to the finals means that they’re going to fly me out to Washington D.C. for an interview this weekend, about which I’m very excited. I’ve never been back east, and it will give me a chance to see my cousin Elizabeth and her husband Wess, who live in Arlington, VA. The KAUST people even let me delay my departure so I could spend Labor Day weekend with them. Hooray!

At a recent dinner, my mother mentioned that she was praying I didn’t get accepted. Something about the perceived danger of living in the Middle East for two years. At any rate, things are looking up, and at the very least, I’ll get a trip to D.C. out of it.

It really looks like a beautiful campus and a unique opportunity. I would be in their first graduating class, getting my Master’s when I finish. I have a knack for being in the early graduating classes for schools. I was in the second class out of my middle school, and I was in the second class to graduate from the IB program at Niwot.

For more on KAUST, visit their website.

Yes, sometimes it’s difficult, but I think we often have a head start on our counterparts in a lot of ways by engineering.

Some of my favorite professors have been from computer science. Granted, that is what I study, but even from high school, many of my favorites were in math / science. My high school calculus teacher motivated me very much, and would spend hours with me after school talking about various proof-projects we had going on (yes, I’m apparently that kid).

Textbooks can be rough, I’ll give him that, but I think that’s in large part symptomatic of the information engineering textbooks must contain. They have to catalog a lot of things, and for my money, I think they generally do a pretty reasonable job.

While other disciplines may have inflated grades, I haven’t been denied opportunities because of my grades. I don’t have quite the GPA I’d like, but people know how difficult sciences are and appreciate it when considering you for positions.

Sometime the beatings the assignments put us through do tend to blend together; I don’t know about Andrew’s coursework, but this semester I’ve had some of the most exhilarating assignments I’ve ever had. We wrote our own shell, we downloaded and parsed RSS feeds using only shell (unfortunately not our own). For Scientific Computing, I admit there was a lot of work, but they were assignments that kept me up at night, out of pure curiosity.

This doesn’t even include the job market situation. Other engineering disciplines don’t feature the growth that computer science has seen in recent years, I admit, but Mines certainly has fantastic job placement. Of my closest high school friends, I have two in international relations, three in linguistics, one in liberal arts, and one in chemical engineering. The engineers enjoy the jealousy of the others with respect to the prospect of work.

All things considered, I’ve really my collegiate career up to this point, and I imagine I will continue to.