Language Quiz!

I took this photograph this weekend in Xenia, Ohio.  Identify the alphabet, the language, and where the photo was taken.

Ambiguous constructors considered harmful

Eclipse is more than just a code editor: it is a very general and flexible framework for building tools for programmers. I've been using it for my own research in building a tool for debugging cognitive models. But I'm finding that the learning curve for writing good tools in Eclipse is incredibly steep.  I don't want to be too harsh about it because it's an incredible effort involving a lot of people over a lot of years, and maybe a project of that complexity is just necessarily going to trade ease of use for power. But there are at least some incremental ways that the situation can be improved. Here are a couple examples I've found in recent months:

I've been working on a debugging plugin for Eclipse, and learning SWT in the process.  SWT is the library Eclipse uses for drawing buttons and windows on the screen. Today I discovered something horrible in their API. I needed to pick colors for a graph, and I wanted them to be a little darker if they were selected.  The right way to do this is to describe the color in terms of three values: hue, saturation, and brightness, but the Color object in SWT thinks of colors in terms of red, green, and blue.

So how do you convert between the two ways of describing color?  Easy: there's an object called RGB: you load it up with the color you want, and then ask it for the hue, saturation, and brightness values you need.  Here are its two constructors:

RGB(float hue, float saturation, float brightness)           Constructs an instance of this class with the given hue, saturation, and brightness.

RGB(int red, int green, int blue)           Constructs an instance of this class with the given red, green and blue values.

If you know H/S/B already and want R/G/B you pass it floating point numbers, then query its member fields; if you know R/G/B already, you pass it integers, and then call a "getHSB" method. As a result, the following code will do something completely insane:

int red = 100;
int green = 100;
int blue = 50;
float pastel = 0.85;
Color normal = new RGB(red,green,blue)
Color bland = new RGB(red*pastel, green*pastel, blue*pastel)

I've seen this in more than one place in SWT: the same class will have multiple constructors with different types, that have different meanings that you'd never guess at, and the types are ones that the compiler is apt to convert automatically for you.  

Another case of this same disease, this time in the JFace library: the story begins with the task of creating three nested widgets: a "Composite" (which is usually a window pane you see as part of Eclipse's interface), a "Tree", which is kind of a spreadsheet-looking control placed in that pane, and a "TreeViewer", an invisible thing between the Composite and the Tree that does some convenient behind-the-scenes accounting work. 

Here's the documentation for a TreeViewer's constructors:

TreeViewer(Composite parent)           Creates a tree viewer on a newly-created tree control under the given parent.

TreeViewer(Composite parent, int style)           Creates a tree viewer on a newly-created tree control under the given parent.

TreeViewer(Tree tree)           Creates a tree viewer on the given tree control.

The first two constructors put a TreeViewer into a Composite, and create the Tree for you automatically.  The third constructor assumes you've already got a Tree inside a Composite, and it sticks the TreeViewer in the middle for you.

The irksome thing here is that a Tree is a subclass of Composite.  In my case, I misread the documentation and tried to pass both a Tree and a "style" argument to the constructor. It matched the second, not the third constructor, treated my Tree as if it were a window pane, and cheerfully put another tree inside my tree (I heard you liked Trees, dawg!). The end result was a Tree viewer managing a different tree than the one I was writing data to, and I spent about a day stepping through SWT code trying to work out exactly where it was going wrong.

There's a growing body of research on how to make APIs usable for programmers. Jeff Stylos' and Steven Clarke's research on constructors, in particular, concluded that it was better (for other reasons) to just have constructors without arguments, and have users set up the objects after their creation with method calls.

In general, making Eclipse easier to write plugins for is a really interesting problem, and I hope after I get over my dissertation hump I'll have time to pitch in and help with it.

Darius and the Dragons

I love the writing system designed for the dragons in Bethesda's new game, Skyrim.

According to a gameinformer interview with the language's designers, the symbols are designed to look like they were gouged into stone by dragons with three talons and a dewclaw (that odd extra toe partway up your cat's leg that seems to serve no purpose)

Dragon writing from Skyrim

Bethesda (the company that makes Skyrim) was not the first bunch to encounter this graphic design problem. Cyrus the Great was an ancient Persian king (around 600 BCE) with the same idea: he wanted to create monumental inscriptions to celebrate his grandiose awesomeness, and he appeared to have a similar aesthetic, and apparently had stoneworking tools not unlike Skyrim dragon talons:

"Newer" cuneiform of Cyrus and Darius,
 about 600 BCE (from Wikipedia)

That's from the Behistun inscription in Iran, which is a really good sample of this writing by a slightly later king, Darius the Great. It is a solemn record of how awesome he is, and what he's conquered: it begins, "I (am) Darius, the great king, the king of kings, the king in Persia, the king of countries."

This is technically cuneiform, "wedge-shaped" writing. Cuneiform is the oldest kind of writing we know of, originally used by the Sumerians. But Sumerian cuneiform was invented for a different medium: they wrote it by pressing a wedge-shaped tool into clay, not by whacking spikes into cliffs.

The Sumerian writing system was already at least two thousand years old and on the decline when Cyrus and Darius came along, but Cyrus's people revived it and adapted it to his language (Old Persian) to make this type of monumental inscription. I don't know why, but I wonder if they had the same feelings about it as the Skyrim artists: it evokes dragon's claws and ancient secret meanings. Maybe Cyrus hoped to co-opt the mystique of ancient Sumerian and Akkadian writing, the same way we invent Latin mottos for modern institutions.

Older cuneiform,
from Code of
Hammurabi
(~1700 BCE)

But I think Cyrus improved on the state of the art. Compare the Behistun inscription with real old skool cuneiform back when it was still in regular use a thousand years before: on the right here is a sample from the Code of Hammurabi. Cyrus's relatively simple script is almost a true alphabet (it represents syllables with about 50 symbols), but Hammurabi was writing the Akkadian language with more complicated cuneiform shapes, with more varied shapes and angles, uneven spacing, and the symbols representing, like modern Japanese, a mix of sounds and meanings. To me, Hammurabi's writing has a sloppy, random look, and as you look at even older inscriptions, it gets even sloppier. I think the simpler systems of Cyrus the Great and Skyrim's Dragon Writing look more imposing as inscription than Hammurabi's did.  Each serves their purpose well enough; Hammurabi was trying to codify a stable system of law in a writing system people at the time understood, while Cyrus was trying to make people's hair stand on end.

It's time for a cuneiform revival: let's come up with a system for writing English using this style of writing, and start marking gravestones and monuments this way. The Roman alphabet has been fun for these last couple thousand years, and it's perfectly adequate for business expense reimbursement vouchers and vampire romance novels, but clearly it lacks the gravitas of stonecarved cuneiform.

Code bloat

Really, Microsoft? It takes 200.2 MB of code to convert an XML file? Really?

Use case stick figures

The stick figures in use case diagrams seem simple and featureless, but they undoubtedly have a rich inner life.

chi2009 -- day 3

Chi day 3. Everyone here has done a formative study to show how users go about some task on the computer or in their lives, generated a list of design criteria from that, mocked up a prototype, tested it in lab study, and shown with bar graphs that people do whatever it is 22% more efficiently.

If I add up all these 22% more efficiencies in every area of my life, I somehow don't see myself as 22% better off. Lots of things seem kind of cool in isolation, and I wouldn't mind being able to use them. But there's some kind of deeper "why" question I don't really see being addressed in the formative studies. Maybe it comes down to an ROI analysis: just because something is more efficient doesn't mean it's worth your time to switch to it: consider the costs versus benefits of switching the world from QWERTY to Dvorak. Dvorak is probably slightly more efficient, but the amount of campaigning it would take to change this, would be better spent campaigning for something else.

So, when studying people doing stuff with an eye towards making that stuff easier, there ought to be a rule of thumb you can apply to find out when to drop the whole thing and say "this person is actually doing OK; they don't really need something new". I don't know what that rule might be; it would have to be pretty subtle. The most comically inappropriate example was the guy presenting on Monday who was studying love with the idea of how a device might "improve" it. Sometimes people miss each other: this emotion serves a purpose, and it's not at all clear to me that we would be better off if we had technological gimmicks to trick ourselves out of it.

That's obviously a more extreme example than someone trying to help you search for restaurant reviews faster, or keep "to-do" notes all in one place, but maybe there's some ineffable factor in all of these things that's being missed.

chi2009 -- day 2

Some interesting stuff at CHI today:

- "Difficulties in establishing common ground in multiparty groups using machine translation": Naomi Yamashita talked about a study where they had a Chinese speaker, and Korean speaker, and a Japanese speaker all working together on a task over machine-translated instant messaging. Apparently this kind of works with two languages, but it's a disaster with three. She did detailed analysis of their chat logs to figure out where things went wrong: because you can't see the translation flaws between the other two speakers, you don't have enough information to understand why they make the adjustments they do in trying to communicate. So things spin out of control. They had some elaborate suggestions for fixing this, by loading up everyone with more information. I didn't really care for their solutions; I think doing three-way translations probably requires a specialized translation engine that maps words consistently among the three languages, even at the cost of some inaccuracy. I bet people could adapt better to some translation weirdness if it least everything was consistent and predictable.

- "Spectator Understanding of Error in Performance": this was a poster in the poster room, and I chatted with one of the authors. They had a psychological theory about how people interpret mistakes in musical performances, especially for unfamiliar kinds of music: did the musician hit the wrong note, or did the listener misunderstand what the musician was trying to do? The guy kindly engaged me in philosophical musing about it as I tried to think how it might relate to a programmer's perception of program errors when debugging.

- "Resilience through technology adoption: merging the old and the new in Iraq": Gloria Mark talked about internet, cell phone, and satellite TV use in Iraq. Fascinating stuff, but the biggest takeaway for me was this: bring a printout of your slides, so if the projectors or computers go horribly, horribly awry, you can give your talk without a computer in front of you. I gather that an Iraqi would have known that!

- Another poster by Anja Austerman, and Seiji Yamada: It was a very clever study about how people try to train robots. They gave people the task of training a robot. But the robot in fact already knew the task, and was programmed to behave or misbehave at certain times, in order to collect data about how the people tried to train it. Her idea was that a robot for the home might go through a pre-training phase like this, to learn its owners training styles; then later they could train it to do real tasks in a way that felt natural to them.

- alt.chi is a set of sessions with stuff that doesn't get accepted into the regular conference. So the talks are a little weird or speculative. There was one about how computers can help "improve love"; one on "thanatosensitivity" in design (product designers should plan for the death of the user and how the family is going to deal with the deceased's online identity, cell phone contacts, etc); and a study about people who do medical research online before they go see their doctor.

- Jet Townsend from CMU had a gizmo you can wear on your arm, that tells you where there are strong electromagnetic signals in the room around you. I wish I had the skills to do hardware hacking like that.

- Todd's been all a-twitter, and I've found out about cool stuff 2nd hand through him, like the MIT tour last night, and a Stockholm University shindig tonight, at which I met many cool people.

CHI2009 -- day 1

I'm in Boston for CHI, a conference about human-computer interaction. It's interesting, although some of the language in the talks is a little fluffy: lots of people doing research through design to explore a process of discovery that reveals empowering affordances.

Some of the actual stuff people are coming up with is pretty cool though, so I don't meant to knock the work itself. These designers have a genuine interest in making technology work better for people, so they're trying new interface techniques all the time and reporting on how well they work for people. For example I saw today some multi-touch pressure-sensitive mousepads, only with no mouse. Simple idea but it works very nicely, and probably a lot cheaper than multi-touch sensitive screens.

There are also some wacky muppet-labs kind of inventions, like the "crowdsourced haptic interface" where you let anonymous strangers give you backrubs over the internet. I'm not sure what problem that solves, but I guess it was worth trying.

After a reception tonight, some nice MIT grad students took a bunch of us over on a tour of the CSAIL building, where we saw roombas watering plants, Richard Stallman's office, discarded dusty robots lurking in corners, and we walked the whole length of an infinite hallway. CSAIL is housed in a Frank Gehry building, and all the plants died when they finally plugged up all the leaks that had been watering them.

Syntonicity

I learned a new word today, "syntonicity". Syntonicity is when you understand something by identifying with it, rather than understanding it purely abstractly. Here's a presentation by Stuart Watt talking about syntonicity in the context of programming language design.

Beginners to programming (and, frankly, all of us sometimes) have problems stepping through a program in their heads to see if it will work. They remember their intent when they wrote the code, but they have trouble putting themselves in the computer's shoes and seeing the program through its eyes. Watt claims that Seymour Papert was trying to aid this kind of thinking when he invented the Logo language. In most graphics languages, there's a set of coordinates on the screen, with (0,0) in the upper left corner, from the viewer's perspective. In Logo, everything's done from a cursor's perspective (the "turtle") as it moves around the screen.

Watt was involved in the creation of Hank, a specialized programming language for psychology students where information is shuffled around by a character named Fido. John Pane's language for children, Hands, does very nearly the same thing, but actually draws a dog ("Handy") on the screen with a pack of cards in its paw.

I don't know one way or the other if explicitly naming some part of your language or environment after an animal helps people think about programming the right way. But I can see where syntonicity could be a way of evaluating more subtle features about a language.

Consider for example the naming difference between F#'s workflows and Haskell's monads. They're similar concepts: a way of kind of quoting some code, so you can later say "execute it in the following peculiar way". The lines of the code are there, and you later say how information is passed from line to line. "Workflow" isn't a perfect description of that, but it gives the impression that you're going to hand it over to the processor later, and say, "do this work". "Monad" on the other hand is a mathematical term from category theory, that describes the mathematical relationship between the type signatures of the lines of code.

"Workflow" leads you to identify with the good old blue-collar working function you're going to hand this thing off to to execute thing, while "Monad", if you're a category theoretician, leads you to think about the types underlying the code as interesting theoretical objects: how can they be twisted around and changed and reapplied in new ways?

This fits nicely with the goals of F# and Haskell: the former is meant to be a practical programming language for scientific and financial programming, and the latter is primarily a research testbed and playground.

So, I think the names for things do matter when designing a language. If you choose names that sound like verbs, and those verbs have subjects that might be imagined to have a consistent point of view about your programming model, maybe it will help guide users into the right mindset for reasoning about your language the way you'd like them to.

Java beginners: beware the black hole of "static"

I've just been grading a bunch of programming assignments from college sophomores taking their second term of Java, and I've noticed a dangerous pattern.

Suppose you have a class like this in your project:

class Foo {
   private int state = 42;
   void doStuff() { System.out.println(state); }
}

The right way to use the doStuff() method is like this:

Foo f = new Foo();
f.doStuff();

But a common mistake is for people to try to access it like this:

Foo.doStuff();

The latter doesn't work, because it's the syntax for referring to a static method: that is, a method that belongs to the class rather than to each particular object in the class.

So, if you make that mistake, Eclipse gives you the option to

change modifier of doStuff() to static?

Don't do it! Run away! That's not the right way to fix it. In most cases, it means you forgot to create a Foo object.

But suppose you take Eclipse's bad advice anyway. Then you get a new error, because now doStuff() can't see the integer state anymore. What does Eclipse offer to do for you? You guessed it:

change modifier of state to static?

If you do that, then your program will seem to work again. But the meaning is very different, and as soon as you try to use multiple instances of this Foo object, your state variable will appear to do crazy things -- because it will be shared between all your Foo's.

So beware of static. If you have a method that refers to variables that aren't either passed in as arguments, or declared inside its brackets (like state, in this example), then it probably shouldn't be static.

As for making data items static, ask yourself this: think of some future modification of this program where you might need to have several different objects made from this class. Do you want them all to have the same value for this variable? If the answer is "no", then using static is bad form, even if it doesn't currently cause a problem.

So do the extra work and eliminate "static" from your Java code if you've added it in "just to make it compile". It'll be tedious to get it to compile again. It might require you passing extra parameters to methods, or creating new object variables. But take the time to figure it out. Learning this will give you the right intuitions, and will save you headaches later on in your programming career.

Learning from game designers

Here's a very readable presentation about how computer game designers
manage to build in a more evenly-sloped learning curve than application designers
do.

Princess-Rescuing Application

What kind of debugging should we be studying?

I really think, but just can't back this up, that a lot of the time when programmers are debugging, they're doing it on code that they're pretty familiar with. The research literature empirically studying people debugging, seems heavily biased towards comprehension and debugging of code that people are seeing for the first time. This has got to be a methodological issue -- it's just so much easier to bring people into a lab and hand them code they haven't seen before.

Some people I've asked about this seem to disagree -- they think it's commonplace for programmers to be thrown into new unfamiliar code in their workplace and have to figure it out. Could be, but I still think they spend most of their time around familiar code, hour by hour. And that will judge their debugging tools by how well they work for them in familiar code.

This makes a big difference, because people who are debugging familiar code already have some understanding of how it works, and they're trying to compare the real code against the understanding in their head, to see where they, or it, are going wrong. So it's kind of a scientific process, coming up with hypotheses and refuting them by testing them out, until they narrow down the exact point where the code goes astray from their notions about it.

People debugging unfamiliar code, on the other hand, are going to start with a comprehension phase where they go bottom up, looking at it and just trying to make sense of it. If academic research, and product development research, puts too much focus into understanding that particular situation, I think it will be giving short shrift to the later phases of code maintenance.

Help with study of functional programmers

Are you currently developing or maintaining a medium to large-sized
program written in a functional language, such as Haskell, F#, OCaml,
or Lisp? I'm doing a study of functional programmers, as part of
a research internship at Microsoft, and I would like the opportunity to look over
your shoulder while you do debugging or coding on your project.

I'm looking for people with at least a year's experience doing
functional programming, and who are currently working on a real
project (i.e. for some purpose other than learning functional
programming). I'm only allowed to use people who can work in the US
(because of the gratuity, which is taxable income). I'd simply come
watch you work, and ask a few questions along the way. You'd do
whatever you would normally be doing. If you're near Seattle or
Portland, I'd come to your office for a couple of hours. If you're
not near Seattle or Portland, then we'd set you up with LiveMeeting
or some other remote screencast software so I can watch you from here.

Obviously security concerns are an issue - I will not share any
proprietary information that I learn about while visiting you.

In exchange for your help, Microsoft will offer you your pick of free
software off its gratuity list (which has about 50 items, including
Visual Studio Professional, Word for Mac, XBOX 360 games) or any book
from MS Press.

We're doing this because expert functional programmers have not been
studied much. We plan to share our findings through academic
publications, to help tool developers create debugging tools that are
genuinely helpful in real-world settings.

I'm hoping to finish my observations by August 8th, so please contact
me immediately if you're interested!

Thank you,

Chris Bogart
425-538-3562
t-chribo@microsoft.com

icfp contest update

Well, I got threads and tcp and everything all figured out, and my simple "spike" solution works the way I intended it: the rover makes a beeline for home base, ignoring craters, martians, and boulders, and usually dying or crashing.

However I can't get it to run in Mono on the LiveCD environment for the contest. I've been using Visual Studio on a virtual machine on my mac. Visual Studio runs a little slow, but the program runs fine. But when I try to run it in mono, I get a big wodge of error messages, none of them pertaining to my code. I used some new features of F#, so it may be that it exercises some corners of mono that haven't been tested yet.

I think at this point I'm going to give up on submitting a solution and just post interesting bits of my code here after the contest ends. I still want to think some about a quick and dirty way of avoiding craters, but I probably won't spend much more time on it: it's nice out today!

I feel like I've gotten what I wanted to out of this: building something significant in F#. I have 238 lines of code, and I used F#'s Erlang-style mailboxes, asynchronous workflow, discriminated unions, class definitions, and I overcame the headaches of separating things into multiple modules while making sure all references went backwards. I got threading and networking in a pretty clean way I think. The ugliest part of the code is actually probably the navigation code, that looks at where the rover is and how fast it's going, and decides how to head for home base.

ICFP contest day 2

I spent a few hours programming last night, and I still don't have a running program yet. F# turns out to have some pretty interesting monad-like things for handling asynchronous code (they're called "workflows") and this appears to be the right way to handle sending and receiving stuff from a TCP/IP connection.

I think I have a pretty good idea how to make this work, but I'm banging my head against the details: for example I want to asynchronously read a line of text from a socket, passing it along to the rest of my program as soon as a carriage return is received. So, do I have to read one byte at a time in a tail-recursive function, adding to a mutable string? Or is there an object or something in the library that will do this for me? There's a tradeoff between searching and searching in the docs to find the most natural way of doing it in the language you're trying to learn, versus writing it from the ground up yourself, probably less efficiently, as a newbie.

My problem may be that I need to sit down and read a book on .NET instead of thinking I can adhoccumulate expertise efficiently.

By the way, I like Expert F# better than Foundations of F#. I wish I'd read the former instead of the latter as preparation.

Wish me luck!

ICFP contest! Rovers on Mars!

The 2008 icfp contest just started! Sadly I'm at work for another several hours, but I skimmed through the description, and hopefully the programmer-lobe of my brain is already hard at work on the problem while my researcher-lobe is busy dealing with participant recruitment for a study.

This one is not all parsey like the last two years: it's all real-time execution and floating point numbers in a navigation problem. It's not the kind of thing I've thought about much, so it should be an interesting challenge. Also, my laptop plug *just* stopped working, so I may have to complete the entire task in 2 hours and 27 minutes. Hopefully they have extras in Seattle somewhere.

My plan this year is to do it in F#, since it's cool and also I'm doing a study on it for my summer internship. But they made it a little harder by not including F# in the list of supported languages. But no worries -- they do have mono (the .NET clone, not the disease), so I'll just have to upload an executable.

5 Hints for Beginning Haskellers

Here are some things that have caused me headaches in learning Haskell.

1. Function calls. Functions are called by giving the name and the arguments separated by spaces. If a function f has two parameters, a and b, you call it with f a b not f(a,b). The latter would be a one-argument function whose argument is a pair. You can do things that way if you like, but it'll make things a lot trickier later.
   
2. Precedence. Those spaces in the function call syntax are about the tightest binding thing in the language. So if distance is an integer, and show is an (Int -> String), then show distance++" miles" is good syntax, because show distance is evaluated before the ++. On the other hand, if c is a character and cs is a string, then length c:cs is a type error. Haskell will try to evaluate length c first, then prefix that number to the front of the list cs. Instead, you want length (c:cs).
   
3. Parens in patterns. This is really an example of the precedence issue above, but it trips me up all the time. You need more parens on the left hand side of function calls than you'd think. I don't know how many times I've written code like the sample below, leaving out the parens at first.
   
• data Coord = Coord Int Int
• manhattandist :: Coord -> Int
• manhattandist Coord x1 y1 = x1 + y1 WRONG
  
• manhattandist (Coord x1 y1) = x1 + y1 RIGHT
  
4. Arrows in type declarations. The type of (+) is Int -> Int -> Int; and we're supposed to accept that the first two Ints are the arguments, and the last Int is the return value. But turns out there's a cool reason for this. An equivalent type declaration would be Int -> (Int -> Int) (because -> groups right). You can think about addition this way, and Haskell cooperates nicely: + is a function that takes one integer, and returns a function. So plus x y = x+y is equivalent to plus x = \y -> x + y (search for lambda in the Haskell docs if you don't know that backslash syntax). So if you need a one-argument function (that returns a function) for some reason, it's OK to go ahead and define it as a two-argument function returning a number; Haskell doesn't make a distinction, and it's more convenient sometimes to express it that way.
   
5. Monads. These are famously hard to get. There are 10,000 tutorials out there, and most of them help a little. I'd recommend you start by learning to use the important standard ones, like Maybe, IO, and List, more or less by rote. Beyond that, if you're in a situation where you'd need to roll your own, I'd say just forget monads exist and write the code yourself. Once you get so you really understand the ugliness involved firsthand, monads will be an obvious benefit. Anyway, that's my plan right now. I think I get them well enough to explain them, but I make a lot of stupid mistakes when I try to use them, so I probably don't understand them as well as I think I do. (More on this later -- I'm taking two classes right now that rely on Haskell, so I'll comment on this more as I have little insights).
   

Types to distinguish parameters and dimensions

Suppose a language had a rule that said a function could only have one parameter of a given type, and each dimension of a multidimensional array had to be indexed by a different type.

This would have several benefits:

1. You'd get extra fine-grained checking; you couldn't possibly get parameters mixed up
2. When making array slices or currying, you wouldn't have to concern yourself with the ordering of the parameters.
3. You wouldn't need to distinguish between optional named parameters and ordinal parameters. The name of a parameter would be (or would be derived from) it's type.

With functions like + that seem to require two items of the same type, you could interpret the two (or more) arguments as a collection. Lists certainly could contain homogeneously typed members.

Python to the rescue: ICFP contest 2007

The ICFP contest was great fun this year, as usual, and I actually arranged travel and moving plans so that I would be in one place for the weekend to work on it.

I resolved to work in Scala this year, because it seemed like a practical functional language I'd like to know better; and it allows imperative programming with the java libraries, so I thought it would be a smooth learning curve.

Well, I spent the first day butting my head up against Scala, and dealing with condo repair contractors and a fussy HOA board in the meantime: at the end of the day I had basically no running code and a splitting headache. The cure: beer and python.

The second day I changed to Python, which I really haven't used *that* much, but it's easier to figure out how to do things. I'm not sure why -- maybe it's a question of documentation, or maybe it's just that I don't get functional programming as well as I thought I did.

In Python I managed to get an interpreter that basically worked, reading the DNA file and creating images using Tkinter. I don't think it was ever flawless. There was a self-test screen that I eventually got all "OK"s on, but I never did see these manual pages that people on the discussion list were talking about.

The Python was still too slow to make further debugging feasible, so I rewrote it in C. Got that working sometime Sunday, and it was faster, but it wasn't faster enough. The real solution needed to be linked lists to pointers to memory, so one wouldn't need to shuffle megabytes of DNA around -- but I had no mental energy to implement anything like that by that point.

Lessons learned:
Either scala documentation needs work, or I just need to spend more time learning the paradigm
When the problem statement says that shifting byte strings around needs to be done in faster than linear time, that's an optimization I should think through up front, not "get to it later". It made all my implementations useless.
Python is the best language I know of for expressing algorithms. It isn't fast enough, though. (although I wonder what if I tried the linked-list thing in python? I may try that as an experiment)

There were some complaints on the discussion list by people saying that the up-front work was too hard, and that most people never got to the meat of the problem. I totally disagree with that. ICFP is all about making functional programming usable and efficient. If, with current tools, we are forced into low-level bit-schlepping in C, it points to a deficiency with the most common tools of functional programming, or in the training of programmers. The contest ought to be about illustrating that kind of issue in a fun way.

Here's an interesting point: languages with higher-order kinds of functionality tend to abstract one away from the machine more; working with heavy integer "objects", for example, instead of raw machine integers as C does. Is it possible to have a language where you can talk about bare metal, even registers and such, but do it in a higher-order abstract way when desired? I guess C++ attempts to do this in some ways, but it's an extremely complex language. Is there no better compromise?

Representin' in Oregon

I haven't posted in a while, but I have good news -- I've been accepted into the PhD program at Oregon State University! I first discovered the program by googling what turned out to be Margaret Burnett's turn of phrase, "HCI of Programming".

It's been 15 years since I got my master's at Colorado State. Turning 40 last year, I've been thinking about where my life is headed and what I wanted to do next. I've been doing very practical programming in the title industry in the daytime, and thinking and writing about programming languages at night. So now I'm back in school, and although the faces have changed, the hassles are all the same.

Up until now I have thought more about the challenges of language design for professional programmers, but at OSU I'll be diving into the whole realm of end user software engineering.

When designing a language for a professional programmer, maybe we can assume that the user knows what he or she is doing. Programmers are expected to have an analytical mindset, and plenty of practice at debugging things. They have endured years of indoctrination into the sacraments of testing, code correctness, documentation and version control. They have fingernails that shine like justice. They have RTFM. It's safe to assume that they have this expertise. It may not actually be true that they have it, but it's safe: our asses are covered, because we said on the box "professionals only".

An "end user" may in fact be a professional programmer, but we no longer have our asses covered; the environment must share in the responsibility. There is an intelligent being sitting in front of us, who thinks in a very different way than the computer does. They have some concept of what they want the computer to do, and we have to cooperate with them in representing that concept. Without being making so many wrong assumptions, or so few right assumptions, that we annoy the user into walking away in frustration.

When I was going through the application process, I went for advice to University of Colorado professor, Clayton Lewis, who introduced me to the notion of Theory of Representation: that a lot of what's interesting in computing can be seen in terms of making valid mappings from one representation system to another. In those terms, HCI is about mapping between formal and human representations. But the human representation system is a pretty quirky thing that cognitive scientists are still attempting to explore; so HCI is kind of an engineering discipline working ahead of its time: trying to translate in and out of human representations before human representations can really be said to be understood by science.

This process has got to be a two-way street, because human representations of programming solutions are invariably wrong to start with, at least at a sufficiently fine level of detail. They need refinement, and the process of formal representation helps with that refinement.

In HCI of programming there is more going on, however. Engineering is not just a set of concepts, it's a discipline. "Discipline" is a deep set of mental habits. So to turn end users into software engineers, or to turn software engineers into better software engineers, it seems to me that an ideal software engineering environment will do more than represent users' ideas; it will help teach the user the mental habits necessary to refine those ideas efficiently. Maybe that goes beyond theory of representation.