Lisp

A Brief History of Cutting Code in a Universe Three Doors Down

| | | | |

Eventually upon a time, in a universe far far away — or three stack frames down the line, to be precise...

Assembly is the ur-knife, known to man from the dawn of time. The flint edge is very sharp, but uneven and brittle. These tools can be fashioned both into knives for cutting and axes for hacking. In either case the edge will wear dull soon. Rarely are these venerated instruments now brought out in the modern day, and when they are, there is great ceremony involved. He who has the fortitude and stamina to hack down a Big Tree using a mere assembly axe, all the while uttering the sacred ancient mantra the meaning of which is now lost, will be much respected among his peers - though perhaps not considered entirely sane.

BASIC is a children's game that remains popular among youngsters. These plastic bricks can be quickly assembled together to form a toy with a certain charmingly crude, and entirely non-lethal, resemblance to an adult's blade. But beware of gravity, little one, as with too long a blade your construction will surely shatter at the midpoint. Some time-honored traditional techniques for counteracting this brittleness are chewing gum and duct tape.

C++ is the leading brand of gas-guzzling, noisy chainsaws. It'll fell the tree quickly all right, but needs serious muscle and experience for correct handling. The attrition rate among woodcutters is high, with many junior apprentices never returning from their first trip to the woods. Even senior loggers are often missing the occasional superfluous limb or two, but no matter: they've developed the forbearance and impressive skill to properly handle these beasts in single-handed combat. As they say, good judgement comes from experience, and experience comes from bad judgement - if you survive it.

Java is now the most widely-used electric kitchen knife in the world. To prevent people accidentally cutting themselves up too badly, the blade is manufactured to a guaranteed consistently dull quality. Indeed this tool was originally developed in response to all the woodcutting accidents taking place with C++, which is perhaps curious considering that its safety features make it somewhat tedious to hack at anything more ambitious than saplings. But, safety first or not, it remains splendid at cutting bread.

Smalltalk is a voodoo stick. Wave it over a carefully constructed model of the real world, and stuff happens - trees fall, timber is cut to spec, and paper flows freely. It has long been a jealously guarded trade secret how this works, exactly, but we do know that the inventor is associated with Lisp - which we'll get to in a minute. Unfortunately, this powerful technology was and remains so far ahead of its time that it languished in obscurity long enough for the patent to expire, allowing some eastern philosophers to popularize a commercially highly successful clone. Catering to slightly more vulgar tastes, this clone may have sacrificed some of the original aesthetics, but not much of the seemingly supernatural power.

Forth self-assembling nanoblades represent the keenest baryonic blade edge currently available. Formed out of femtoscale building blocks that spontaneously stack themselves into arrangements of factorially increasing complexity, these axes will hack through a trunk with a single heave. They will also continuously and autonomously repair themselves so as to maintain the perfect edge down to the molecular level. The best neurosurgeons now refuse to use any other scalpel, and the razor blades made of this material provide for the closest shave you've ever had - just watch out that you don't shave off a little extra.

These are all ingenious inventions of man. But of all the cutting and hacking devices known to us, there is one exalted above all others, and it lacks an earthly origin.

Lisp is, as far as we can determine, magical. Obscure in origin but manifesting itself in our corporeal reality in the form of lightsabers, the crystal nanocircuitry of these devices appears to transcend the ordinary laws of nature as we presently understand them. Infinitely sharper than any scalpel, indestructible under a thousand atmospheres of pressure, these elegant instruments are made of an unknown alloy and keyed to their owner in such a way as to ensure they will never cut him.

Brace yourself when you initially receive yours from your elders. When you grasp the 'saber in your hands for the first time, parentheses and other cuneiform symbols will flicker in and out of your inner vision, and your ears may ring with thunderclaps not unlike the sound of continuous stack overflows. You may indeed have reason to wonder if the red pill you washed down with that lambda-tinted kool-aid at your initiation ceremony wasn't some form of psychedelic drug after all.

Fear not; while research is still ongoing, this is thought to be a harmless side-effect of nanites that have slipped into your bloodstream through the handle and are reconfiguring your brain to accept their base code. The symbols you may see are hypothesized to be a transient interface adapter between the human visual cortex and the alien technology of the 'saber, quickly supplanted by some more direct mechanism that we haven't quite figured out yet.

If you persist in confusion for a moment longer, the hieroglyphics will soon begin to fade into the background, becoming a subconscious part of your inner vision and lending your perception an acuity you didn't even know was missing. You will perceive not merely the apparent hierarchy of nature but penetrate yet deeper to the underlying infinitely more subtle relationships and interconnections between things.

Once the symbiotic link with your 'saber is fully established, you will be able to, at will, slice through the molecular bonds of any Gordian Knot as if they were made out of air. Given that you may will the width of the blade's swath to reconfigure itself from picometers upwards, your 'saber is easily the most precise and capable instrument known to man. The rush and elation, and yes, the pride, of wielding a lightsaber is enough to make anyone giddy. You should understand that with great power comes great responsibility, and you are advised to review the ethical code from time to time.

After further meditation and practice, you may eventually come to understand that it is not the blade that cuts, but your mind that sculps reality according to your imagination. This insight is reported to be the last thing uttered by some of the legendary Lisp grand masters before they, somehow, cut the ties that bound them to our sphere of reality, thus vanishing into thin air.

There is some speculation that, after contemplating year upon year on the grid lines that all Lispers are able to perceive in the night sky, these Lisp masters held infinity in the palm of their hand and found a nearby quantum activation frame, unlocking a suspended gateway continuation that allowed them to ascend to some higher plane of being that, evidently, encompasses and sustains our own.

A competing hypothesis among those of a more hedonistic bent is that perhaps the vanished masters managed to construct a private, entirely self-contained lexical scope, i.e. sub-universe, of their own, a form of personal nirvana obeying their own choice of evaluation rules and containing the promise of limitless futures to explore and indulge in.

There is only one way to find out - apply for a wizard's robe today and begin your own journey from caveman to first-order hacker of the heavenly spheres in this universe and beyond.

Finding Clojure, or How I Learned to Stop Locking and Eval the JVM

| | | |

Ever since the Scheme R6RS roadblock, I've been looking to branch out into other Lisps outside the Scheme ecosystem. The good news is that there's no shortage of nascent Lisps and Lisp-wannabes, with many interesting new undertakings being launched month upon month.

There's one in particular, called Clojure (pronounced closure), that was unveiled recently and effortlessly stands out from the bunch. There's much to like about Clojure, but first and foremost, it follows a principle I'm very fond of: code talks, bullshit walks.

R5.97RS Scheme Considered Harmful

| | | | |

The electoral vote regarding the ratification of the final R6RS draft closes momentarily. After spending much of the weekend giving this latest draft a fair reading (which took some work as it has ballooned to over 160 pages, contrasted with the 50 pages that sufficed to define R5RS), I’ve bit the bullet and voted against ratifying it. As all votes and their explanations will eventually (as of August 26) be a matter of public record, here is the ballot I submitted, with formatting and relevant hyperlinks added:

Deliberating on the latest R5.97RS draft I find much that is agreeable about it, and it would be tempting to vote for its ratification. Indeed if but the programming language described therein would be named anything else than Scheme, it might seem the promising start of a practical and useful modern Lisp dialect.

Yet as the R5.97RS main and library reports still read today, the resulting language is most definitely not Scheme. My primary issue with the draft is that it betrays the spirit of its very own opening paragraph: “Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary.”

I should very much hope that paragraph will never be removed in future RnRS reports! Evaluating the rest of the main and library report against that principle, I think it stands out that the draft has simply suffered considerable feature creep and is vastly overspecified and overcomplex. In a word, it is too ambitious (or presumptuous, depending on your point of view).

I’m convinced the draft as it stands would derail Scheme, in the sense of putting the language on a course that it is unlikely could be easily corrected through later RnRS iterations. Thus I must at this time, with apologies to the editors, vote against ratification and hope that there will be enough dissenting voices to reopen the editorial process.

If the vote concerned only the main report in something not too far from its present form, ratification would not be an unfeasible prospect; and this despite the report having almost doubled in size. I do share others’ misgivings on issues such as requiring the full numeric tower and chaining Scheme at the hip to Unicode; still, the main report all in all appears decent, and a standardized library definition facility, even if the current proposal seems rather overcomplex, would be a most important practical benefit provided by core R6RS.

However, the proposed standard library report is simply unacceptable. Not only does it gratuitously eschew important and widely-used existing SRFIs (such as the SRFI-1 list library, and SRFI-69 hash tables) in favor of its equivalent incompatible functionality, but one must ask why features such as, say, byte vectors, enumerations and hash tables should even be included in the RnRS reports in the first place?

The desire for a standard library is widespread, both for reasons of portability and practicability, but I think it is clear that when it comes to library functionality, the SRFI process has proven itself and works better than the committee process being voted on here. Attempting to evolve a standard library through the rightly slow-moving RnRS process seems to me shortsighted and counterproductive in the long run.

Thus, I consider the standard library report largely superfluous and indeed harmful. Instead of attempting to set in stone a controversially large standard library with the RnRS process, I believe the R6RS committee should strictly confine itself to the core language only, and fully embrace the SRFI process for the standardization and dissemination of all non-essential library features.

God Sure Didn't Write in Java

| | | | |

It bemuses me to read about people who are seriously working on artificial intelligence using stone-age programming languages like C++ or, heaven forbid, Java. Take this following backstory excerpt, for example:

Sure, the thinking machine might not get finished in the 3 months that our seed money would last us. But, in the meantime, to tide us over, we’d solve a simpler problem: we’d use some of the bits and pieces of our unfinished AI engine to predict the financial markets.

The technical co-founders and I had been working on the first version of the AI engine for many months, by the time the seed funding came in. A healthy amount of software code existed (although the code itself wasn’t entirely healthy). ...

Now that we knew what we were doing better, we moved further and further away from the brain as a concrete design inspiration. ...

We found the increasing complexity of the various agents in the system was stressing the codebase. After a lot of difficult debate, we decided to grit our teeth and rewrite the core of the system from scratch. ...

Cassio proved to be an outstanding manager as well as an excellent software engineer and designer, and we let him accumulate assistants until, at one point, we had 60 people there out of a total company staff of 130. ...

We were quickly realizing what should have been obvious from the get-go – that getting our thinking machine to work could well be a multi-year pursuit. ...

The Webmind system we had a month ago consisted of over 750,000 lines of Java code. ... Most of the 750,000 lines of Java is still useful — it covers issues like communicating with other software processes, balancing processing among different machines, reading parameters from files, and so on and so forth. Necessary infrastructure. ...

I reckon that, at this point, I’m at serious risk of becoming the Charles Babbage of AI. Babbage designed the first computer — a purely mechanical computer, pre-electronics. But it was just too damn complicated to build using the technology he had at his disposal. He spent all his money and his life on it, and never got it done.

— Ben Goertzel, Waking Up from the Economy of Dreams

If you’re going to attempt scaling the formidable, cloud-obscured peak of Mount Intelligence, you’d think that choosing the best possible toolset would be of some importance. While you might certainly overcome a small hill even crawling barefoot, blindfolded, and with your hands tied behind your back, why in the world would you suppose that this approach could possibly apply to the single most difficult climb out there?

Lessons from the Science of Nothing at All

| | | | |

Richard P. Gabriel is musician, poet, and respected Lisp hacker. Probably generally best known for coining the phrase “worse is better“ with his essays on why Unix and C prevailed over superior alternatives, he’s also the author of several books including Innovation Happens Elsewhere (with Ron Goldman).

I’ve been greatly enjoying reading his various works from the past couple decades, and recently stumbled across a particularly relevant essay of his that I want to share. It’s called Lessons From The Science of Nothing At All, and begins poetically enough: “Where I come from we make things from nothing – from dreams and fantasies.”

The Road to Enlightenment Is Littered with Irritating, Superfluous Parentheses

| | |

I, Arto Bendiken, do solemnly offer these my responses to The Road to Lisp Survey:

When did you first try Lisp seriously, and which Lisp family member was it?

I read Peter Seibel’s Practical Common Lisp and Paul Graham’s On Lisp sometime in mid-2005 or thereabouts. Both proved to be useful introductions to Common Lisp and gave me an initial appreciation of the unique characteristics of Lisp-derived programming languages.

After being sufficiently enlightened by those two books, I moved onward to Scheme. This came about primarily as a result of struggling my way through the Wizard Book and watching the 6.001 video lectures, as well as perusing various online tutorials and papers. I’ve been programming in Scheme ever since, with the occasional foray into Common Lisp.

Spam Filters, Alien Technology and Ruby on Rails

| | | | | | | |

Lisp - made with secret alien technology

When Paul Graham’s A Plan for Spam made its dramatic entrance into the anti-spam battle four years ago, it heralded the beginning of the end for spam — as we knew spam back then, anyway. Applying a simple statistical approach, based on word frequency analysis with a naive Bayesian classifier, Graham described how to create a spam filter accurate enough (99+%) that false positives effectively ceased to be an issue.

The central idea of the Graham Algorithm was quickly adopted en masse by spam filters, and as a result, the spam arms race has in the past few years tipped in favor of the good guys. “Successful” spam has devolved into exactly what Graham predicted it would: “some completely neutral text followed by a URL.” For me personally, the combination of good server- and client-side filters has made spam yesterday’s problem. (Well, that, and using Gmail as a front-end for lower-priority e-mail; spam all you want, it’s Google’s problem and they’re up to the task.)

Recently at $WORK, we’ve succeeded in applying similar text classification principles to another unrelated problem area, with the intent of forcing the computer to do the tedious job it was invented for, allowing us super-apes, in turn, to spend more time under a palm tree on the nearby beach, sipping tinto de verano and working out answers to deep existential questions, or whatever else it is that one does on the beach (note to self: need more practice).

GeoIP API for Chicken Scheme

| | | | | |

As part of a distributed, geographically load-balanced DNS daemon I’ve been developing in Scheme, I’ve put together a set of GeoIP API bindings for Chicken Scheme that I’m hereby making available under the MIT License.

This was the first egg I’ve developed for Chicken — not to mention the first time I’d needed to make any real use of Chicken’s FFI — and I have to say I was pleasantly surprised at how easy it was to create.

It was gratifying to be able to go from zero to completed egg in only a couple hours simply by looking at some other eggs for examples on how to use foreign-lambda , create a setup spec, and write the eggdoc documentation.

The GeoIP API subset provided by the egg is straightforward to use. Basically, you feed in a hostname or an IP address, and you get back a country code or country name. Country codes can be returned as either the 2-letter or 3-letter codes defined in ISO-3166-1.

Poor Man's MySQL API for Gambit-C

| | | | |

Lately I’ve been familiarizing myself with Gambit-C. This has mostly been preparatory work in anticipation of the imminent release of Termite, Guillaume Germain distributed computing system inspired by Erlang and implemented using Gambit’s serializable continuations and lightweight processes, scalable up to millions of concurrent threads.

Gambit is definitely one of the more interesting Scheme implementations out there, not to mention among the fastest; its creator Marc Feeley has been described as a performance junkie, and according to the Gambit manual, “with appropriate declarations in the source code the executable programs generated by the compiler run roughly as fast as equivalent C programs.”

However, despite its technical excellence, Gambit doesn’t, as of yet, have much of a standard library.

Yesterday I needed to interface Gambit with MySQL, and couldn’t find any existing binding to let me do that (though an SQLite 2.x interface does exist). As I wasn’t familiar enough with Gambit’s FFI system, I took the über-simple, lazy man’s approach: I created a trivial wrapper API on top of the mysql and mysqladmin command line utilities.

Macro Expansion Hook for Gauche

| | | |

Inspired by Common Lisp’s macroexpand-hook, here’s a pretty useful snippet I recently hacked together to assist me in debugging Scheme macros for Gauche :

(define-macro (define-macroexpand-hook hook)
  `((with-module gauche define-macro) (define-macro formals . body)
    (cond
      ((list? formals)
        `(define-macro ,(car formals)
          (lambda (,@(cdr formals)) ,@body)))
      ((symbol? formals)
        `((with-module gauche define-macro) ,formals
          (lambda args (apply ,',hook ,@body ',formals args))))
      (else
        (error "symbol required, but got" formals)))))

define-macroexpand-hook is a straightforward (well, not too hairy, anyway) macro-generating macro that overwrites the binding of the define-macro special form in the current module, with the result that any macros subsequently defined will include a call to a user-defined procedure (passed as the argument to define-macroexpand-hook), which will be invoked whenever the macro is applied.

(The immense power inherent in this brief code snippet will not, sadly, be something most programmers unfamiliar with Lisp can appreciate. What this is doing is essentially dynamically injecting additional code into the Scheme compiler itself — it’s not every language that lets you patch the compiler on the fly, while running.)

Syndicate content