Recently I set out to train a Transformer model, based on Distil-GPT2, to write something like my mothers’ poetry. After much searching for the most concise way to do this, I think I’ve figured out a reasonable easy-to-understand approach that works for me in Google Colab. In [2]: !pip install fastai Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/Continue reading “Fine-tuning a Transformer model using fast.ai”
Author Archives: dan
Thoughts on GPT-3 and creativity
Over the last few days, since gaining beta access to the GPT-3 API, I have feed it many prompts with a few different parameters and wanted to share some observations. What is it for? For me, this is the biggest open question about GPT-3. While there is a lot of useful discussion to be hadContinue reading “Thoughts on GPT-3 and creativity”
GPT-3 Story sample
Recently I got early access to OpenAI’s remarkable GPT-3 API, which you can apply for here: https://beta.openai.com/ The GPT-3 model is based on concepts described in the paper “Language Models are Unsupervised Multitask Learners”, and you can find code to generate these kinds of models on Github, and a nice overview in this post fromContinue reading “GPT-3 Story sample”
This site is coming back to life
This site has languished in WSOD land, and now that I’m on lockdown and have a few months without regular work I’m bringing it back to life. Stay tuned for various projects I will be using to fill the time and, hopefully, make myself useful.
Irony overload as undecided voters criticise flip-flopper
Check out the buried lede in this article from Reuters: “Romney just has to prove that he’s conservative enough for me,” said Eleanor Stump, a 70-year-old Tea Party member from Sheldon, Iowa. “I don’t like the way he’s flip-flopped.” Stump said she initially supported Herman Cain, who dropped out of the race after charges ofContinue reading “Irony overload as undecided voters criticise flip-flopper”
Reminder: Pay ghostwriters better
I bet @markdavison deletes these in a hurry.
MoJo week 2: Adoption
In this week’s MoJo lectures, we listened to Mark Surman and Chris Heilmann of Mozilla talk about the new HTML5 standards and capabilities in the Firefox browser, and from John Resig on how to build and maintain an open source product.
The Power of Prototyping
This is the first in what will be a series of blog posts for the Mozilla-Knight Journalism Challenge. This week we heard from Aza Raskin, former creative lead for Firefox and head of user experience for Mozilla, talking about the power of prototyping for understanding problems and building momentum behind your solution.
We’re back
So I totally got hacked. Reminder: be vigilant about upgrading wordpress and its plugins, or bad things happen. The hack looks like this, and appears all over the place, at the top of some files and the bottom of others. It decodes a huge blob of encoded text that was stuffed into one of theContinue reading “We’re back”
One of my favourite stories ever
As told by Douglas Adams a few years ago. I laugh every time I read this, and today I shared it with my wife for the first time and so felt compelled to share it with you, too. This actually did happen to a real person, and the real person is me. I had goneContinue reading “One of my favourite stories ever”