Category Archives: General

Recently I set out to train a Transformer model, based on Distil-GPT2, to write something like my mothers’ poetry. After much searching for the most concise way to do this, I think I’ve figured out a reasonable easy-to-understand approach that works for me in Google Colab. In [2]: !pip install fastai Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/Continue reading “Fine-tuning a Transformer model using fast.ai”

Over the last few days, since gaining beta access to the GPT-3 API, I have feed it many prompts with a few different parameters and wanted to share some observations. What is it for? For me, this is the biggest open question about GPT-3. While there is a lot of useful discussion to be hadContinue reading “Thoughts on GPT-3 and creativity”

Recently I got early access to OpenAI’s remarkable GPT-3 API, which you can apply for here: https://beta.openai.com/ The GPT-3 model is based on concepts described in the paper “Language Models are Unsupervised Multitask Learners”, and you can find code to generate these kinds of models on Github, and a nice overview in this post fromContinue reading “GPT-3 Story sample”