I’ve been building out a small suite of command-line tools for working with ChatGPT, GPT-4 and potentially other language models in the future.

The three tools I’ve built so far are:

• llm—a command-line tool for sending prompts to the OpenAI APIs, outputting the response and logging the results to a SQLite database. I introduced that a few weeks ago.
• ttok—a tool for counting and truncating text based on tokens
• strip-tags—a tool for stripping HTML tags from text, and optionally outputting a subset of the page based on CSS selectors

The idea with these tools is to support working with language model prompts using Unix pipes.

You can install the three like this:

pipx install llm
pipx install ttok
pipx install strip-tags

Or use pip if you haven’t adopted pipx yet.

llm depends on an OpenAI API key in the OPENAI_API_KEY environment variable or a ~/.openai-api-key.txt text file. The other tools don’t require any configuration.

Now let’s use them to summarize the homepage of the New York Times:

curl -s https://www.nytimes.com/ 
  | strip-tags .story-wrapper 
  | ttok -t 4000 
  | llm --system 'summary bullet points' -s

Here’s what that command outputs when you run it in the terminal:

Let’s break that down.

• curl -s https://www.nytimes.com/ uses curl to retrieve the HTML for the New York Times homepage—the -s option prevents it from outputting any progress information.
• strip-tags .story-wrapper accepts HTML to standard input, finds just the areas of that page identified by the CSS selector .story-wrapper, then outputs the text for those areas with all HTML tags removed.
• ttok -t 4000 accepts text to standard input, tokenizes it using the default tokenizer for the gpt-3.5-turbo model, truncates to the first 4,000 tokens and outputs those tokens converted back to text.
• llm --system 'summary bullet points' -s accepts the text to standard input as the user prompt, adds a system prompt of “summary bullet points”, then the -s option tells the tool to stream the results to the terminal as they are returned, rather than waiting for the full response before outputting anything.

It’s all about the tokens

I built strip-tags and ttok this morning because I needed better ways to work with tokens.

LLMs such as ChatGPT and GPT-4 work with tokens, not characters.

This is an implementation detail, but it’s one that you can’t avoid for two reasons:

1. APIs have token limits. If you try and send more than the limit you’ll get an error message like this one: “This model’s maximum context length is 4097 tokens.