2025-08-13
Coming up for air after a bunch of time spent on a project trying to use llm as a router to different models to create a compatibility layer for the OpenAI /v1/chat/completions API and the Anthropic /v1/messages API.
Tag: llm
8 entries
2025-08-12
Today, I published a package that exposes an OpenAI-compatible API on top of Simon's llm tool. It's been interesting to develop a better feel for the least common denominator of features and what it looks like to try and support them well.
Using `llm` to handle large input context
Today, I ran into an issue where I wanted to use repomix to pack a large codebase into a single file to pass to an LLM, but I couldn't paste the output into any of the UIs I typically use. The React apps all became sluggish as I waited for ~500,000 tokens to paste.
`llm` upgrade pip
The llm package uses a plugin architecture to support numerous different language model API providers and frameworks. Per the documentation, these plugins are installed using a version of pip, the popular Python package manager Use the llm install command (a thin wrapper around pip install) to...
Calling Deepseek with `llm` using OpenAI Compatible APIs
Deepseek V3 was recently released: a cheap, reliable, supposedly GPT-4 class model.
2024-12-01
I'm working on a conversation branching tool called "Delta" (for now). The first thing that led me to this idea came from chatting with Llama 3.2 and experimenting with different system prompts. I was actually trying to build myself a local version of an app I've been fascinated by called Dot.
Using Nix to Install `llm` with Plugins
I use Simon's llm to quickly run LLM prompts. This package is easily installed with brew or pip, so if you want to use it, I recommend those approaches. The following approach is not for the faint of heart and assumes a bit of familiarity with Nix and home-manager. We are going to install the llm...
2024-03-04
I was using an LLM to assist when playing with Alacritty and tmux configs. The LLM (gpt-4-turbo) gave some amusing advice Embrace the Limitation In some cases, the minor visual imperfections may be something to adjust to. Given tmux's text-based nature and dependency on terminal emulator behavior,...