2023 Review
End of year review
At the beginning of 2023, I set some goals for myself. Here are those goals and how the year turned out.
Posts
Embracing Failure in Software Development
Exploring the Importance of Embracing Failure in Software Development and the Value of Learning from Mistakes
To write software is to experience constant failure until you get a success. When you start learning to write code, very little works, especially on your first try. You make a lot of mistakes. Maybe you copied example code to get started, then modify it to try and do something new. Reading errors...
Promptfoo and standardizing output structure across models
promptfoo is a Javascript library and CLI for testing and evaluating LLM output quality. It's straightforward to install and get up and running quickly. As a first experiment, I've used it to compare the output of three similar prompts that specify their output structure using different modes of...
Using Marvin for Structured Data Extraction
I've been following the "AI engineering framework" marvin for several months now. In addition to openaifunctioncall, it's currently one of my favorite abstractions built on top of a language model. The docs are quite good, but as a quick demo, I've ported over a simplified version of an example...
OpenAI Function Calling
This past week, OpenAI added function calling to their SDK. This addition is exciting because it now incorporates schema as a first-class citizen in making calls to OpenAI chat models. As the example code and naming suggest, you can define a list of functions and schema of the parameters required...
Tradeoffs of Using a Cache at Scale
Imagine we have a query to an application that has become slow under load demands. We have several options to remedy this issue. If we settle on using a cache, consider the following failure domain when we design an architecture to determine whether using a cache actually is a good fit for the use...
Protobuf contracts with LLMs
I've written several posts on using JSON and Pydantic schemas to structure LLM responses. Recently, I've done some work using a similar approach with protobuf message schemas as the data contract. Here's an example to show what that looks like.
Input data schemas and token efficiency
Plenty of data is ambiguous without additional description or schema to clarify its meaning. It's easy to come up with structured data that can't easily be interpreted without its accompanying schema. Here's an example:
Figuring out how to use LLMs in production
Code needs structure output
The most popular language model use cases I've seen around have been chatbots agents chat your X use cases
Shaping LLM Responses
It's necessary to pay attention to the shape of a language model's response when incorporating it as a component in a software application. You can't programmatically tap into the power of a language model if you can't reliably parse its response. In the past, I have mostly used a combination of...