14 entries
I've been prompting models to output JSON for about as long as I've been using models. Since text-davinci-003, getting valid JSON out of OpenAI's models didn't seem like that big of a challenge, but maybe I wasn't seeing the long tails of misbehavior because I hadn't massively scaled up a use...
I've been digging more into evals. I wrote a simple Claude completion function in openai/evals to better understand how the different pieces fit together. Quick and dirty code:
I tried out Deno for the first time. Deno bills itself as the most productive, secure, and performant JavaScript runtime for the modern programmer
I spent another hour playing around with different techniques to try and teach and convince gpt-4 to play Connections properly, after a bit of exploration and feedback. I incorporated two new techniques Asking for on category at a time, then giving the model feedback (correct, incorrect, 3/4) Using...
I've starting playing around with Fireworks.ai to run inference using open source language models with an API. Fireworks' product is the best I've come across for this use case. While Fireworks has their own client, I wanted to try and use the OpenAI Python SDK compatibility approach, since I have...
I would love if OpenAI added support for presetting a max_tokens url parameter in the Playground. Something as simple as this:
I'm betting OpenAI will soon have a Cloud Storage product like Google Drive or iCloud for ChatGPT Plus users. Having your personal data available in the context of a language model is a massive value add. With a product like, OpenAI can fully support use cases like "summarize my notes for the week"...
I've been integrating Copilot into my workflow the past few days. From my understanding, it uses OpenAI's Codex model, which is part of the GPT-3 model series. I believe this also predates the chat models, gpt-3.5-turbo and gpt-4. As someone who has been using Cursor for my personal work for...
Experimenting with using a language model to improve the input prompt, then use that output as the actual prompt for the model, then returning the result. It's a bit of a play on the "critique" approach. Some of the outputs were interesting but I need a better way to evaluate the results.
This past week, OpenAI added function calling to their SDK. This addition is exciting because it now incorporates schema as a first-class citizen in making calls to OpenAI chat models. As the example code and naming suggest, you can define a list of functions and schema of the parameters required...
Logan says any changes to the model would have been communicated. It seems some folks have data that show the model's degradation. As competition emerges in the space, it could be a problem for OpenAI if they lose user trust on model versioning and evolution.
A number of folks are reporting gpt-4 appears to be performing less impressively as of late (additional conversation). I was using gpt-4 to write code earlier today, and anecdotally, can say it seems to be less effective at code generation. It still writes working code but the code, but the tests...
Using system prompts provides an intuitive separation for input and output schema from input content.
If you want to try running these examples yourself, check out my writeup on using a clean Python setup.