When chat is the wrong UI

We’re 176 years into this AI experiment.

Wait…it’s only been three years?

Are you sure?

Sometimes I remember things that happened last year, but it feels like it was so long ago. Did that really happen to me, or was it something that happened to my dad and he just told me about it when I was a kid?

Anyway… we’re three years into this AI experiment and the primary interaction surface we have with LLMs is still chat. I’m pretty sure it was @pmarca who first argued for a textarea component in the HTML spec. And I’m pretty sure of that fact because I asked AI about it. In a textarea.

But I would like to propose to you that perhaps, just maybe, chat is the wrong UI. Well, at least most of the time.

The Academic Steven Pinker said it this way…

It’s kind of a shame that the first large-scale implementation of AI was kind of a gimmick—a first-person chatbot. But there is tremendous promise for AI if it is task oriented.

Steven Pinker, academic

And if an academic says it and then I write it down as a quote in a blog post, then you know it’s true.

Chat is the primary way of working with AI because it was the first thing that people clicked with. And chat works really well as a universal solution simply because we don’t know what people are going to try to do with AI.

But as the user you know what you’re going to do with it, and at that point chat is often the wrong UI.

What you need, dear reader, is some sort of customizable UI that you can manifest out of thin air to work with the AI (or fight with—you do you) in the way that works for what you are trying to do in the moment.

There are lots of ways to pull this off, but in the GitHub Copilot app, this is called a canvas.

A canvas is a little full-stack application that runs inside of the GitHub Copilot app with no browser chrome. The agent can communicate with the server part of that app and the server can communicate back. So what you end up with is a surface that can do anything a normal computer program can do, plus can communicate bi-directionally with the GitHub Copilot agent.

That sounds very hand-wavy and I did use the word bi-directional which sounds like it’s straight off of a PowerPoint, so let’s look at this concept in practice and see if we can solve actual problems with the mighty canvas.

We’ll start with a simple example: using a canvas to create a Connect 4 game where you can play with the agent inside of the GitHub Copilot app.

Watch me absolutely DESTROY GPT-5.6 Sol on high reasoning…

OK. But I definitely did beat GPT-5.6 Luna with no reasoning so…LISTEN…CONNECT 4 IS A HARD GAME!!

Building a canvas is as simple as asking for it…

Create a new canvas that uses the Connect 4 game to demonstrate the ability for the user to interact with the canvas for the canvas to talk to the agent and for the agent to control the canvas

The GitHub Copilot app knows what a canvas is, so we don’t need to explain ourselves any more than that.

Now because these canvases are actually full stack apps and not just web pages, they can call third-party APIs, yes, but they can also execute code locally on your machine.

For example, here is a UI for Winget that can browse for packages on the registry, as well as manage my local packages, including installing and uninstalling.

There’s no AI here, but that’s the point.

When chat is the primary interface, it encourages you to use the agent to do everything. This is often just a pure waste of tokens. It’s almost always better to have the agent build a tool where all future interactions are free vs treating the agent itself as the tool. Stop asking GPT-5.6 Sol Max to “stage and commit”. (I know you do that. Because I’ve done it. Don’t token shame me, I have a fragile ego.)

Another great example would be instead of using the chat to ask the agent to do things with your SQLite database, how about you throw up a canvas and, you know, do it yourself.

I mean, you can even have intellisense here. Why not? Its 2026, and AI is a magical box that just does whatever you want.

Isn’t that pleasant?

It’s nice to write a little SQL every now and again. I said every now and again. Relax.

Or why write your Jekyll blog posts in plain Markdown when you can literally bring Windows Live Writer back from the dead.

Ok, these are all fun and mildly useful examples, but the value of custom UI gets a lot more clear when you use it to automate your development workflows.

I’m not going to tell you how to live your life, but my process for working with agents looks more or less like this…

• Research
• Prototype
• Plan
• Implement
• Iterate
• Finalize

It’s pretty simple, but each one of these steps requires me to be at the keyboard to interact, view prototypes, guide and move between steps.

But here’s the thing. I don’t actually need to be around for most of this process. The agent is capable of researching and generating prototypes and then letting me know when it’s ready for a review. The goal with agents is always to take yourself out of the loop as much as you can. That’s difficult though, because it’s not clear how to do that when all you have is a chat box.

Here’s a full example of how you can use a canvas to automate your own workflow, removing yourself from the loop as much or as little as you like.

I’m not saying you should copy this workflow or that this is the perfect example of how to work with agents. I mean, it might be. It probably is. Let’s ask AI…

OMG.

In all seriousness, I do think being boxed in by the chat UI may be working against us all right now. This is making it hard to figure out how to solve actual problems, because it’s not at all obvious what to do when your only method of interaction is a textarea.

Give canvases a try today. Some things like the SQLite canvas you can one shot. The workflow one took the better part of a day to get the design and automation right.

But I think you’ll find that you can go much further with AI when you think…wait for it…outside the chat box.

Written by

Technologist, author, speaker, and creator. Working at GitHub where I focus on AI-powered development.

Related posts