Building on ATProto

If you prefer to receive all of my writing in your inbox, you can subscribe here. I strive to publish weekly, on Tuesday mornings.

Bluesky’s Atmosphere Protocol has the chance to become the base for a whole generation of applications. But it’s developing in surprising and disappointing ways.

I spent part of last week at the Local First Conference in Berlin. I was not surprised to find that AI coding was on everyone’s minds. Less expected was how pervasive ATProto was.

I’ve been considering trying to build on ATProto for a few months, so this was a great chance to ask a bunch of questions from both people building on it, and the people actually building it (multiple key Bluesky people were there).

I want ATProto to be the answer for me. I want to again live in a world where application developers use public, community driven standards to create interoperable applications with common data standards. I think it can be. But unfortunately I think it’s not on track to be that right now.

In this post I’m going to describe what I have been wanting to build, and how ATProto’s current and proposed designs do and don’t work with that. And I’ll also throw in some comments about what’s been surprising to me.

I’m probably trying to do too much here. But my main goal is to provide feedback on the proposed design, and it’s hard to do so without being clear about what I want to build and why.

The smallest version of what I want to build is a suite of applications for recording reviews. In an ideal world, these applications would supplant Yelp, GoodReads, Letterboxd, and a bunch of similar applications for just about anything you might consider reviewing. (Yes, I know there are already people building versions of these on ATProto; I’d want to collaborate with them.)

I want to replace these applications not because of their function, but because of their business models. Neither my wife nor I will use them the way they’re intended, and I think most people are like one or the other of us.

For me, I don’t want these companies to own my data. We have more than a thousand bookmarks in Yelp, but… they’re stuck in Yelp. I can’t do anything else useful with them: I can’t publish them, I can’t share them, I can’t write scripts across them, I can’t version control them. They belong to Yelp, not me. And, frankly, the experience they provide for managing lists of bookmarks kinda sucks.

This is a classic local first use case. I could adopt one tool that was great for writing reviews, another that used the same data but was better at publishing best-of lists for my friends, and another that made it easy to share “best business books” lists on my web site. No single app is going to do all of these, but a shared, public data model allows everyone to have what they want.

I’ve lost the willingness to give big corporations a permanent right to control how I use my own data. (That’s why I’m writing this in plain text and publishing via Hugo .) The apps only get worse over time. And if I’m going to invest that much time in recording my opinions, I want to control its use, not let Yelp do it.

As to my wife, she will just never publish anything online. Never. If I want her to review books or restaurants or recipes, it has to be in a place where I – and maybe the rest of our family – will see it, but no one else.

Personally, I am willing to publish public reviews. And sometimes I specifically want to, like sharing books I found useful as a founder. But doing so is complicated. The information I would record about a restaurant for myself is quite different from what I would want the wider world to know. Yet… most apps don’t allow you to draw this distinction. Reviews are public. Your opinion affects the subject’s reputation, even if you didn’t want it to. (I’m vegetarian and autistic, so a place working for me is not necessarily relevant for other people.)

Fundamentally, most of these applications are built to enable influencers : People who want to build a following online for their reviews. “Oh man, she has the best book recommendations!” “If he likes the restaurant, I know I will too!”

But most of us don’t want to become influencers. If anything, we’ve generally learned the lesson that it sucks to be followed by a bunch of people online. I want to keep my stuff private. I don’t want my opinions to matter to anyone else.

I mean sure, when someone comes to town I want to be able to give them a list of restaurants they should visit and activities they should do. But I want to send it to them , not the rest of the world.

We can split the world into three kinds of people:

The current crop of applications works fine for the first group. But it leaves the latter two out in the cold.

And I bet anything that those other two groups are much larger.

So, what we need is a system that allows application developers to easily let their users choose how public or private they want to be – entirely public, entirely private, or sharing piecemeal with individual groups.

Let’s start with what’s great about ATProto: It seems to be the first protocol designed to solve identity at scale, enabling any application to build ATProto identities into their application for authentication, tracking followers and following, and the other related features that basically every application needs.

Its identity system not perfect (I wish it were more human readable, especially). But it seems to be close enough today that nearly every speaker at Local First this year mentioned it. No more does every app developer have to create their own social graph, their own identity and authentication systems, their own means of finding friends.

Unfortunately, at least for now, the rest of the protocol is of limited use for my goals.

Today, ATProto is public-only: It assumes that everything you do will be published online for the whole world to see. That design decision is baked into everything from the storage systems to the structure of the publishing services.

The community is currently designing what they call “ permissioned data ”. (I think this is a silly name, and it should just be called “private data”, which is better both because “private” is actually a word, and also because it describes user behavior instead of technical implementation, which is a far better naming practice).

Unfortunately, while most of the assumptions driving the design seem reasonable, the resulting design looks very hard to build on. A lot of where I think it goes wrong is captured perfectly in the above linked post:

Before we get into it, the through-line on all of this is that public broadcast data is substantially different from permissioned data.

I fundamentally disagree with this. ( I am not the only one .)

Private and public data are basically identical. A restaurant review is a restaurant review whether I am the only one who ever sees it or it gets shouted to the rooftops. A book review I share with my book club is identical to a book review I share with my wife or publish on my web site.

Data I publish to the world is just a special case: The permission is world-read. Just like data I never publish is a special case: It has no readers other than me.

Unfortunately ATProto is already used out in the wild for Bluesky, and it is designed and built as a public-only protocol. It would be hard to change it to support access control, limited distribution, and the other features you need with private data.

So, instead, the community has, from what I can tell, just… designed a completely independent system for private data.

Permissioned Data relies on the existing identity system and lexicons (for defining data types). But adds entirely new data structures, plus new methods for managing and validating that data. (Note that the link above is to one in a series of posts on the design; you can get to all of them from there.)

So, as an app developer, you have to essentially write two applications: One for public data, and one for private. But, your users don’t think of it as two apps. “These are restaurant reviews. All the restaurant reviews should go together.” So your job, as the developer, is to support these two data systems, and two protocols, but never let the user see that they’re totally different.

That gets messy real fast. You’ve got a private post you want to make public? You’re not modifying it – you’re deleting the old one and making a new one in the public subsystem. Does it keep its likes? Its reposts? Links to it? 🤷‍♂️ No idea, but… probably not.

It’s actually worse than that. ATProto’s storage subsystem, the “ Personal Data Server ”, provides direct access to your data. In other words, you should expect more than one application to want to read and write your data. So now, anyone who is interested in this data has to write two versions, but lie to their users and hide any differences.

Because, again, there aren’t actual differences. The actual data being stored is the same. The way the user thinks about it is the same.

To me this is a sign that the current proposal is flawed.

I don’t know how to fix it. I’m nowhere near close enough to conversation to think I can propose solutions yet. But hopefully I can help people see problems, at least.

I only had a high-level understanding of ATProto went I went to Local First, so I learned a lot while I was there. One thing I was definitely wrong about was how the PDS worked.

I naively assumed it worked a lot like a git repository. The ATProto community regularly says that I own my own data. I tend to assume that means that I have a copy of it, and that when I make changes, I am operating against my copy then distributing it. This is exactly how git works: I clone a repository, make my changes, then push them back up to the server for distribution. I have a hard time imagining “owning” my data without always having a copy of it.

But no, that’s not at all how ATProto works. The PDS is “your” server, but… it’s actually a server. You do not push data up and pull it down; you speak a protocol to it. You only “own” your data because it and the protocols for accessing it are public, and you can change where it is stored.

It does use cryptography-based data integrity guarantees like git does, which is one more reason I assumed I could easily manage the data like git does. But those guarantees are only used on the server, not in the protocols you use to provide new data.

ATProto meets about half of the design principles in the Local First essay linked above. But it clearly misses on the data being at your fingertips or the network being optional, and also misses out on a lot of implied abilities because you’re always talking to a remote server.

If you want to record a review while you’re offline… best practice is to essentially store those reviews in a temporary area, then push them to the server when you’re back online. In other words, you have to build your own custom storage and sync system if you want offline support. Obviously not impossible, but also obviously not local-first.

If you then cross this with the current design for Permissioned Data, I’ve got a bit of a mess:

As an application developer committed to both local first principles and letting the user choose between public and private, ATProto is working at least as much against me as with me. Am I really better off building on this, versus designing my own system? That question is doubly scary when you recognize that the protocol designer and I are so far apart philosophically.

If I believe public and private data are fundamentally the same, just with different access rights, but the community believes they are and should be different… I am fighting the protocol, storage system, and community every step of the way. I’ve done this before, and it sucks.

I’m still excited about ATProto. And thankfully, the Permissioned Data design is early enough that it can still be improved. And I’m not