Sam Altman doesn’t understand the technology he’s building. In a recent New Yorker profile, neither he nor anyone else interviewed could quite explain how large language models produce what they produce, or what they’ll do next, or whether they pose an existential threat to the species. Altman holds this opacity up as a reason he should be the one in control of it. The move should be familiar to anyone who has watched an institution justify its authority by pointing to the complexity of what it manages and it is the latest, most aggressive instance of a much older pattern of governance, one that is also reshaping cultural institutions in ways the AI conversation may have missed.
In Seeing Like a State, the anthropologist James C. Scott describes how modern states make populations governable by making them legible: imposing surnames, fixed addresses, standardized measures, censuses, and cadastral maps until a messy social reality resolves into something a bureaucracy can act on. The defining feature of this legibility is its asymmetry. The state sees you; you do not see the state. Anyone who has tried to correct an error on a tax record, contest a charge with an insurer, or navigate an immigration office knows the texture of this asymmetry: the sense that you are intimately known by an entity you cannot reach.
David Graeber, in The Utopia of Rules, names what this asymmetry costs. He calls it interpretive labor: the work of understanding flows uphill from the weaker party to the stronger. Servants must read their masters; the master need not bother. Citizens must learn the state’s forms and categories; the state processes them as data. The labor of being understood is paid by whichever side can’t afford opacity. Bureaucracy, on Graeber’s account, is not really a system for managing complexity. It is a system for distributing the costs of being managed.
This is the framework worth bringing to the AI moment, because the AI moment is a legibility regime with new properties. Prompt engineering is the clearest example: we iterate endlessly to extract usable outputs from systems we are not permitted to see into, learning the system’s preferences without the system disclosing them. We trim our questions to fit token limits, reshape our intentions to navigate content policies, and contort our data into API schemas. Every interaction makes us slightly more legible to the systems behind the interface, while the systems remain (by design and by the admission of their builders) illegible to us. Altman is not unusual in claiming AI is too complex to be made transparent. He is simply naming, as a feature, what the entire industry treats as a structural commitment.
Scott’s later work, The Art of Not Being Governed, asks where people go when they want to live outside such regimes. He studies the highland peoples of Southeast Asia, a region he calls Zomia, and argues that their apparent backwardness is a deliberate political achievement. They chose crops that ripen underground and resist auditing, settlement patterns that resist mapping, fluid ethnic identities that resist classification. He calls some of them post-literate, meaning that they have set aside or refused writing systems because writing is a technology of capture. A literate population is a recordable population, and a recordable population is one that can be taxed, conscripted, and forced into labor.
The honest version of this story has to acknowledge what Scott himself eventually conceded: the uplands are mostly gone. The roads, helicopters, satellite maps, biometric IDs, and financial dragnets of the modern state have closed the escape valves. The Americans who live most ungovernably today, the undocumented, the unhoused, the deeply rural poor, did not choose the condition and pay severely for it. Vulnerability to violence, no medical infrastructure, foreclosed futures for their children. The territory in which to construct a life outside state and corporate operators is shrinking, and what little remains is not romantic.
What’s worth attending to is that this shrinkage was not sudden, and AI did not start it. The legibility regime has been advancing across sectors at different speeds for decades, and the sectors that have been under it longest can tell us what comes next for the ones just entering it. Cultural production is one of those sectors, and what has happened there is a preview.
Historically, the avant-garde lived in territory that markets and states could not see: bohemias, backrooms, immigrant neighborhoods, queer undergrounds, the cheap rents of cities the financial system had given up on. These were uplands in Scott’s sense, and they were generative precisely because they were illegible. The institutional compact that emerged in the postwar period gave museums and galleries a specific role in this ecosystem. The avant-garde happened in territory the institutions could not see; the institutions translated it into the public sphere; the public metabolized it into broader culture. The institutions’ job depended on the uplands existing.
That compact has broken down, and it broke down before AI arrived. The contemporary art market financialized through the 1990s and 2000s. Nonprofit cultural funding professionalized in parallel, with grants demanding measurable outcomes, audience metrics, demographic legibility, and theory-of-change diagrams. Social media added a third layer of measurement, where work that does not photograph well or summarize quickly is functionally invisible. The result is that museums and galleries now overwhelmingly present work that is already legible to financial systems, grant rubrics, and algorithmic discovery. The avant-garde function of bringing illegible cultural production into the public has been hollowed out from the inside, because the institutions stopped maintaining relationships with the territories that produced anything genuinely new.
This is the dead zone of imagination Graeber describes, applied to cultural life. People spend more energy managing their legibility to the systems that fund and platform them than they spend trying to understand each other. Curators write grant prose. Artists write artist statements optimized for committees. Museums present exhibitions that will brief well to a board. The labor of being understood has consumed the labor of understanding.
And this is the sector AI is now entering. The framing in the broader AI debate, that we are about to lose something that currently works, gets the cultural case backward. We are not about to lose it. We have already been losing it. AI is the accelerant, not the cause, and it is arriving in an institutional landscape that has lost the muscle memory of how to operate in illegible territory. Museums that respond to the AI moment by becoming more measurable, more discoverable, more grant-fundable, more marketable (that is, by accelerating their own legibility to the systems that have already broken the compact) are not adapting. They are completing a transition that has already cost them their reason to exist.
The public has registered this as museum attendance is largely dropping, which indicates that trust in museums as cultural arbiters is waning. Compelling work is increasingly happening outside the institutional frame, in places that the legibility apparatus has not yet found or cannot price. The attention is fracturing and going elsewhere. The institutions are watching this happen and reaching for more metrics and hoping that donors will bail them out.
What would it look like to do the opposite? Not for a museum to go off-grid itself, which is neither possible nor desirable, but to take seriously its function as a structure that protects illegibility on behalf of the public. That would mean building durable, long-horizon relationships with practitioners and communities whose work maybe doesn’t photograph well for Instagram, be summarized in a grant report, or scored by an algorithm. It would mean accepting that the avant-garde, if there is one, is by definition something the legibility systems can’t see, and that the museum’s job is to see it first and to translate it carefully, without flattening it into the very grids the public is trying to escape. It would mean tolerating risk that the funding apparatus has been designed to eliminate, and resisting the temptation to make every exhibition a demonstration of institutional legitimacy. It would mean, in Scott’s terms, being one of the few institutions left that can keep an upland alive.
This is not a small ask, and it is not only about museums. The cultural sector is a leading indicator, not a special case. The legibility regime is now arriving, through AI, in sectors that have so far been overlooked. The question of whether anything can be kept illegible inside the gathering grid, whether any institution can preserve the conditions for things that have not yet been seen, is one of the genuinely open questions of the next decade. Museums are one of the few institutions whose entire historical purpose was to do exactly that. Whether they remember how to is not just a question for the cultural sector. It is a test case for the rest.