Given how murky the internal activities of AI labs are and how new the AI industry is, it is unclear whether or to what extent AI labs are currently engaging in RKA. However, the labs have a strong incentive to do so. New data, better models, and new markets will be critical to maintain the stratospheric growth that the market valuations of labs imply. Recent macroeconomic work on AI bubble dynamics helps formalize this pressure: High valuations can be sustained only with high growth and eventually high profits. Given that AI is a “general purpose technology,” there has already been a natural incentive for AI providers to integrate their models into a wide range of industries. The next step is for those providers to expand their market vertically to downstream uses, owning those uses, not merely servicing them.

The industries most vulnerable to RKA are those in which information shared with AI providers directly implicates the industries’ products or services—for instance, software or legal. In such industries, AI usage data from customers could most plausibly be employed to develop competing products or services. For example, employees of an enterprise software company might use AI models to write code or analyze data, disclosing critical information about enterprise software to AI labs. Similarly, in the legal industry, lawyers might employ AI models to execute legal workflows, revealing proprietary legal strategies. And in the financial services industry, AI models might reveal sensitive information about investment strategies. Moreover, if many employees within the same enterprise are using the same AI models simultaneously, the available information, once linked and synthesized, could reveal much more about the enterprise than any single employee has access to.

To make matters worse, there is no reason to think that the laptop-text-based interface for AI will remain predominant. AI startups are already building ambient note-taking devices, integrating agents directly into workflows, and incorporating visual AI into augmented reality products. Further in the future, embedded AI will escalate this effect—for example, deployed industrial robots can be used as data collection devices, extending the threat of RKA to industries like manufacturing, where incumbents may license robots only to have their industrial information appropriated by them. As the possible use-cases for AI expand, so too will the set of information available for acquisition and, thus, so too will the possible targets of vampiric inference. In other words, there may one day be no industry safe from the threat of RKA.

I don't agree with the almost implicit assumption that AI works well and can actually replace a lot of industries and workers... but the broader point of this article is definitely worth taking seriously, the clear incentive structure here is for AI companies to try to escape their collapsing business model by attempting to steal the business model of other companies by vacuuming up all of their data in a business to business contract for AI and then turning it against them.