The field of artificial intelligence (AI) strives to build rational agents, capable of perceiving the world around them and taking actions to advance specified goals. Put another way, AI researchers aim to construct a synthetic homo economicus, the mythical perfectly rational agent of neoclassical economics.
We review progress towards creating this new species of machine, machina economicus, and discuss some challenges in designing AIs that can reason effectively in economic contexts. Supposing that AI succeeds in this quest, or at least comes close enough that it is useful to think about AIs in rationalistic terms, we ask how to design the rules of interaction in multi-agent systems that come to represent an economy of AIs. Theories of normative design from economics may prove more relevant for artificial agents than human agents, with AIs that better respect idealized assumptions of rationality than people, interacting through novel rules and incentive systems quite distinct from those tailored for people.
Indeed, economics often assumes rational economic reasoning (e.g. maximize some utility function under some income constraints) as a good first approximation of human behavior and AI agents could be the closest to these idealized rational agents. In AI however, the agents are taking inputs and produce outputs without optimizing a utility function. They do minimize a loss function when they try to fit a machine learning model but once in production they mechanically use trained models to produce outcomes that the agent designer (a human) has ordered. The utility function of the agent inherited from the designer could be as irrational as the utility function of a human being. For instance, AI agents could easily reproduce the tragedy of the commons if the designer only optimizes the individual AI agent strategy and ignores the negative externalities.