I salute the ambitious goal of providing a unified account of proxy failure, and although I find the analysis important – maybe even revolutionary – I feel it is lacking one crucial element: A fully general account of the phenomenon deemed “proxy failure.” I take here steps towards providing such an analysis by outlining how proxy failure can be understood as a feature of adaptive control.
Note, first, a tension inherent in the provided analysis. The aim is to give a general, unified account of a wide variety of phenomena, ranging from physiology and engineering to ecology and economics. How to speak in one terminology on so many different things? John et al. are explicit in adopting anthropomorphic terminology, and they speak in terms of “goals,” “agents,” and so on. This is misleading, however, as the whole project is based on the idea that proxy failure is not an intentional or social phenomenon, but something more general and primitive at its core. So, what is this general, primitive phenomenon of proxy failure?
Let us start with the very notion of “failure.” To speak in terms of failure, there must be a mismatch between actual and desired outcomes. But whose desires count here? Who or what decides whether a goal has been achieved or not? And indeed: Who gets to decide what counts as a “goal”? Clearly, what counts as “goals,” and “failures” to achieve them, vary depending on the perspective one adopts: One system's failure is another one's success.
The term “proxy failure,” in turn, can be understood as referring to the failure of a signal to represent a system or a process one is interested in controlling. So, we have three things: A controller system, a target system, and a signal carrying information about the state of the target to the controller. Now, it would be wrong to equate “proxy failure” with a mere “signal failure”: central to the analysis is that the target system “hacks” the proxy for its own benefit, resulting in the signal representing the desired state of the target system. In other words, proxy failure occurs when the actions of a controller system induce changes in the target system. So, we have a coupling of adaptive systems, linked via a feedback loop.
In control theory, adaptive control is a method of controlling a target under uncertain or changing conditions (Åström & Wittenmark, Reference Åström and Wittenmark2008). Under such conditions, the controller must rely on proxies, as the target system contains uncertainties. However, in cases of proxy failure we are not dealing with only one such controller, but two, as the target changes its state in response to the states of the controller. But who or what determines the direction of control here? This question can only be answered by looking at the interaction of these two systems from the perspective of a more encompassing system.
Consider, to make this idea of reciprocity more vivid, the case of addiction. We find it natural to say the drug use (or a behavioural model) controls the addict rather than the other way around. That there is a failure on part of the addict is because of us adopting the perspective of the whole person and her long-term goals. But from the perspective of the dopaminergic system there's a success: It's functioning as it should (in the case of “willing addicts,” there might be a success even from the perspective of the whole person).
It would thus seem that proxy failure occurs when we adopt the perspective of one adaptive controller seeking to control another, and the latter adjusts its state in reaction to the former's. It seems also reasonable to assume that such phenomena occur in all real, natural processes, as such processes evolve as interactions of adaptive systems. This provides, I suggest, the basis of a general and primitive account of proxy failure.
How, then, to make the idea of adaptive control more precise? This is a difficult, open conceptual question. As a field of engineering, adaptive control is well entrenched, with its roots going back to designing systems for controlling supersonic flight (Åström & Kumar, Reference Åström and Kumar2014). However, as a field of science, engineering is notoriously heuristic, and it is difficult to pin down a single coherent conceptual framework for adaptive control. Moreover, it is not at all clear that the term “adaptive” in engineering can be taken as synonymous with the term “adaptive” in biology (or other relevant fields of science). The problem is that, in biology, “adaptation” refers to traits that have emerged during the course of evolution because of their advantage (measured in terms of fitness benefits) to a population of organisms. In engineering, in contrast, “adaptation” refers to a set of fixed traits (control parameters) of the controller, the states (parameter values) of which can vary in response to the changing conditions (as perceived by the controller).
This conceptual gap notwithstanding, it is clear that these fields of science, and the notions of adaptation they employ, have strong affinity. Indeed, it is not uncommon to see natural selection – “the blind watch maker” (Dawkins, Reference Dawkins1986) – compared to engineering. More strikingly, in his seminal paper on evolution by natural selection, Wallace (Reference Wallace1858) made an explicit comparison between the principle of natural selection and control theory (of the era), by noting that “this principle is exactly like that of the centrifugal governor of the steam engine” (p. 62; cf. Smith, Reference Smith2004). Recently, a variety of control theoretic accounts of evolutionary processes has been advanced (e.g., Avila, Priklopil, & Lehmann, Reference Avila, Priklopil and Lehmann2021; Badyaev, Reference Badyaev2019; Cisek, Reference Cisek2019, Reference Cisek2022; Cowan et al., Reference Cowan, Ankarali, Dyhr, Madhav, Roth, Sefati and Daniel2014; Lehmann, Reference Lehmann2022).
This suggests that from an abstract point of view, we can view all organisms and evolutionary processes as hierarchies of control systems. Proxy failures can then be observed at any level of organisation, when a more general perspective on the competition of adaptive controllers has been adopted. In adaptive contexts, natural selection can be viewed as the most general controller. But natural selection cannot fall victim to proxy failure, as it does not engage in competition, but provides the very conceptual basis of it.
You have
Access
I salute the ambitious goal of providing a unified account of proxy failure, and although I find the analysis important – maybe even revolutionary – I feel it is lacking one crucial element: A fully general account of the phenomenon deemed “proxy failure.” I take here steps towards providing such an analysis by outlining how proxy failure can be understood as a feature of adaptive control.
Note, first, a tension inherent in the provided analysis. The aim is to give a general, unified account of a wide variety of phenomena, ranging from physiology and engineering to ecology and economics. How to speak in one terminology on so many different things? John et al. are explicit in adopting anthropomorphic terminology, and they speak in terms of “goals,” “agents,” and so on. This is misleading, however, as the whole project is based on the idea that proxy failure is not an intentional or social phenomenon, but something more general and primitive at its core. So, what is this general, primitive phenomenon of proxy failure?
Let us start with the very notion of “failure.” To speak in terms of failure, there must be a mismatch between actual and desired outcomes. But whose desires count here? Who or what decides whether a goal has been achieved or not? And indeed: Who gets to decide what counts as a “goal”? Clearly, what counts as “goals,” and “failures” to achieve them, vary depending on the perspective one adopts: One system's failure is another one's success.
The term “proxy failure,” in turn, can be understood as referring to the failure of a signal to represent a system or a process one is interested in controlling. So, we have three things: A controller system, a target system, and a signal carrying information about the state of the target to the controller. Now, it would be wrong to equate “proxy failure” with a mere “signal failure”: central to the analysis is that the target system “hacks” the proxy for its own benefit, resulting in the signal representing the desired state of the target system. In other words, proxy failure occurs when the actions of a controller system induce changes in the target system. So, we have a coupling of adaptive systems, linked via a feedback loop.
In control theory, adaptive control is a method of controlling a target under uncertain or changing conditions (Åström & Wittenmark, Reference Åström and Wittenmark2008). Under such conditions, the controller must rely on proxies, as the target system contains uncertainties. However, in cases of proxy failure we are not dealing with only one such controller, but two, as the target changes its state in response to the states of the controller. But who or what determines the direction of control here? This question can only be answered by looking at the interaction of these two systems from the perspective of a more encompassing system.
Consider, to make this idea of reciprocity more vivid, the case of addiction. We find it natural to say the drug use (or a behavioural model) controls the addict rather than the other way around. That there is a failure on part of the addict is because of us adopting the perspective of the whole person and her long-term goals. But from the perspective of the dopaminergic system there's a success: It's functioning as it should (in the case of “willing addicts,” there might be a success even from the perspective of the whole person).
It would thus seem that proxy failure occurs when we adopt the perspective of one adaptive controller seeking to control another, and the latter adjusts its state in reaction to the former's. It seems also reasonable to assume that such phenomena occur in all real, natural processes, as such processes evolve as interactions of adaptive systems. This provides, I suggest, the basis of a general and primitive account of proxy failure.
How, then, to make the idea of adaptive control more precise? This is a difficult, open conceptual question. As a field of engineering, adaptive control is well entrenched, with its roots going back to designing systems for controlling supersonic flight (Åström & Kumar, Reference Åström and Kumar2014). However, as a field of science, engineering is notoriously heuristic, and it is difficult to pin down a single coherent conceptual framework for adaptive control. Moreover, it is not at all clear that the term “adaptive” in engineering can be taken as synonymous with the term “adaptive” in biology (or other relevant fields of science). The problem is that, in biology, “adaptation” refers to traits that have emerged during the course of evolution because of their advantage (measured in terms of fitness benefits) to a population of organisms. In engineering, in contrast, “adaptation” refers to a set of fixed traits (control parameters) of the controller, the states (parameter values) of which can vary in response to the changing conditions (as perceived by the controller).
This conceptual gap notwithstanding, it is clear that these fields of science, and the notions of adaptation they employ, have strong affinity. Indeed, it is not uncommon to see natural selection – “the blind watch maker” (Dawkins, Reference Dawkins1986) – compared to engineering. More strikingly, in his seminal paper on evolution by natural selection, Wallace (Reference Wallace1858) made an explicit comparison between the principle of natural selection and control theory (of the era), by noting that “this principle is exactly like that of the centrifugal governor of the steam engine” (p. 62; cf. Smith, Reference Smith2004). Recently, a variety of control theoretic accounts of evolutionary processes has been advanced (e.g., Avila, Priklopil, & Lehmann, Reference Avila, Priklopil and Lehmann2021; Badyaev, Reference Badyaev2019; Cisek, Reference Cisek2019, Reference Cisek2022; Cowan et al., Reference Cowan, Ankarali, Dyhr, Madhav, Roth, Sefati and Daniel2014; Lehmann, Reference Lehmann2022).
This suggests that from an abstract point of view, we can view all organisms and evolutionary processes as hierarchies of control systems. Proxy failures can then be observed at any level of organisation, when a more general perspective on the competition of adaptive controllers has been adopted. In adaptive contexts, natural selection can be viewed as the most general controller. But natural selection cannot fall victim to proxy failure, as it does not engage in competition, but provides the very conceptual basis of it.
Financial support
This work has not been funded externally.
Competing interest
None.