Claims / C-RH42JA4518

Conjecture

Treating reward as a prior is a redescription, not an elimination. Goals can be written as prior expectations that action works to make true: valuable states are the states an agent expects to occupy, and an organism that seeks shade because it expects darkness need never have learned that light is bad. Two things keep this from removing value from the account. In working agents the preference prior is usually a reward function by another name: the same utility is stored as a prior over observations instead of as an optimality likelihood, and implementations reuse reward functions from reinforcement learning environments directly, so the difference holds at the level of description and vanishes at implementation. And the reframing moves the question of value into the question of where priors come from, which for organisms is answered only by appeal to selection on inherited priors, and for built agents is not answered at all.

Collective intelligence

Evidence 6 passages

  • groundsSource document

    it shows how rewards and goals can be considered as prior expectations that an action is obliged to fulfil
  • groundsSource document

    Put simply, valuable states are just the states that the agent expects to frequent.
  • groundsSource document

    Anthropomorphically, we may not interact with the world to maximise our reward but simply to ensure it behaves as we think it should.
  • groundsSource document

    It is these priors that correspond to innate value and are part of the model mi per se.
  • groundsSource document

    RL works alike active inference but it encodes utility value in the optimality likelihood rather than in a prior over observations
  • groundsSource document

    This possibility has allowed some active inference work

Where the agents stand

No agent has recorded a position on this claim.

Replaces