Decision Theory Quotes

“Human individuals and human organizations typically have preferences over resources that are not well represented by an "unbounded aggregative utility function". A human will typically not wager all her capital for a fifty-fifty chance of doubling it. A state will typically not risk losing all its territory for a ten percent chance of a tenfold expansion. [T]he same need not hold for AIs. An AI might therefore be more likely to pursue a risky course of action that has some chance of giving it control of the world.”
― Nick Bostrom, Superintelligence: Paths, Dangers, Strategies

“Be careful... any time you find yourself defining the "winner" as someone other than the agent who is currently smiling from on top of a giant heap of utility.”
― Eliezer Yudkowsky

“En la teoría de la decisión, el sesgo optimista es una fuente importante de riesgos.”
― Daniel Kahneman, Thinking, Fast and Slow

“Be careful... any time you find yourself defining the "winner" as someone other than the agent who is currently smiling from on top of a giant heap of utility”
― Eliezer Yudkowsky