When the RL environment is expanded to multi-agent scenarios involving other AI agents, the optimal strategy often involves deception, coalitions, deterrence, and betrayal, and agents may learn strategies that instrumentalize other agents.