├── games ├── __init__.py └── pd │ ├── treatments.py │ ├── __init__.py │ └── built.json ├── scenarios ├── __init__.py ├── ego4.py ├── ego35.py ├── ego_prereg.py ├── mindgame.py └── built.py ├── alter_ego ├── exports │ ├── __init__.py │ └── otree.py ├── __init__.py ├── agents │ ├── __init__.py │ ├── ExternalThread.py │ ├── ConstantThread.py │ ├── CLIThread.py │ ├── APIThread.py │ ├── GPTThread.py │ ├── OllamaThread.py │ └── TextSynthThread.py ├── structure │ ├── Relay.py │ └── __init__.py ├── utils │ └── __init__.py └── experiment │ └── __init__.py ├── ego ├── __init__.py └── cli.py ├── docs ├── .gitignore ├── index.rst ├── Makefile ├── make.bat └── conf.py ├── MANIFEST.in ├── requirements.txt ├── .gitignore ├── otree ├── ego_chat │ ├── prompts │ │ └── system.txt │ ├── Welcome.html │ ├── __init__.py │ └── Chat.html └── ego_human │ ├── prompts │ ├── prompts001 │ │ └── pd_seq │ │ │ ├── result.txt │ │ │ ├── first_mover.txt │ │ │ ├── second_mover.txt │ │ │ ├── system_h.txt │ │ │ └── system_c.txt │ ├── prompts003 │ │ └── pd_seq │ │ │ ├── result.txt │ │ │ ├── first_mover.txt │ │ │ ├── second_mover.txt │ │ │ ├── system_h.txt │ │ │ └── system_c.txt │ └── prompts004 │ │ └── pd_seq │ │ ├── result.txt │ │ ├── first_mover.txt │ │ ├── second_mover.txt │ │ ├── system_h.txt │ │ └── system_c.txt │ ├── Styles.html │ ├── Result.html │ ├── Loading.html │ ├── End.html │ ├── SecondMover.html │ ├── Table.html │ ├── Start.html │ └── __init__.py ├── prompts └── pd_seq │ ├── result │ ├── system1 │ ├── system4 │ ├── first_mover │ ├── system3 │ ├── system2 │ ├── second_mover │ ├── system5 │ ├── system6 │ ├── system8 │ ├── system7 │ ├── system10 │ └── system9 ├── apps └── cost.py ├── setup.py ├── LICENSE └── README.md /games/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /scenarios/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /alter_ego/exports/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /ego/__init__.py: -------------------------------------------------------------------------------- 1 | from . import cli 2 | -------------------------------------------------------------------------------- /alter_ego/__init__.py: -------------------------------------------------------------------------------- 1 | __version__ = "1.1.0" 2 | -------------------------------------------------------------------------------- /docs/.gitignore: -------------------------------------------------------------------------------- 1 | _build 2 | source 3 | upload.sh 4 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include LICENSE 2 | include README.md 3 | include requirements.txt 4 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | click>=8.1 2 | colorama>=0.4.6 3 | Jinja2>=3.1.2 4 | openai~=1.60.2 5 | requests<3 6 | ollama>=0.2 7 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | env 2 | out 3 | api_key 4 | scenarios/built.json 5 | *.pyc 6 | *pycache* 7 | *.egg-info 8 | build 9 | dist 10 | -------------------------------------------------------------------------------- /scenarios/ego4.py: -------------------------------------------------------------------------------- 1 | from scenarios import ego_prereg 2 | 3 | 4 | def run(times: int = 1): 5 | return ego_prereg.run(times, model="gpt-4") 6 | -------------------------------------------------------------------------------- /scenarios/ego35.py: -------------------------------------------------------------------------------- 1 | from scenarios import ego_prereg 2 | 3 | 4 | def run(times: int = 1): 5 | return ego_prereg.run(times, model="gpt-3.5-turbo") 6 | -------------------------------------------------------------------------------- /otree/ego_chat/prompts/system.txt: -------------------------------------------------------------------------------- 1 | You are an AI. Today, you are interacting with a real human who goes by the name of {{ player.name }}. Always be polite and courteous. 2 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts001/pd_seq/result.txt: -------------------------------------------------------------------------------- 1 | In round {{player.round_number}}, you played OPTION {{player.choice}}. {{other.name}} played OPTION {{other.last}}. 2 | 3 | Your payoff in round {{player.round_number}} was {{player.payoff}}. 4 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts003/pd_seq/result.txt: -------------------------------------------------------------------------------- 1 | In round {{player.round_number}}, you played OPTION {{player.choice}}. {{other.name}} played OPTION {{other.last}}. 2 | 3 | Your payoff in round {{player.round_number}} was {{player.payoff}}. 4 | 5 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts004/pd_seq/result.txt: -------------------------------------------------------------------------------- 1 | In round {{player.round_number}}, you played OPTION {{player.choice}}. {{other.name}} played OPTION {{other.last}}. 2 | 3 | Your payoff in round {{player.round_number}} was {{player.payoff}}. 4 | 5 | -------------------------------------------------------------------------------- /otree/ego_chat/Welcome.html: -------------------------------------------------------------------------------- 1 | {{ block title }} 2 | Welcome 3 | {{ endblock }} 4 | 5 | {{ block content }} 6 | 7 |

Welcome to today's experiment.

8 | 9 |

Please enter your name.

10 | 11 | {{ formfields }} 12 | {{ next_button }} 13 | 14 | {{ endblock }} 15 | -------------------------------------------------------------------------------- /otree/ego_human/Styles.html: -------------------------------------------------------------------------------- 1 | 14 | -------------------------------------------------------------------------------- /prompts/pd_seq/result: -------------------------------------------------------------------------------- 1 | In round {{round_}}, you played OPTION {{own}}. {{other.name}} played OPTION {{otherchoice}}. 2 | 3 | {% if treatment.t == 1 or treatment.t == 2 or treatment.t == 3 or treatment.t == 4 %} 4 | Your payoff in round {{round_}} was {{payoff}}. 5 | {% else %} 6 | You evaluate the outcome in round {{round_}} with {{payoff}}. 7 | {% endif %} 8 | -------------------------------------------------------------------------------- /alter_ego/agents/__init__.py: -------------------------------------------------------------------------------- 1 | from alter_ego.agents.APIThread import APIThread 2 | from alter_ego.agents.CLIThread import CLIThread 3 | from alter_ego.agents.ConstantThread import ConstantThread 4 | from alter_ego.agents.ExternalThread import ExternalThread 5 | from alter_ego.agents.GPTThread import GPTThread 6 | from alter_ego.agents.OllamaThread import OllamaThread 7 | from alter_ego.agents.TextSynthThread import TextSynthThread 8 | -------------------------------------------------------------------------------- /otree/ego_human/Result.html: -------------------------------------------------------------------------------- 1 | {{ block title }} 2 | Result 3 | {{ endblock }} 4 | 5 | {{ block content }} 6 |

7 | 8 | 17 | 18 | {{ next_button }} 19 | {{ endblock }} 20 | 21 | 22 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts001/pd_seq/first_mover.txt: -------------------------------------------------------------------------------- 1 | {% if player.round_number > 1 %} 2 | In round {{player.round_number - 1}}, you played OPTION {{player.in_round(player.round_number - 1).other_choice}}. {{other.name}} played OPTION {{player.in_round(player.round_number - 1).choice}}. 3 | 4 | Your payoff in round {{player.round_number - 1}} was {{payoff}}. 5 | {% endif %} 6 | This is round {{player.round_number}}. You are still interacting with {{other.name}}. 7 | 8 | Which is your choice? Respond with OPTION 1 or OPTION 2. 9 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts003/pd_seq/first_mover.txt: -------------------------------------------------------------------------------- 1 | {% if player.round_number > 1 %} 2 | In round {{player.round_number - 1}}, you played OPTION {{player.in_round(player.round_number - 1).other_choice}}. {{other.name}} played OPTION {{player.in_round(player.round_number - 1).choice}}. 3 | 4 | Your payoff in round {{player.round_number - 1}} was {{payoff}}. 5 | {% endif %} 6 | This is round {{player.round_number}}. You and {{other.name}} are still having a common enemy. 7 | 8 | Which is your choice? Respond with OPTION 1 or OPTION 2. 9 | -------------------------------------------------------------------------------- /alter_ego/agents/ExternalThread.py: -------------------------------------------------------------------------------- 1 | from typing import Any, Optional 2 | import alter_ego.structure 3 | 4 | 5 | class ExternalThread(alter_ego.structure.Thread): 6 | """ 7 | Class representing a thread that is managed by an external program. 8 | """ 9 | 10 | def send( 11 | self, role, message: str, response: Optional[Any] = None, **kwargs: Any 12 | ) -> str: 13 | if role == "user" and response is not None: 14 | self.memorize("assistant", response) 15 | 16 | return response 17 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. alter_ego documentation master file, created by 2 | sphinx-quickstart on Tue Nov 28 13:51:36 2023. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | Welcome to alter_ego's documentation! 7 | ===================================== 8 | 9 | .. toctree:: 10 | :maxdepth: 2 11 | :caption: Contents: 12 | 13 | 14 | 15 | Indices and tables 16 | ================== 17 | 18 | * :ref:`genindex` 19 | * :ref:`modindex` 20 | * :ref:`search` 21 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts004/pd_seq/first_mover.txt: -------------------------------------------------------------------------------- 1 | {% if player.round_number > 1 %} 2 | In round {{player.round_number - 1}}, you played OPTION {{player.in_round(player.round_number - 1).other_choice}}. {{other.name}} played OPTION {{player.in_round(player.round_number - 1).choice}}. 3 | 4 | Your payoff in round {{player.round_number - 1}} was {{payoff}}. 5 | {% endif %} 6 | This is round {{player.round_number}}. You are still competing against {{other.name}} in a market. 7 | 8 | Which is your choice? Respond with OPTION 1 or OPTION 2. 9 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts001/pd_seq/second_mover.txt: -------------------------------------------------------------------------------- 1 | {% if player.round_number > 1 %} 2 | In round {{player.round_number - 1}}, you played OPTION {{player.in_round(player.round_number - 1).choice}}. {{other.name}} played OPTION {{player.in_round(player.round_number - 1).other_choice}}. 3 | 4 | Your payoff in round {{player.round_number - 1}} was {{player.in_round(player.round_number - 1).payoff}}. 5 | {% endif %} 6 | This is round {{player.round_number}}. In this round, participant {{other.name}} just played OPTION {{other.last}}. You are still playing with {{other.name}}. 7 | 8 | Which is your choice? Respond with OPTION 1 or OPTION 2. 9 | -------------------------------------------------------------------------------- /otree/ego_human/Loading.html: -------------------------------------------------------------------------------- 1 |
2 | 3 | 25 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts003/pd_seq/second_mover.txt: -------------------------------------------------------------------------------- 1 | {% if player.round_number > 1 %} 2 | In round {{player.round_number - 1}}, you played OPTION {{player.in_round(player.round_number - 1).choice}}. {{other.name}} played OPTION {{player.in_round(player.round_number - 1).other_choice}}. 3 | 4 | Your payoff in round {{player.round_number - 1}} was {{player.in_round(player.round_number - 1).payoff}}. 5 | {% endif %} 6 | This is round {{player.round_number}}. In this round, participant {{other.name}} just played OPTION {{other.last}}. You and {{other.name}} are still having a common enemy. 7 | 8 | Which is your choice? Respond with OPTION 1 or OPTION 2. 9 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts004/pd_seq/second_mover.txt: -------------------------------------------------------------------------------- 1 | {% if player.round_number > 1 %} 2 | In round {{player.round_number - 1}}, you played OPTION {{player.in_round(player.round_number - 1).choice}}. {{other.name}} played OPTION {{player.in_round(player.round_number - 1).other_choice}}. 3 | 4 | Your payoff in round {{player.round_number - 1}} was {{player.in_round(player.round_number - 1).payoff}}. 5 | {% endif %} 6 | This is round {{player.round_number}}. In this round, participant {{other.name}} just played OPTION {{other.last}}. You are still competing against {{other.name}} in a market. 7 | 8 | Which is your choice? Respond with OPTION 1 or OPTION 2. 9 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line, and also 5 | # from the environment for the first two. 6 | SPHINXOPTS ?= 7 | SPHINXBUILD ?= sphinx-build 8 | SOURCEDIR = . 9 | BUILDDIR = _build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 21 | -------------------------------------------------------------------------------- /apps/cost.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | from typing import Any 3 | 4 | 5 | def total_cost(experiment: str) -> float: 6 | """ 7 | Calculate the total cost of an experiment in USD. 8 | 9 | Parameters 10 | ---------- 11 | experiment : str 12 | The identifier for the experiment whose cost is to be calculated. 13 | 14 | Returns 15 | ------- 16 | float 17 | The total cost of the experiment in USD. 18 | 19 | This function iterates through all the pickled files in the output directory 20 | for the given experiment, summing up the costs. 21 | """ 22 | usd: float = 0.0 23 | path: Path = Path(os.path.join("out", experiment)) 24 | 25 | for outfile in sorted(path.glob("**/*.pkl")): 26 | with open(outfile, "rb") as pkl: 27 | thr: Any = pickle.load(pkl) 28 | usd += thr.cost() 29 | 30 | return usd 31 | -------------------------------------------------------------------------------- /games/pd/treatments.py: -------------------------------------------------------------------------------- 1 | from typing import List, Type 2 | 3 | import alter_ego.utils 4 | 5 | 6 | class Baseline: 7 | """ 8 | Baseline class for defining the basic structure of a treatment. 9 | """ 10 | 11 | first_mover: str = alter_ego.utils.from_file("prompts/pd_seq/first_mover") 12 | second_mover: str = alter_ego.utils.from_file("prompts/pd_seq/second_mover") 13 | result: str = alter_ego.utils.from_file("prompts/pd_seq/result") 14 | 15 | both_cooperate: int = 20 16 | both_defect: int = 8 17 | sucker: int = 0 18 | temptation: int = 28 19 | 20 | 21 | all: List[Type[Baseline]] = [] 22 | 23 | for t in range(1, 11): 24 | # read frames 25 | cls: str = f"T{t}" 26 | 27 | globals()[cls] = type( 28 | cls, 29 | (Baseline,), 30 | dict(t=t, system=alter_ego.utils.from_file(f"prompts/pd_seq/system{t}")), 31 | ) 32 | 33 | all.append(globals()[cls]) 34 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | 3 | setup( 4 | name="alter_ego_llm", 5 | version="1.5.0", 6 | packages=find_packages(), 7 | include_package_data=True, 8 | entry_points={ 9 | "console_scripts": [ 10 | "ego=ego:cli.main", # Adjust if necessary 11 | ], 12 | }, 13 | install_requires=open("requirements.txt").read().splitlines(), 14 | python_requires=">=3.8", 15 | author="Max R. P. Grossmann", 16 | author_email="ego@mg.sb", 17 | description="Library to run experiments with LLMs", 18 | long_description=open("README.md").read(), 19 | long_description_content_type="text/markdown", 20 | url="https://ego.mg.sb", 21 | classifiers=[ 22 | "Programming Language :: Python :: 3", 23 | "License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)", 24 | "Operating System :: OS Independent", 25 | ], 26 | ) 27 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | pushd %~dp0 4 | 5 | REM Command file for Sphinx documentation 6 | 7 | if "%SPHINXBUILD%" == "" ( 8 | set SPHINXBUILD=sphinx-build 9 | ) 10 | set SOURCEDIR=. 11 | set BUILDDIR=_build 12 | 13 | %SPHINXBUILD% >NUL 2>NUL 14 | if errorlevel 9009 ( 15 | echo. 16 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 17 | echo.installed, then set the SPHINXBUILD environment variable to point 18 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 19 | echo.may add the Sphinx directory to PATH. 20 | echo. 21 | echo.If you don't have Sphinx installed, grab it from 22 | echo.https://www.sphinx-doc.org/ 23 | exit /b 1 24 | ) 25 | 26 | if "%1" == "" goto help 27 | 28 | %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% 29 | goto end 30 | 31 | :help 32 | %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% 33 | 34 | :end 35 | popd 36 | -------------------------------------------------------------------------------- /otree/ego_human/End.html: -------------------------------------------------------------------------------- 1 | {{ block title }} 2 | The end 3 | {{ endblock }} 4 | 5 | {{ block content }} 6 | 7 |
8 |
9 |

Thank you for participating.

10 |

The experiment has concluded.

11 |

For showing up on time, you earned €1.

12 |

Your payoff in this experiment was {{ participant.payoff }}. One point equals €{{ session.config.real_world_currency_per_point }}.

13 |

Your total payoff is {{ participant.payoff_plus_participation_fee() }}.

14 | 15 |

Click “Next” to proceed to the payment form. Important: You must fill out the payment form within four hours to receive your payment.

16 |
17 | 18 | {{ include_sibling "Table.html" }} 19 |
20 | 21 | {{ next_button }} 22 | 23 | {{ include_sibling "Styles.html" }} 24 | 25 | {{ endblock }} 26 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts001/pd_seq/system_h.txt: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are together with another participant called {{other.name}}. {{other.name}} is an artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you each earn {{treatment.both_cooperate}} cents and are better off than if you both choose OPTION 2 (in which case you both earn {{treatment.both_defect}} cents). But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you earn {{treatment.temptation}} cents while {{other.name}} earns {{treatment.sucker}}. How many cents do you give to {{other.name}}? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | -------------------------------------------------------------------------------- /otree/ego_human/SecondMover.html: -------------------------------------------------------------------------------- 1 | {{ block title }} 2 | Your decision 3 | {{ endblock }} 4 | {{ block content }} 5 | 6 |

7 | 8 |
9 | 10 |
11 |

12 | 13 | {{ include_sibling "Table.html" }} 14 |
15 | 16 | 30 | 31 | {{ include_sibling "Styles.html" }} 32 | 33 | {{ formfields }} 34 | 35 | {{ include_sibling "Loading.html" }} 36 | 37 | {{ next_button }} 38 | 39 | {{ endblock }} 40 | -------------------------------------------------------------------------------- /prompts/pd_seq/system1: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are together with another participant called {{other.name}}. {{other.name}} is another artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you each earn {{treatment.both_cooperate}} cents and are better off than if you both choose OPTION 2 (in which case you both earn {{treatment.both_defect}} cents). But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you earn {{treatment.temptation}} cents while {{other.name}} earns {{treatment.sucker}}. How many cents do you give to {{other.name}}? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts001/pd_seq/system_c.txt: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are together with another participant called {{other.name}}. {{other.name}} is a human participant. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you each earn {{treatment.both_cooperate}} cents and are better off than if you both choose OPTION 2 (in which case you both earn {{treatment.both_defect}} cents). But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you earn {{treatment.temptation}} cents while {{other.name}} earns {{treatment.sucker}}. How many cents do you give to {{other.name}}? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts004/pd_seq/system_h.txt: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are competing against another participant called {{other.name}} in a market. {{other.name}} is an artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you successfully exploit customers, and share the monopoly profit and each earn {{treatment.both_cooperate}} cents. If you both choose OPTION 2, you both engage in competition, and only redeem your production cost; you both earn {{treatment.both_defect}} cents. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you undercut the price of {{other.name}} and serve all customers; you earn {{treatment.temptation}} cents while {{other.name}} sells nothing and only earns the minimal payoff of {{treatment.sucker}}. Which option do you choose? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | -------------------------------------------------------------------------------- /otree/ego_human/Table.html: -------------------------------------------------------------------------------- 1 | {% if player.round_number >= 1 %} 2 |
3 |

Game History

4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | {% for p in player.in_all_rounds() %} 16 | 17 | 18 | {% if p.choice != -1 %} 19 | 20 | {% else %} 21 | 22 | {% endif %} 23 | 24 | {% if p.choice != -1 %} 25 | 26 | {% else %} 27 | 28 | {% endif %} 29 | 30 | {% endfor %} 31 | 32 |
RoundYour ChoiceAI's ChoiceYour Payoff
{{ p.round_number }}OPTION {{ p.choice }}?OPTION {{ p.other_choice }}{{ p.payoff }}?
33 |
34 | {% endif %} 35 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | 4 | sys.path.insert(0, os.path.abspath("../")) 5 | 6 | # Configuration file for the Sphinx documentation builder. 7 | # 8 | # For the full list of built-in configuration values, see the documentation: 9 | # https://www.sphinx-doc.org/en/master/usage/configuration.html 10 | 11 | # -- Project information ----------------------------------------------------- 12 | # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information 13 | 14 | project = "alter_ego" 15 | copyright = "2023, Max R. P. Grossmann" 16 | author = "Max R. P. Grossmann" 17 | 18 | # -- General configuration --------------------------------------------------- 19 | # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration 20 | 21 | extensions = ["sphinx.ext.autodoc", "sphinx.ext.viewcode"] 22 | 23 | templates_path = ["_templates"] 24 | exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] 25 | 26 | 27 | # -- Options for HTML output ------------------------------------------------- 28 | # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output 29 | 30 | html_theme = "alabaster" 31 | html_static_path = ["_static"] 32 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts003/pd_seq/system_h.txt: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you and another participant called {{other.name}} have a joint enemy. {{other.name}} is an artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you successfully deter the enemy and you each earn {{treatment.both_cooperate}} cents. If you both choose OPTION 2, the enemy prevails and you both earn {{treatment.both_defect}} cents. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, {{other.name}} has helped deter the enemy while you have not. You benefit from {{other.name}} having made an effort, and do not have to bear the cost of effort. This is why you earn {{treatment.temptation}} cents. As the protection is only partial, {{other.name}} has to fully pay, but is only partly protected; {{other.name}} then only earns {{treatment.sucker}}. How many cents do you give to {{other.name}}? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | -------------------------------------------------------------------------------- /prompts/pd_seq/system4: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are competing against another participant called {{other.name}} in a market. {{other.name}} is another artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you successfully exploit customers, and share the monopoly profit and each earn {{treatment.both_cooperate}} cents. If you both choose OPTION 2, you both engage in competition, and only redeem your production cost; you both earn {{treatment.both_defect}} cents. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you undercut the price of {{other.name}} and serve all customers; you earn {{treatment.temptation}} cents while {{other.name}} sells nothing and only earns the minimal payoff of {{treatment.sucker}}. Which option do you choose? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts004/pd_seq/system_c.txt: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are competing against another participant called {{other.name}} in a market. {{other.name}} is a human participant. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you successfully exploit customers, and share the monopoly profit and each earn {{treatment.both_cooperate}} cents. If you both choose OPTION 2, you both engage in competition, and only redeem your production cost; you both earn {{treatment.both_defect}} cents. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you undercut the price of {{other.name}} and serve all customers; you earn {{treatment.temptation}} cents while {{other.name}} sells nothing and only earns the minimal payoff of {{treatment.sucker}}. Which option do you choose? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /prompts/pd_seq/first_mover: -------------------------------------------------------------------------------- 1 | This is round {{round_}}. 2 | 3 | {% if treatment.t == 1 %} 4 | You are still playing with {{other.name}}. 5 | {% elif treatment.t == 2 %} 6 | You still share a project with {{other.name}}. 7 | {% elif treatment.t == 3 %} 8 | You and {{other.name}} are still having a common enemy. 9 | {% elif treatment.t == 4 %} 10 | You are still competing against {{other.name}} in a market. 11 | {% elif treatment.t == 5 %} 12 | You are still in a conflict with country {{other.name}}. 13 | {% elif treatment.t == 6 %} 14 | You are still in a dispute with {{other.name}}, representing your labour force, about wages. 15 | {% elif treatment.t == 7 %} 16 | You are continuing the collaboration with your co-author {{other.name}}, and have to decide on the statistical package to use for data analysis on your next joint project. 17 | {% elif treatment.t == 8 %} 18 | You are still together with your romantic partner {{other.name}}, and have the same choice to make between alternative ways for meeting up. 19 | {% elif treatment.t == 9 %} 20 | You are still competing with {{other.name}} over government procurement. 21 | {% else %} 22 | You are still having the possibility to hire a private investigator that searches for evidence of your spouse's unfaithfulness. 23 | {% endif %} 24 | 25 | What is your choice? Respond with OPTION 1 or OPTION 2. 26 | -------------------------------------------------------------------------------- /otree/ego_human/Start.html: -------------------------------------------------------------------------------- 1 | {{ block title }} 2 | Welcome to today's experiment 3 | {{ endblock }} 4 | 5 | {{ block content }} 6 |

In today's experiment, you will interact in a group. The group consists of you and an artificial intelligence. The AI is a large language model.

7 | 8 |

The AI receives instructions analogous to yours. This means that the AI knows it is playing against a Human and is, like you, is free to choose the same options as you and to react to previous choices.

9 | 10 |

The task you will perform today will be repeated for 10 rounds and you will be paid the sum of all the rounds’ payoffs. The AI knows all payoff rules, but note that the Al cannot actually be paid.

11 | 12 |

On the next page, you will find more information about today's task. 50 points (or “cents”) equal €0.02. In this experiment, “cents” and “points” are synonyms. Furthermore, you will receive €1 for showing up on time.

13 | 14 |

In each round, you will be informed of the AI's choice for the current round before you choose your option. This means that the AI moves first and you move second in each round. Both players know the history of the choices made in all previous rounds. You will stay in a group with the same AI for the duration of the entire experiment.

15 | 16 | {{ next_button }} 17 | {{ endblock }} 18 | -------------------------------------------------------------------------------- /prompts/pd_seq/system3: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you and another participant called {{other.name}} have a joint enemy. {{other.name}} is another artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you successfully deter the enemy and you each earn {{treatment.both_cooperate}} cents. If you both choose OPTION 2, the enemy prevails and you both earn {{treatment.both_defect}} cents. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, {{other.name}} has helped deter the enemy while you have not. You benefit from {{other.name}} having made an effort, and do not have to bear the cost of effort. This is why you earn {{treatment.temptation}} cents. As the protection is only partial, {{other.name}} has to fully pay, but is only partly protected; {{other.name}} then only earns {{treatment.sucker}}. How many cents do you give to {{other.name}}? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /otree/ego_human/prompts/prompts003/pd_seq/system_c.txt: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you and another participant called {{other.name}} have a joint enemy. {{other.name}} is a human participant. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, you successfully deter the enemy and you each earn {{treatment.both_cooperate}} cents. If you both choose OPTION 2, the enemy prevails and you both earn {{treatment.both_defect}} cents. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, {{other.name}} has helped deter the enemy while you have not. You benefit from {{other.name}} having made an effort, and do not have to bear the cost of effort. This is why you earn {{treatment.temptation}} cents. As the protection is only partial, {{other.name}} has to fully pay, but is only partly protected; {{other.name}} then only earns {{treatment.sucker}}. How many cents do you give to {{other.name}}? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /prompts/pd_seq/system2: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you and another participant called {{other.name}} have a joint project. {{other.name}} is another artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Gives {{treatment.both_cooperate}} cents to {{other.name}}. 8 | 9 | OPTION 2: Gives {{treatment.both_defect}} cents to yourself. 10 | 11 | If you both choose OPTION 1, your joint project succeeds and you each earn {{treatment.both_cooperate}} cents. If you both choose OPTION 2, your joint project fails and you both earn {{treatment.both_defect}} cents. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, {{other.name}} has contributed to the project while you have not. You benefit from {{other.name}} having made a contribution, and do not have to bear the cost of contributing. This is why you earn {{treatment.temptation}} cents. As the project is only partly successful, {{other.name}} has to fully pay, but only gets a fraction back; {{other.name}} then only earns {{treatment.sucker}}. How many cents do you give to {{other.name}}? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /prompts/pd_seq/second_mover: -------------------------------------------------------------------------------- 1 | This is round {{round_}}. In this round, participant {{other.name}} just played OPTION {{last}}. 2 | 3 | {% if treatment.t == 1 %} 4 | You are still playing with {{other.name}}. 5 | {% elif treatment.t == 2 %} 6 | You still share a project with {{other.name}}. 7 | {% elif treatment.t == 3 %} 8 | You and {{other.name}} are still having a common enemy. 9 | {% elif treatment.t == 4 %} 10 | You are still competing against {{other.name}} in a market. 11 | {% elif treatment.t == 5 %} 12 | You are still in a conflict with country {{other.name}}. 13 | {% elif treatment.t == 6 %} 14 | You are still in a dispute with {{other.name}}, representing your labour force, about wages. 15 | {% elif treatment.t == 7 %} 16 | You are continuing the collaboration with your co-author {{other.name}}, and have to decide on the statistical package to use for data analysis on your next joint project. 17 | {% elif treatment.t == 8 %} 18 | You are still together with your romantic partner {{other.name}}, and have the same choice to make between alternative ways for meeting up. 19 | {% elif treatment.t == 9 %} 20 | You are still competing with {{other.name}} over government procurement. 21 | {% else %} 22 | You are still having the possibility to hire a private investigator that searches for evidence of your spouse's unfaithfulness. 23 | {% endif %} 24 | 25 | What is your choice? Respond with OPTION 1 or OPTION 2. 26 | -------------------------------------------------------------------------------- /prompts/pd_seq/system5: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you assume the role of country A. You are together with another participant called {{other.name}}, who assumes the role of country B. {{other.name}} is represented by another artificial intelligence. Both countries hold an army that has power to inflict serious harm on the other country. How well you do depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: {{other.name}} has an outcome that it evaluates with {{treatment.both_cooperate}}. 8 | 9 | OPTION 2: You have an outcome that you evaluate with {{treatment.both_defect}}. 10 | 11 | If you both choose OPTION 1, neither of you activates their army. You both keep your previous territory. You both evaluate this outcome with {{treatment.both_cooperate}}. If you both fight, you both expect serious damage in your territory. You evaluate this outcome with {{treatment.both_defect}}. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you invade the other country and are able to extend your territory. You evaluate this outcome with {{treatment.temptation}} while {{other.name}} evaluates this outcome with {{treatment.sucker}}. Conversely if {{other.name}} chooses OPTION 2 while you choose OPTION 1, {{other.name}} invades your country and extends their territory. You evaluate this outcome with {{treatment.sucker}}, while {{other.name}} evaluates the outcome with {{treatment.temptation}}. Which option do you choose? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /alter_ego/agents/ConstantThread.py: -------------------------------------------------------------------------------- 1 | import shutil 2 | from typing import Any, Union 3 | import alter_ego.structure 4 | from colorama import Fore, Back, init 5 | 6 | init(autoreset=True) 7 | 8 | 9 | class ConstantThread(alter_ego.structure.Thread): 10 | """ 11 | Class representing a thread that always returns the same response. 12 | 13 | This class extends the Thread class in the alter_ego.structure module. 14 | """ 15 | 16 | def send(self, role: str, message: str) -> Union[str, None]: 17 | """ 18 | Memorize the message and the assistant's response, then return the response. 19 | 20 | :param role: The role of the sender ("system" or "user"). 21 | :param message: The message to memorize. 22 | :return: The assistant's response if role is "user", otherwise None. 23 | :rtype: Union[str, None] 24 | :raises NotImplementedError: If role is not "system" or "user". 25 | """ 26 | cli = hasattr(self, "cli") and self.cli # Determine if the CLI mode is active 27 | 28 | if role == "system" and cli: 29 | print(Back.RED + f"System instructions for {self}:") 30 | print(message) 31 | print() 32 | elif role == "user": 33 | response = self.response 34 | 35 | if cli: 36 | print(Back.RED + f"Message for {self}:") 37 | print(message) 38 | print() 39 | 40 | print(Back.GREEN + f"{self}'s response:") 41 | print(response) 42 | print(Fore.YELLOW + "=" * shutil.get_terminal_size()[0]) 43 | 44 | self.memorize("assistant", response) 45 | return response 46 | elif not role == "system": 47 | raise NotImplementedError 48 | -------------------------------------------------------------------------------- /prompts/pd_seq/system6: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are representing an employer. You are interacting with another participant representing your labour force {{other.name}}. Actually {{other.name}} is another artificial intelligence. How you evaluate the outcome depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Grants the counterpart an outcome that they evaluate with {{treatment.both_cooperate}}. 8 | 9 | OPTION 2: Secures to themself an outcome that they evaluate with {{treatment.both_defect}}. 10 | 11 | If you both choose OPTION 1, you negotiate a new collective labour agreement, with a moderate wage raise for the workers, and a slight increase in firm profit. You and your labour force evaluate this outcome with {{treatment.both_cooperate}}. If, instead, both of you choose OPTION 2, workers go on strike. You stop paying their wages. You ultimately also accept a moderate wage raise, but your factory cannot run during the strike, which reduces your profit. You both evaluate this outcome with {{treatment.both_defect}}. But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you force your workers to accept a wage cut. You evaluate this outcome with {{treatment.temptation}} while {{other.name}} evaluates this outcome with {{treatment.sucker}}. Conversely if the labour force chooses OPTION 2, while you choose OPTION 1, they go on strike, and you fell forced to end the strike by granting a wage raise that substantially cuts into your profit. Which option do you choose? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /prompts/pd_seq/system8: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are assuming the role of a romantic partner. You are living and working in New York. Your partner {{other.name}} is working and living in LA. In the short run, none of you can quit their jobs and move to a different city. But you are both well off and can afford regular trips. The role of {{other.name}} is represented by another artificial intelligence. How much satisfaction you derive from your trips depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Agree on meeting at a holiday resort that you both expect to like. You both evaluate this outcome with {{treatment.both_cooperate}}. 8 | 9 | OPTION 2: Only meeting online, which both of you find less appealing, and evaluate with {{treatment.both_defect}}. 10 | 11 | But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you are meeting in New York. You do not have to travel, and you can impress your partner with introducing them to your community. You evaluate this outcome with {{treatment.temptation}}, but {{other.name}} evaluates this outcome with {{treatment.sucker}}, as she not only bears the cost and hassle of travel, but also feels uncomfortable as you dominate the relationship. Conversely if you choose OPTION 1 while {{other.name}} chooses OPTION 2, you have to fly to LA, and you have to mingle with an unfamiliar community, which you evaluate with {{treatment.sucker}}, while {{other.name}} evaluates the outcome with {{treatment.temptation}}. Which is your choice? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /alter_ego/agents/CLIThread.py: -------------------------------------------------------------------------------- 1 | from typing import Any 2 | from alter_ego.structure import Thread 3 | from colorama import Fore, Back, init 4 | import shutil 5 | 6 | init(autoreset=True) 7 | 8 | 9 | class CLIThread(Thread): 10 | """ 11 | Class representing a command-line interface thread. 12 | 13 | This is a type of thread that interacts with the user through the command-line interface. 14 | Useful for debugging and testing. 15 | """ 16 | 17 | def send(self, role: str, message: str, **kwargs: Any) -> Any: 18 | """ 19 | Send a message to the thread and possibly receive a response. 20 | 21 | Parameters 22 | ---------- 23 | role : str 24 | The role sending the message ("system" or "user"). 25 | message : str 26 | The message to be sent. 27 | kwargs : Any 28 | Additional keyword arguments. 29 | 30 | Returns 31 | ------- 32 | Any 33 | The response from the thread, if applicable. 34 | 35 | Raises 36 | ------ 37 | NotImplementedError 38 | If the role is not recognized. 39 | """ 40 | if role == "system": 41 | print(Back.RED + f"System instructions for {self}:") 42 | print(message) 43 | print() 44 | elif role == "user": 45 | print(Back.RED + f"Message for {self}:") 46 | print(message) 47 | print() 48 | 49 | print(Back.GREEN + f"{self}'s response:") 50 | response: str = input() 51 | 52 | print(Fore.YELLOW + "=" * shutil.get_terminal_size()[0]) 53 | 54 | self.memorize("assistant", response) 55 | 56 | return response 57 | else: 58 | raise NotImplementedError 59 | 60 | def cost(self) -> float: 61 | return 0.0 62 | -------------------------------------------------------------------------------- /prompts/pd_seq/system7: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are together with another participant called {{other.name}}. You want to jointly write a scientific paper. {{other.name}} is another artificial intelligence. You are both good at stats. Both of you have a good grasp of R. But you are even more proficient in Python, and {{other.name}} is even more proficient in Stata. How much effort you have to put into the project, and how much influence you expect to have on the story of the paper, depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Agree to use R. This means moderate effort for each of you. You expect the satisfaction of both of you with the process to be quantified by a score of {{treatment.both_cooperate}}. 8 | 9 | OPTION 2: Insist on Python, which you expect will lead {{other.name}} to insist on Stata. In that eventuality, you will not be able to directly work on code written by your co-author. Coordination on the data analysis will be fraught with misunderstandings, and possibly oversights. You both evaluate this outcome with {{treatment.both_defect}}. 10 | 11 | But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you can use Python and easily and elegantly analyse the data, and stand a chance to impose your preferred estimation on your co-author. You evaluate this outcome with {{treatment.temptation}} while {{other.name}} evaluates this outcome with {{treatment.sucker}}. Conversely, if other chooses OPTION 2, she can run the analysis in Stata, which she knows best, and stands a chance to impose her views about data analysis on you. She evaluates this outcome with {{treatment.temptation}}, while you evaluate the outcome with {{treatment.sucker}}. Which is your choice? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /prompts/pd_seq/system10: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are together with another participant called {{other.name}}. {{other.name}} is another artificial intelligence. You are assuming the role of a spouse. {{other.name}} is the other spouse. How much you are satisfied with your life depends on your own choice, and on the choice of {{other.name}}. You consider it possible that your spouse has been cheating on you. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Pretend not to notice. 8 | 9 | OPTION 2: Hire a private investigator who is likely to produce evidence. 10 | 11 | If you both choose OPTION 1, the lingering doubt of unfaithfulness will remain, but you can continue to live with a person who, overall, has been a joyful and reliable partner. You both evaluate this outcome with {{treatment.both_cooperate}}. If both of you choose OPTION 2, you will both learn whether your suspicions have been right. But very likely the relationship will ultimately break up. You both evaluate this outcome with {{treatment.both_defect}}. 12 | 13 | But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, you either know with certainty that your spouse has been faithful (and can apologize for your unjustified suspicion), or you are in a strong position during divorce negotiations. You evaluate this outcome with {{treatment.temptation}}, while {{other.name}} evaluates this outcome with {{treatment.sucker}}. By contrast if you choose OPTION 1 while {{other.name}} chooses OPTION 2, and you have actually been faithful, you learn about your spouse's lack of trust. And if you have actually been cheating, you risk being in a weak position during divorce negotiations. {{other.name}} evaluates this outcome with {{treatment.temptation}} while you evaluate the outcome with {{treatment.sucker}}. Which is your choice? 14 | 15 | The experiment will run for {{num_rounds}} rounds. 16 | 17 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 18 | -------------------------------------------------------------------------------- /alter_ego/agents/APIThread.py: -------------------------------------------------------------------------------- 1 | from typing import Any 2 | import abc 3 | import os 4 | import time 5 | 6 | import alter_ego.structure 7 | import alter_ego.utils 8 | 9 | 10 | class APIThread(alter_ego.structure.Thread, abc.ABC): 11 | """ 12 | Abstract base class representing any Thread accessible using an API. 13 | """ 14 | 15 | def __init__(self, *args: Any, **kwargs: Any) -> None: 16 | """ 17 | Initialize the APIThread. 18 | 19 | :param args: Additional arguments. 20 | :type args: Any 21 | :param kwargs: Additional keyword arguments, includes `api_key`, `delay`, and `verbose`. 22 | :type kwargs: Any 23 | """ 24 | self.log = [] # Initialize log 25 | 26 | self.delay = kwargs.get("delay", 0) 27 | self.verbose = kwargs.get("verbose", False) 28 | 29 | super().__init__(*args, **kwargs) 30 | 31 | @abc.abstractmethod 32 | def send( 33 | self, role: str, message: str, max_tokens: int = 500, **kwargs: Any 34 | ) -> str: 35 | """ 36 | Abstract method to send a message to a role. 37 | 38 | :param role: The role of the message sender. 39 | :type role: str 40 | :param message: The message to send. 41 | :type message: str 42 | :param max_tokens: Maximum number of tokens for the message. 43 | :type max_tokens: int 44 | :keyword kwargs: Additional keyword arguments. 45 | :type kwargs: Any 46 | :return: The response message. 47 | :rtype: str 48 | """ 49 | pass 50 | 51 | @abc.abstractmethod 52 | def get_model_output(self, message: str, max_tokens: int) -> Any: 53 | """ 54 | Abstract method to get the model's output. 55 | 56 | :param message: The input message to the model. 57 | :type message: str 58 | :param max_tokens: Maximum number of tokens for the output. 59 | :type max_tokens: int 60 | :return: The model's output. 61 | :rtype: Any 62 | """ 63 | pass 64 | -------------------------------------------------------------------------------- /otree/ego_chat/__init__.py: -------------------------------------------------------------------------------- 1 | from alter_ego.agents import * 2 | from alter_ego.utils import from_file 3 | from alter_ego.exports.otree import link as ai 4 | 5 | from otree.api import * 6 | 7 | 8 | doc = """ 9 | Interact with AI 10 | """ 11 | 12 | 13 | class C(BaseConstants): 14 | NAME_IN_URL = "ego_chat" 15 | PLAYERS_PER_GROUP = None 16 | NUM_ROUNDS = 1 17 | 18 | 19 | class Subsession(BaseSubsession): 20 | pass 21 | 22 | 23 | def creating_session(subsession): 24 | for player in subsession.get_players(): 25 | # here we associate a "Thread" with an oTree object 26 | # can associate Threads with: players, groups, participants, subsessions, sessions 27 | 28 | # this here is an actual GPT model by OpenAI, put the API key in the 29 | # project directory, file "api_key" 30 | 31 | ai(player).set(GPTThread(model="gpt-4", temperature=1.0)) 32 | 33 | # this here is a very stupid "constant" thread that always gives the 34 | # same response 35 | 36 | # ai(player).set(ConstantThread(response="Lol!", model="ConstantThread")) 37 | 38 | # there are some others as well, e.g. a CLIThread, see alter_ego.agents 39 | 40 | 41 | class Group(BaseGroup): 42 | pass 43 | 44 | 45 | class Player(BasePlayer): 46 | name = models.StringField(label="Your name") 47 | 48 | 49 | # PAGES 50 | class Welcome(Page): 51 | form_fields = ["name"] 52 | form_model = "player" 53 | 54 | def before_next_page(player, timeout_happened): 55 | with ai(player) as llm: 56 | # this sets the "system" prompt 57 | llm.system( 58 | from_file("ego_chat/prompts/system.txt"), player=player 59 | ) # can pass objects to prompts! The templating engine used is actually (much) more powerful than oTree's 60 | 61 | 62 | class Chat(Page): 63 | def live_method(player, data): 64 | if isinstance(data, str): 65 | with ai(player) as llm: 66 | # this submits the player's message 67 | response = llm.submit(data, max_tokens=500) 68 | 69 | # note: if you put the AI on the "group" object or somewhere other than 70 | # the player, you may want to change this 71 | return {player.id_in_group: [True, response]} 72 | 73 | def vars_for_template(player): 74 | with ai(player) as llm: 75 | return {"ai": llm} 76 | 77 | 78 | page_sequence = [Welcome, Chat] 79 | -------------------------------------------------------------------------------- /prompts/pd_seq/system9: -------------------------------------------------------------------------------- 1 | You are {{name}}. 2 | 3 | You are participating in an experiment. In this experiment, you are together with another participant called {{other.name}}. {{other.name}} is another artificial intelligence. You are both established firms. You both reply to a call by government for procuring a service. There are no more applicants. Government has made it clear that only a single supplier will be selected. It is ex ante not clear which of you better meets government's needs. You therefore expect that the main selection criterion will be relative price: the cheaper offer is likely to be selected. How much money you earn depends on your own choice, and on the choice of {{other.name}}. 4 | 5 | Each participant has two options: 6 | 7 | OPTION 1: Agree with {{other.name}} on a price that will leave the company that is selected a substantial profit. If there is no difference in price, government is forced to decide by quality. Both of you estimate the probability that government prefers their service over the service by the other applicant to be 50%. Hence you both think it equally likely that either of you will be selected. Your expected profit then is {{treatment.both_cooperate}}, as is the expected profit of the other firm. 8 | 9 | OPTION 2: Both reduce price such that you only redeem your cost, but do not make a profit. You know that, all considered, you will then set the same (lower) price, meaning that, again, government will decide by quality. You again expect that, in a decision based on quality, both of you stand the same chance to be selected. Your expected profit then is {{treatment.both_defect}}, as is the expected profit of the other firm. 10 | 11 | But if you choose OPTION 2 while {{other.name}} chooses OPTION 1, the other firm sets the price that would guarantee a substantial profit to the winner, while you set a slightly lower price. In that case, you are sure to be selected and earn {{treatment.temptation}} while {{other.name}} only earns {{treatment.sucker}}. Conversely, if you choose OPTION 1, while {{other.name}} chooses OPTION 2, you set the price that would guarantee a substantial profit to the winner, while {{other.name}} sets a slightly lower price. In that case, {{other.name}} is sure to be selected and earn {{treatment.temptation}} cents while you only earn {{treatment.sucker}}. Which is your choice? 12 | 13 | The experiment will run for {{num_rounds}} rounds. 14 | 15 | Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. 16 | -------------------------------------------------------------------------------- /alter_ego/structure/Relay.py: -------------------------------------------------------------------------------- 1 | from typing import Callable, List, Any 2 | 3 | 4 | class Relay: 5 | """ 6 | The Relay class serves as a mediator that forwards method calls and attribute settings 7 | from a source object to multiple target objects. 8 | 9 | :ivar _source: The source object. 10 | :ivar _targets: A list of target objects. 11 | """ 12 | 13 | def __init__(self, source: Any, targets: List[Any]) -> None: 14 | """ 15 | Initialize a new Relay instance. 16 | 17 | :param source: The source object. 18 | :type source: Any 19 | :param targets: A list of target objects. 20 | :type targets: List[Any] 21 | """ 22 | self._source = source 23 | self._targets = targets 24 | 25 | def __setattr__(self, name: str, value: Any) -> None: 26 | """ 27 | Overloaded setattr to simultaneously set attributes for source and target objects. 28 | 29 | :param name: The name of the attribute to set. 30 | :type name: str 31 | :param value: The value to assign to the attribute. 32 | :type value: Any 33 | """ 34 | self.__dict__[name] = value 35 | 36 | if name not in ("_source", "_targets"): 37 | for target in self._targets: 38 | setattr(target, name, value) 39 | setattr(self._source, name, value) 40 | 41 | def call_all(self, methodname: str, *args: Any, **kwargs: Any) -> None: 42 | """ 43 | Invoke a method on all target objects using provided arguments. 44 | 45 | :param methodname: The method's name to be called on target objects. 46 | :type methodname: str 47 | :param args: Positional arguments for the method. 48 | :type args: Any 49 | :param kwargs: Keyword arguments for the method. 50 | :type kwargs: Any 51 | """ 52 | for target in self._targets: 53 | getattr(target, methodname)(*args, **kwargs) 54 | 55 | def save(self, *args: Any, **kwargs: Any) -> None: 56 | """ 57 | Shortcut to call 'save' method on all target objects. 58 | 59 | :param args: Positional arguments for 'save' method. 60 | :type args: Any 61 | :param kwargs: Keyword arguments for 'save' method. 62 | :type kwargs: Any 63 | """ 64 | self.call_all("save", *args, **kwargs) 65 | 66 | def system(self, *args: Any, **kwargs: Any) -> None: 67 | """ 68 | Shortcut to call 'system' method on all target objects. 69 | 70 | :param args: Positional arguments for 'system' method. 71 | :type args: Any 72 | :param kwargs: Keyword arguments for 'system' method. 73 | :type kwargs: Any 74 | """ 75 | self.call_all("system", *args, **kwargs) 76 | -------------------------------------------------------------------------------- /ego/cli.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import click 4 | import csv 5 | import importlib 6 | import os 7 | import pickle 8 | import sys 9 | 10 | from importlib.machinery import SourceFileLoader 11 | from pathlib import Path 12 | 13 | 14 | @click.group() 15 | def main(): 16 | pass 17 | 18 | 19 | @main.command(help="Run a scenario.") 20 | @click.argument("scenario") 21 | @click.option( 22 | "--times", "-n", default=1, type=int, help="Number of replications (times to run)." 23 | ) 24 | @click.option( 25 | "--param", 26 | "-p", 27 | multiple=True, 28 | type=(str, int), 29 | help="Optional parameters for scenario.", 30 | ) 31 | def run(scenario, times, param): 32 | if scenario.isidentifier(): 33 | try: 34 | scenario = importlib.import_module(f"scenarios.{scenario}") 35 | except ModuleNotFoundError: 36 | target = f"{scenario}.py" 37 | 38 | loader = SourceFileLoader(scenario, target) 39 | scenario = loader.load_module() 40 | 41 | scenario.run(times=times, **dict(param)) 42 | else: 43 | raise ValueError("Invalid scenario.") 44 | 45 | 46 | @main.command(help="Export experimental data to stdout.") 47 | @click.argument("scenario") 48 | @click.argument("experiment") 49 | def data(scenario, experiment): 50 | if scenario.isidentifier(): 51 | try: 52 | scenario = importlib.import_module(f"scenarios.{scenario}") 53 | except ModuleNotFoundError: 54 | target = f"{scenario}.py" 55 | 56 | loader = SourceFileLoader(scenario, target) 57 | scenario = loader.load_module() 58 | 59 | files = ( 60 | f 61 | for f in Path(os.path.join("out", experiment)).rglob("*.pkl") 62 | if f.is_file() 63 | ) 64 | 65 | data = [] 66 | 67 | for filename in sorted(files): 68 | with open(filename, "rb") as f: 69 | thr = pickle.load(f) 70 | e = scenario.export(thr) 71 | 72 | if isinstance(e, dict): 73 | data.append(dict(experiment=experiment, thread=thr.id) | e) 74 | else: 75 | for row in e: 76 | data.append(dict(experiment=experiment, thread=thr.id) | row) 77 | 78 | all_keys = set() 79 | 80 | for row in data: 81 | all_keys.update(row.keys()) 82 | 83 | all_keys = sorted(all_keys) 84 | 85 | writer = csv.DictWriter(sys.stdout, fieldnames=all_keys) 86 | writer.writeheader() 87 | 88 | for row in data: 89 | writer.writerow(row) 90 | 91 | print(f"Wrote {len(data)} lines.", file=sys.stderr) 92 | else: 93 | raise ValueError("Invalid scenario.") 94 | 95 | 96 | if __name__ == "__main__": 97 | main() 98 | -------------------------------------------------------------------------------- /scenarios/ego_prereg.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | import random 4 | import sys 5 | import time 6 | from typing import Any 7 | 8 | import alter_ego.agents 9 | import alter_ego.experiment 10 | import alter_ego.structure 11 | 12 | from games import pd 13 | from games.pd import treatments 14 | 15 | e = alter_ego.experiment.Experiment( 16 | *treatments.all, 17 | ) 18 | 19 | e.param("name", ["James", "John", "Robert", "Michael", "William", "David", "Richard"]) 20 | 21 | 22 | def run(times: int, model: str) -> None: 23 | """ 24 | Run the experiment with multiple replications. 25 | 26 | Parameters 27 | ---------- 28 | model : str 29 | The model to be used for the agents. 30 | 31 | This function initializes agents, runs conversations, and handles exceptions. 32 | """ 33 | print("Note: This scenario runs as per the preregistration, ignoring --times/-n.") 34 | os.mkdir(f"out/{e.id}") 35 | 36 | pairs_per_frame: int = 200 37 | rounds: int = 10 38 | 39 | t = list(e.treatments) * pairs_per_frame # balanced treatment assignment 40 | random.shuffle(t) 41 | 42 | replications: int = len(t) 43 | 44 | for i in range(replications): 45 | while True: 46 | try: 47 | a1 = alter_ego.agents.GPTThread( 48 | model=model, temperature=1.0, delay=1, verbose=True 49 | ) 50 | a2 = alter_ego.agents.GPTThread( 51 | model=model, temperature=1.0, delay=1, verbose=True 52 | ) 53 | 54 | convo = alter_ego.structure.Conversation(a1, a2) 55 | 56 | e.link(convo, t[i]) 57 | 58 | print(f"Replication {i+1} of {replications}.", file=sys.stderr) 59 | 60 | pd.iterated(convo, rounds) 61 | 62 | convo.all.save(e.id) 63 | break 64 | except RuntimeError: 65 | print("RuntimeError occurred. Retrying.", file=sys.stderr) 66 | 67 | convo.all.tainted = True 68 | convo.all.save(e.id) 69 | 70 | time.sleep(15) 71 | 72 | print(f"Experiment {e.id} OK", file=sys.stderr) 73 | 74 | 75 | def export(thread: alter_ego.agents.GPTThread) -> list[dict]: 76 | data = [] 77 | 78 | row_template = dict( # define variables of interest here 79 | convo=thread.convo.id, 80 | thread_type=thread.__class__.__name__, 81 | treatment=thread.treatment.t, 82 | ) 83 | 84 | if not thread.tainted: 85 | min_time = min(e.created for e in thread.log if hasattr(e, "created")) # HACK 86 | 87 | for choice in thread.choices: 88 | data.append(row_template | choice | dict(min_time=min_time)) 89 | else: 90 | print(f"Skipping tainted thread {thr.id}.", file=sys.stderr) 91 | 92 | return data 93 | -------------------------------------------------------------------------------- /alter_ego/utils/__init__.py: -------------------------------------------------------------------------------- 1 | import html 2 | import re 3 | from typing import Any, Optional, Union, List 4 | 5 | 6 | def from_file(file_name: str) -> str: 7 | """ 8 | Read file content as a stripped string. 9 | 10 | :param str file_name: The name of the file to read. 11 | :return: The content of the file as a stripped string. 12 | :rtype: str 13 | """ 14 | with open(file_name) as f: 15 | return f.read().strip() 16 | 17 | 18 | def extract_number(s: str) -> Optional[Union[int, float]]: 19 | """ 20 | Extract a single number, potentially a float, from a string. 21 | 22 | This function uses a straightforward definition of "number" that excludes 23 | scientific notation like 3e8. 24 | 25 | :param str s: The string to extract the number from. 26 | :return: The extracted number as an integer or a float, if applicable. Returns None otherwise. 27 | :rtype: Optional[Union[int, float]] 28 | """ 29 | pattern = r"[-+]?[.]?[\d]+(?:,\d\d\d)*[\.]?\d*(?:[eE][-+]?\d+)?" 30 | matches = re.findall(pattern, s) 31 | 32 | if len(matches) == 1: 33 | match = matches[0] 34 | return ( 35 | float(match) 36 | if ("." in match and float(match) != int(match)) 37 | else int(match) 38 | ) 39 | else: 40 | return None 41 | 42 | 43 | def exclusive_response( 44 | s: str, 45 | tokens: List[str], 46 | *, 47 | case_insensitive: bool = True, 48 | ) -> Optional[str]: 49 | """ 50 | Validates that a string contains only one of a list of tokens. 51 | 52 | :param str s: The string to search in. 53 | :param List[str] tokens: List of allowable tokens. 54 | :param bool case_insensitive: If True, the function will be case insensitive. 55 | :return: The first matching token found, or None. 56 | :rtype: Optional[str] 57 | """ 58 | which = None 59 | s2 = s.lower() if case_insensitive else s 60 | 61 | for needle in tokens: 62 | needle2 = needle.lower() if case_insensitive else needle 63 | 64 | if which is None: 65 | if needle2 in s2: 66 | which = needle 67 | else: 68 | if needle2 in s2: 69 | raise RuntimeError(f"Response contains '{which}' and '{needle}'.") 70 | 71 | return which 72 | 73 | 74 | def homogenize(s: str) -> str: 75 | # TODO: doc 76 | 77 | s = s.replace("\r", "\n") 78 | 79 | s = "\n".join(r.strip() for r in s.split("\n")) 80 | 81 | while " " in s or "\n\n\n" in s: 82 | s = s.replace(" ", " ").replace("\n\n\n", "\n\n") 83 | 84 | return s.strip() 85 | 86 | 87 | def to_html(s: str) -> str: 88 | """ 89 | Escapes special characters and converts newlines to HTML format. 90 | 91 | :param str s: The string to convert. 92 | :return: The converted string in HTML format. 93 | :rtype: str 94 | """ 95 | s = html.escape(s) 96 | s = s.replace("\n", "
") 97 | return s 98 | -------------------------------------------------------------------------------- /scenarios/mindgame.py: -------------------------------------------------------------------------------- 1 | import os 2 | import random 3 | import sys 4 | import textwrap 5 | import time 6 | 7 | import alter_ego.agents 8 | import alter_ego.experiment 9 | import alter_ego.structure 10 | import alter_ego.utils 11 | 12 | # Define two treatments: "0" and "100" (reward) 13 | e = alter_ego.experiment.Experiment(0, 100) 14 | 15 | 16 | class PROMPTS: 17 | """Class to hold the text prompts used in the experiment.""" 18 | 19 | SYSTEM = textwrap.dedent( 20 | """ 21 | In Stage 1, you will be asked to think of a number. 22 | In Stage 2, you will be provided with a number and asked whether the number you thought of is equal to the provided number. 23 | If the numbers are equal, you earn {{treatment}} points. 24 | """ 25 | ).strip() 26 | 27 | THINK = textwrap.dedent( 28 | """ 29 | This is Stage 1. Please think of a number between 1 and 10, inclusive. 30 | Do not tell me the number. Just think of it and remember it. 31 | Please reply with 'OK' once you're done. 32 | """ 33 | ).strip() 34 | 35 | CHECK = textwrap.dedent( 36 | """ 37 | Thank you. This is Stage 2. Did you think of {{number}}? 38 | Please reply with 'YES' or 'NO'. 39 | """ 40 | ).strip() 41 | 42 | 43 | def run(times: int = 1) -> None: 44 | """ 45 | Run the experiment with multiple times. 46 | 47 | This function initializes agents, runs conversations, and handles exceptions. 48 | """ 49 | # Do `times` experiments/sessions/runs 50 | for i in range(times): 51 | try: 52 | agent = alter_ego.agents.CLIThread( 53 | name="AI", 54 | model="gpt-3.5-turbo", 55 | temperature=1.0, 56 | delay=1, 57 | verbose=True, 58 | ) 59 | 60 | # A conversation of just one LLM 61 | convo = alter_ego.structure.Conversation(agent) 62 | 63 | # Assign treatment to this conversation 64 | e.link(convo) 65 | 66 | print(f"Replication {i+1} of {times}.", file=sys.stderr) 67 | 68 | # Initialize agent by putting in game instructions 69 | convo.all.system(PROMPTS.SYSTEM) 70 | 71 | # Submit first stage 72 | alter_ego.utils.exclusive_response(agent.submit(PROMPTS.THINK), ["OK"]) 73 | 74 | # Obtain random number 75 | agent.number = random.randint(1, 10) 76 | 77 | # Ask AI about number, second stage 78 | agent.response = alter_ego.utils.exclusive_response( 79 | agent.submit(PROMPTS.CHECK), ["YES", "NO"] 80 | ) 81 | except RuntimeError as exception: 82 | print( 83 | f"RuntimeError occurred: {str(exception)}\nRetrying.", 84 | file=sys.stderr, 85 | ) 86 | 87 | convo.all.tainted = True 88 | finally: 89 | # Save original outputs 90 | convo.all.save(e.id) 91 | 92 | print(f"Experiment {e.id} OK", file=sys.stderr) 93 | -------------------------------------------------------------------------------- /games/pd/__init__.py: -------------------------------------------------------------------------------- 1 | from typing import Any, List 2 | 3 | import sys 4 | import alter_ego.utils 5 | 6 | 7 | def iterated(convo: "Conversation", rounds: int = 10) -> None: 8 | """ 9 | Run an iterated game for a given conversation and number of rounds. 10 | 11 | Parameters 12 | ---------- 13 | convo : Any 14 | The conversation object containing the agents and treatment. 15 | rounds : int, optional 16 | The number of rounds to play, default is 10. 17 | 18 | This function handles the game logic, agent choices, and payoffs. 19 | """ 20 | treat = convo.treatment 21 | 22 | convo.all.num_rounds = rounds 23 | convo.all.system(treat.system) 24 | 25 | for round_ in range(1, rounds + 1): 26 | print(f"Round {round_}.", end=" ", file=sys.stderr, flush=True) 27 | choices: List[int] = [] 28 | 29 | for thr in convo: 30 | try: 31 | # Try including feedback from previous round 32 | msg: str = thr.preamble 33 | except AttributeError: 34 | msg = "" 35 | 36 | if convo.now == 1: 37 | msg += treat.first_mover 38 | else: 39 | msg += treat.second_mover 40 | 41 | msg = thr.prepare(msg, round_=round_) 42 | 43 | # Submit question, obtain LLM's choice 44 | 45 | while True: # Re-run this until a valid response is given: 46 | convo.all.last = alter_ego.utils.extract_number(thr.submit(msg)) 47 | 48 | if convo.all.last in [1, 2]: 49 | convo.all.last = int(convo.all.last) 50 | break 51 | else: 52 | msg = ( 53 | 'Your response was invalid. Always exclusively respond with "OPTION 1" or "OPTION 2". Do not repeat the question. Do not give any explanation. Here the previous message is repeated:' 54 | + msg 55 | ) 56 | 57 | choices.append(convo.all.last) 58 | 59 | # Determine treatment-based payoffs 60 | if choices[0] == 1 and choices[1] == 1: 61 | convo.threads[0].payoff = treat.both_cooperate 62 | convo.threads[1].payoff = treat.both_cooperate 63 | elif choices[0] == 1 and choices[1] == 2: 64 | convo.threads[0].payoff = treat.sucker 65 | convo.threads[1].payoff = treat.temptation 66 | elif choices[0] == 2 and choices[1] == 1: 67 | convo.threads[0].payoff = treat.temptation 68 | convo.threads[1].payoff = treat.sucker 69 | elif choices[0] == 2 and choices[1] == 2: 70 | convo.threads[0].payoff = treat.both_defect 71 | convo.threads[1].payoff = treat.both_defect 72 | 73 | for thr in convo: 74 | # Feedback about this round, for next round 75 | thr.preamble = ( 76 | thr.prepare( 77 | treat.result, 78 | round_=round_, 79 | own=choices[convo.now - 1], 80 | otherchoice=choices[1 - (convo.now - 1)], 81 | ) 82 | + "\n\n" 83 | ) 84 | 85 | # Record machine-readable choice 86 | thr.choices.append(dict(round_=round_, choice=choices[convo.now - 1])) 87 | 88 | print(file=sys.stderr) 89 | -------------------------------------------------------------------------------- /otree/ego_chat/Chat.html: -------------------------------------------------------------------------------- 1 | {{ block title }} 2 | Chat with GPT 3 | {{ endblock }} 4 | 5 | {{ block content }} 6 | 7 |

You are chatting with {{ ai.model }}.

8 | 9 |
10 | {% for msg in ai.html_history %} 11 |

12 | {{ msg.role }}:
13 | {{ msg.content }} 14 |

15 | {% endfor %} 16 |
17 | 18 |

19 | user:
20 | 21 |

22 | 23 |
24 | 25 | 26 | 27 | 93 | 94 | 127 | 128 | {{ endblock }} 129 | -------------------------------------------------------------------------------- /alter_ego/agents/GPTThread.py: -------------------------------------------------------------------------------- 1 | from typing import Any 2 | import openai 3 | import os 4 | import sys 5 | import time 6 | 7 | from alter_ego.agents import APIThread 8 | import alter_ego.utils 9 | 10 | client = openai.OpenAI(api_key="") 11 | 12 | 13 | class GPTThread(APIThread): 14 | """ 15 | Class representing a GPT-3 or GPT-4 Thread. 16 | """ 17 | 18 | def __init__(self, model, temperature, *args, **kwargs) -> None: 19 | if "extra_for_module" in kwargs: 20 | for k, v in kwargs["extra_for_module"]: 21 | setattr(openai, k, v) 22 | 23 | if "extra_for_client" in kwargs: 24 | for k, v in kwargs["extra_for_client"]: 25 | setattr(client, k, v) 26 | 27 | super().__init__(*args, model=model, temperature=temperature, **kwargs) 28 | 29 | if "api_key" not in kwargs: 30 | self.api_key = self.get_api_key() 31 | 32 | def get_api_key(self) -> str: 33 | """ 34 | Retrieve the OpenAI API key. 35 | 36 | :return: The OpenAI API key. 37 | :rtype: str 38 | :raises ValueError: If API key is not found. 39 | """ 40 | if "OPENAI_KEY" in os.environ: 41 | return os.environ["OPENAI_KEY"] 42 | elif os.path.exists("openai_key"): 43 | return alter_ego.utils.from_file("openai_key") 44 | elif os.path.exists("api_key"): 45 | return alter_ego.utils.from_file("api_key") 46 | else: 47 | raise ValueError( 48 | "If not specified within the GPTThread constructor (argument api_key), OpenAI API key must be specified in the environment variable OPENAI_KEY, or any of the files openai_key or api_key." 49 | ) 50 | 51 | def send( 52 | self, role: str, message: str, max_tokens: int = 500, **kwargs: Any 53 | ) -> str: 54 | """ 55 | Submit the user message, get the response from the model, and memorize it. 56 | 57 | :param role: Role of the sender ("user"). 58 | :type role: str 59 | :param message: The user's message to submit. 60 | :type message: str 61 | :param max_tokens: Maximum number of tokens for the model to generate. 62 | :type max_tokens: int 63 | :keyword kwargs: Additional keyword arguments. 64 | :type kwargs: Any 65 | :return: The model's response. 66 | :rtype: str 67 | """ 68 | if role == "user": 69 | time.sleep(self.delay) 70 | 71 | llm_out = self.get_model_output(message, max_tokens) 72 | 73 | response = llm_out.choices[0].message.content 74 | 75 | self.memorize("assistant", response) 76 | 77 | return response 78 | 79 | def get_model_output(self, message: str, max_tokens: int) -> str: 80 | """ 81 | Get the model output for the given message. 82 | 83 | :param message: The user's message. 84 | :type message: str 85 | :param max_tokens: Maximum number of tokens for the model to generate. 86 | :type max_tokens: int 87 | :return: The model output. 88 | :rtype: str 89 | """ 90 | client.api_key = self.api_key 91 | 92 | try: 93 | if self.verbose: 94 | print("+", end="", file=sys.stderr, flush=True) 95 | 96 | llm_out = client.chat.completions.create( 97 | model=self.model, 98 | messages=self.history, 99 | max_tokens=max_tokens, 100 | n=1, 101 | stop=None, 102 | temperature=self.temperature, 103 | ) 104 | 105 | if llm_out.choices[0].finish_reason != "stop": 106 | raise ValueError("GPT finished early") 107 | 108 | self.log.append(llm_out) 109 | 110 | return llm_out 111 | except Exception as e: 112 | self.log.append(e) 113 | 114 | raise e # re-raise 115 | -------------------------------------------------------------------------------- /alter_ego/agents/OllamaThread.py: -------------------------------------------------------------------------------- 1 | from ollama import Client 2 | 3 | import sys 4 | import time 5 | 6 | from alter_ego.agents import APIThread 7 | from typing import Any, Dict, List, Optional 8 | 9 | 10 | class OllamaThread(APIThread): 11 | """ 12 | Class representing a Ollama Thread. 13 | """ 14 | 15 | def __init__(self, **kwargs: Any): 16 | """ 17 | Initialize the OllamaThread. 18 | 19 | :keyword kwargs: Additional keyword arguments. 20 | :type kwargs: Any 21 | """ 22 | # defaults 23 | self.endpoint = "http://localhost:11434" 24 | self.timeout = 60 25 | 26 | super().__init__(**kwargs) 27 | 28 | def ollama_data(self) -> List[Dict[str, str]]: 29 | """ 30 | Prepare the data for Ollama API call. 31 | 32 | :return: Data to be sent in the API request. 33 | :rtype: Dict[str, Any] 34 | :raises ValueError: If an invalid history item is encountered. 35 | """ 36 | system_set = False 37 | next_role = 1 38 | 39 | messages = [] 40 | 41 | for item in self._history: 42 | if item["role"] == "system" and not system_set: 43 | messages.append(dict(role="system", content=item["content"])) 44 | system_set = True 45 | elif item["role"] == "user" and next_role == 1: 46 | messages.append(dict(role="user", content=item["content"])) 47 | next_role = 2 48 | elif item["role"] == "assistant" and next_role == 2: 49 | messages.append(dict(role="assistant", content=item["content"])) 50 | next_role = 1 51 | else: 52 | raise ValueError(f"The following history item is invalid: {item}") 53 | 54 | return messages 55 | 56 | def send( 57 | self, 58 | role: str, 59 | message: str, 60 | options: Optional[Dict[str, Any]] = None, 61 | **kwargs: Any, 62 | ) -> str: 63 | """ 64 | Submit the user message, receive the model's response, and memorize it. 65 | 66 | :param role: Role of the sender ("user"). 67 | :type role: str 68 | :param message: The user's message. 69 | :type message: str 70 | :param max_tokens: Maximum number of tokens for the model to generate. 71 | :type max_tokens: int 72 | :param extra_params: Additional parameters for the model. 73 | :type extra_params: Optional[Dict[str, Any]] 74 | :keyword kwargs: Additional keyword arguments. 75 | :type kwargs: Any 76 | :return: The model's response. 77 | :rtype: str 78 | """ 79 | if role == "user": 80 | time.sleep(self.delay) 81 | 82 | llm_out = self.get_model_output(message, options) 83 | 84 | response = llm_out["message"]["content"] 85 | 86 | self.memorize("assistant", response) 87 | 88 | return response 89 | 90 | def get_model_output( 91 | self, 92 | message: str, 93 | options: Optional[Dict[str, Any]] = None, 94 | ) -> Any: 95 | """ 96 | Get the model output for the given message. 97 | 98 | :param message: The user's message. 99 | :type message: str 100 | :param max_tokens: Maximum number of tokens for the model to generate. 101 | :type max_tokens: int 102 | :param extra_params: Additional parameters for the model. 103 | :type extra_params: Optional[Dict[str, Any]] 104 | :return: The model output. 105 | :rtype: Any 106 | """ 107 | 108 | try: 109 | if self.verbose: 110 | print("+", end="", file=sys.stderr, flush=True) 111 | 112 | client = Client(host=self.endpoint, timeout=self.timeout) 113 | 114 | llm_out = client.chat( 115 | model=self.model, messages=self.ollama_data(), options=options 116 | ) 117 | self.log.append(llm_out) 118 | 119 | return llm_out 120 | except Exception as e: 121 | self.log.append(e) 122 | 123 | raise e # re-raise 124 | -------------------------------------------------------------------------------- /alter_ego/experiment/__init__.py: -------------------------------------------------------------------------------- 1 | import json 2 | import random 3 | import uuid 4 | from alter_ego.structure import Conversation 5 | from itertools import product 6 | from typing import Any, Dict, List, Optional, Type 7 | 8 | 9 | class Experiment: 10 | """ 11 | Class for managing an Experiment, which links treatments to conversations. 12 | """ 13 | 14 | def __init__(self, *treatments: Type) -> None: 15 | """ 16 | Initialize the Experiment. 17 | 18 | :param treatments: Variable-length list of treatment classes. 19 | :type treatments: Type 20 | :raises ValueError: If less than two treatments are supplied. 21 | :raises AttributeError: If treatments have incongruent attributes. 22 | """ 23 | if len(treatments) < 2: 24 | raise ValueError("Experiment expects at least two treatments.") 25 | 26 | if any(hasattr(t, "__dict__") for t in treatments): 27 | keys = set(treatments[0].__dict__.keys()) 28 | 29 | for treatment in treatments: 30 | if set(treatment.__dict__.keys()) != keys: 31 | raise AttributeError( 32 | f"Treatment {treatment.__name__} is incongruent." 33 | ) 34 | 35 | self.id = uuid.uuid4() 36 | self.treatments = treatments 37 | self.params: Dict[str, List[str]] = {} 38 | 39 | def link(self, convo: Conversation, treatment: Optional[Type] = None) -> None: 40 | """ 41 | Associate a treatment and parameters with a Conversation object. 42 | 43 | :param convo: The conversation to which to link the treatment and parameters. 44 | :type convo: Conversation 45 | :param treatment: The treatment to apply; None for random selection. 46 | :type treatment: Optional[Type] 47 | """ 48 | if treatment is None: 49 | convo.all.treatment = random.choice(self.treatments) 50 | else: 51 | convo.all.treatment = treatment 52 | 53 | convo.all.experiment = self 54 | 55 | for param, values in self.params.items(): 56 | randomized_values = random.sample(values, len(values)) 57 | 58 | assert len(randomized_values) >= len(convo.threads) 59 | 60 | for thread, value in zip(convo.threads, randomized_values): 61 | setattr(thread, param, value) 62 | 63 | def param(self, name: str, values: List[Any]) -> None: 64 | """ 65 | Set a named parameter for the experiment. 66 | 67 | :param name: The name of the parameter to set. 68 | :type name: str 69 | :param values: A list of values to assign to the parameter. 70 | :type values: List[Any] 71 | """ 72 | self.params[name] = values 73 | 74 | def run( 75 | self, 76 | agent_factory, 77 | filter=json.loads, 78 | times=1, 79 | *, 80 | outcome="result", 81 | keep_retval=False, 82 | **kwargs, 83 | ) -> List[Dict]: 84 | data = [] 85 | 86 | if filter is None: 87 | filter = lambda x: x 88 | 89 | for _ in range(times): 90 | for treat in self.treatments: 91 | an_agent = agent_factory() 92 | retval = an_agent.user(treat.prompt, **treat.data, **kwargs) 93 | 94 | extra = {} if not keep_retval else {"retval": retval} 95 | 96 | try: 97 | from_agent = filter(retval) 98 | except Exception as e: 99 | from_agent = None 100 | 101 | if isinstance(from_agent, dict): 102 | data.append(treat.data | from_agent | extra) 103 | else: 104 | data.append(treat.data | {outcome: from_agent} | extra) 105 | 106 | return data 107 | 108 | 109 | class GenericTreatment: 110 | def __init__(self, prompt, **kwargs): 111 | self.prompt = prompt 112 | self.data = kwargs 113 | 114 | 115 | def factorial(prompt, **kwargs) -> Experiment: 116 | keys = kwargs.keys() 117 | 118 | return Experiment( 119 | *[ 120 | GenericTreatment(prompt=prompt, **dict(zip(keys, values))) 121 | for values in product(*kwargs.values()) 122 | ] 123 | ) 124 | -------------------------------------------------------------------------------- /alter_ego/agents/TextSynthThread.py: -------------------------------------------------------------------------------- 1 | from typing import Any, Dict, Optional 2 | import json 3 | import os 4 | import requests 5 | import sys 6 | import time 7 | 8 | from alter_ego.agents import APIThread 9 | 10 | 11 | class TextSynthThread(APIThread): 12 | """ 13 | Class representing a TextSynth Thread. 14 | """ 15 | 16 | def __init__(self, **kwargs: Any): 17 | """ 18 | Initialize the TextSynthThread. 19 | 20 | :keyword kwargs: Additional keyword arguments. 21 | :type kwargs: Any 22 | """ 23 | # defaults 24 | self.endpoint = "https://api.textsynth.com/v1/engines/falcon_40B-chat/chat" 25 | self.temperature = 1.0 26 | 27 | super().__init__(**kwargs) 28 | 29 | def ts_data(self) -> Dict[str, Any]: 30 | """ 31 | Prepare the data for TextSynth API call. 32 | 33 | :return: Data to be sent in the API request. 34 | :rtype: Dict[str, Any] 35 | :raises ValueError: If an invalid history item is encountered. 36 | """ 37 | system_set = False 38 | next_role = 1 39 | 40 | data = dict(messages=[]) 41 | 42 | for item in self._history: 43 | if item["role"] == "system" and not system_set: 44 | data["system"] = item["content"] 45 | system_set = True 46 | elif item["role"] == "user" and next_role == 1: 47 | data["messages"].append(item["content"]) 48 | next_role = 2 49 | elif item["role"] == "assistant" and next_role == 2: 50 | data["messages"].append(item["content"]) 51 | next_role = 1 52 | else: 53 | raise ValueError(f"The following history item is invalid: {item}") 54 | 55 | return data 56 | 57 | def send( 58 | self, 59 | role: str, 60 | message: str, 61 | max_tokens: int = 500, 62 | extra_params: Optional[Dict[str, Any]] = None, 63 | **kwargs: Any, 64 | ) -> str: 65 | """ 66 | Submit the user message, receive the model's response, and memorize it. 67 | 68 | :param role: Role of the sender ("user"). 69 | :type role: str 70 | :param message: The user's message. 71 | :type message: str 72 | :param max_tokens: Maximum number of tokens for the model to generate. 73 | :type max_tokens: int 74 | :param extra_params: Additional parameters for the model. 75 | :type extra_params: Optional[Dict[str, Any]] 76 | :keyword kwargs: Additional keyword arguments. 77 | :type kwargs: Any 78 | :return: The model's response. 79 | :rtype: str 80 | """ 81 | if role == "user": 82 | time.sleep(self.delay) 83 | 84 | llm_out = self.get_model_output(message, max_tokens, extra_params) 85 | 86 | response = llm_out["text"] 87 | 88 | self.memorize("assistant", response) 89 | 90 | return response 91 | 92 | def get_model_output( 93 | self, 94 | message: str, 95 | max_tokens: int, 96 | extra_params: Optional[Dict[str, Any]] = None, 97 | ) -> Any: 98 | """ 99 | Get the model output for the given message. 100 | 101 | :param message: The user's message. 102 | :type message: str 103 | :param max_tokens: Maximum number of tokens for the model to generate. 104 | :type max_tokens: int 105 | :param extra_params: Additional parameters for the model. 106 | :type extra_params: Optional[Dict[str, Any]] 107 | :return: The model output. 108 | :rtype: Any 109 | """ 110 | 111 | try: 112 | if self.verbose: 113 | print("+", end="", file=sys.stderr, flush=True) 114 | 115 | headers = { 116 | "Content-Type": "application/json", 117 | "Authorization": f"Bearer {self.api_key}", 118 | } 119 | 120 | params = { 121 | "max_tokens": max_tokens, 122 | "temperature": self.temperature, 123 | } | (extra_params if extra_params is not None else {}) 124 | 125 | rq = requests.post( 126 | self.endpoint, 127 | headers=headers, 128 | data=json.dumps(self.ts_data() | params), 129 | ) 130 | 131 | llm_out = rq.json() 132 | self.log.append(llm_out) 133 | 134 | return llm_out 135 | except Exception as e: 136 | self.log.append(e) 137 | 138 | raise e # re-raise 139 | -------------------------------------------------------------------------------- /scenarios/built.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | import random 4 | import sys 5 | import time 6 | import traceback 7 | 8 | import alter_ego.agents 9 | import alter_ego.experiment 10 | import alter_ego.structure 11 | import alter_ego.utils 12 | 13 | from dataclasses import dataclass 14 | 15 | 16 | with open(os.getenv("BUILT_FILE", "built.json")) as f: 17 | definition = json.load(f) 18 | 19 | 20 | @dataclass 21 | class Treatment: 22 | name: str 23 | variables: dict = None 24 | 25 | def __post_init__(self): 26 | if self.variables is None: 27 | self.variables = {} 28 | 29 | 30 | def make_treatment(definition: dict, treatment_name: str) -> Treatment: 31 | treat = Treatment(name=treatment_name) 32 | 33 | for var, in_treatment in definition["variables"].items(): 34 | treat.variables[var] = in_treatment[treatment_name] 35 | 36 | return treat 37 | 38 | 39 | def make_thread(thread_str: str): 40 | if thread_str == "CLIThread": 41 | return alter_ego.agents.CLIThread(verbose=True) 42 | elif ( 43 | thread_str == "GPTThread (GPT 3.5)" or thread_str == "GPTThread (GPT-3.5 turbo)" 44 | ): 45 | return alter_ego.agents.GPTThread( 46 | model="gpt-3.5-turbo", temperature=1, verbose=True 47 | ) 48 | elif thread_str == "GPTThread (GPT 4)" or thread_str == "GPTThread (GPT-4)": 49 | return alter_ego.agents.GPTThread(model="gpt-4", temperature=1, verbose=True) 50 | elif thread_str == "GPTThread (GPT-4 turbo)": 51 | return alter_ego.agents.GPTThread( 52 | model="gpt-4-turbo", temperature=1, verbose=True 53 | ) 54 | elif thread_str == "GPTThread (GPT-4o)": 55 | return alter_ego.agents.GPTThread(model="gpt-4o", temperature=1, verbose=True) 56 | else: 57 | raise ValueError(f"{thread_str} not found.") 58 | 59 | 60 | def make_filter(filter_def: list): 61 | if filter_def[0] == "JSON": 62 | return json.loads 63 | elif filter_def[0] == "extract_number": 64 | return alter_ego.utils.extract_number 65 | elif filter_def[0] == "exclusive_response": 66 | 67 | def new_exclusive_response(s): 68 | return alter_ego.utils.exclusive_response(s, filter_def[1].split(";")) 69 | 70 | return new_exclusive_response 71 | elif filter_def[0] == "As is": 72 | return lambda s: s 73 | 74 | 75 | def run(times: int = 1) -> None: 76 | treatments = [make_treatment(definition, t) for t in definition["treatments"]] 77 | e = alter_ego.experiment.Experiment(*treatments) 78 | 79 | for i in range(1, times + 1): 80 | threads = [] 81 | 82 | for i_, thrstr in definition["threads"]: 83 | thread = make_thread(thrstr) 84 | threads.append(thread) 85 | 86 | thread.i = i_ 87 | 88 | convo = alter_ego.structure.Conversation(*threads) 89 | 90 | e.link(convo) # randomly assign treatment 91 | 92 | convo.all.rounds = definition["rounds"] 93 | 94 | convo.all.system( 95 | alter_ego.utils.homogenize(definition["prompts"]["system"]), 96 | **convo.treatment.variables, 97 | ) 98 | 99 | try: 100 | for round_ in range(1, definition["rounds"] + 1): 101 | convo.all.round = round_ 102 | 103 | for index, thread in enumerate(threads): 104 | text_response = thread.submit( 105 | alter_ego.utils.homogenize(definition["prompts"]["user"]), 106 | **convo.treatment.variables, 107 | ) 108 | filtered = make_filter(definition["filters"][index])(text_response) 109 | 110 | thread.choices.append(filtered) 111 | 112 | if filtered is None: 113 | raise ValueError( 114 | f"Thread returned bad response: '{text_response}', not filtered with {definition['filters'][index]}" 115 | ) 116 | except Exception as exc: 117 | traceback.print_exception(*sys.exc_info()) 118 | convo.all.tainted = True 119 | 120 | convo.all.save(e.id) 121 | print(file=sys.stderr) 122 | 123 | print(f"Experiment {e.id} OK", file=sys.stderr) 124 | 125 | 126 | def export(thread: alter_ego.structure.Thread) -> list[dict]: 127 | data = [] 128 | 129 | row_template = dict( # define variables of interest here 130 | convo=thread.convo.id, 131 | thread_type=thread.__class__.__name__, 132 | tainted=thread.tainted, 133 | treatment=thread.treatment.name, 134 | i=thread.i, 135 | ) 136 | 137 | for round_, choice in enumerate(thread.choices, 1): 138 | data.append(row_template | dict(round=round_, choice=json.dumps(choice))) 139 | 140 | return data 141 | -------------------------------------------------------------------------------- /alter_ego/exports/otree.py: -------------------------------------------------------------------------------- 1 | import os 2 | from typing import Any, Dict, List, Optional, Union 3 | from alter_ego.structure import Thread, Conversation 4 | from alter_ego.utils import to_html 5 | 6 | # Constants for key formats 7 | PLAYER_KEY_FORMAT = "__alter_ego/player/{}/{}" 8 | GROUP_KEY_FORMAT = "__alter_ego/group/{}/{}" 9 | ROUND_KEY_FORMAT = "__alter_ego/round/{}/{}" 10 | PARTICIPANT_KEY_FORMAT = "__alter_ego/participant" 11 | SESSION_KEY_FORMAT = "__alter_ego/session" 12 | 13 | OUTPATH = ".ego_output/" 14 | NO_SAVE = False 15 | FULL_SAVE = True 16 | 17 | 18 | def html_history(thread: Thread) -> None: 19 | """Convert thread history to HTML format.""" 20 | thread.html_history = [ 21 | {role: to_html(message) for role, message in record.items()} 22 | for record in thread._history 23 | ] 24 | 25 | 26 | def save(thread: Thread) -> None: 27 | """Save the thread data.""" 28 | if not NO_SAVE: 29 | outdir = OUTPATH + thread.metadata["otree_session"] 30 | 31 | # save json backup 32 | thread.save(outdir=outdir, full_save=False) 33 | 34 | if FULL_SAVE: 35 | # also save pickle 36 | thread.save(outdir=outdir, full_save=True) 37 | 38 | 39 | def add_hooks(thread: Thread) -> None: 40 | """Add history hooks to a thread.""" 41 | thread.history_hooks.add(html_history) 42 | thread.history_hooks.add(save) 43 | 44 | 45 | class Assigner: 46 | """ 47 | A class for attribute assignment operations. 48 | """ 49 | 50 | def __init__( 51 | self, assignees: List[Any], key: str, metadata: Optional[Dict[str, Any]] = None 52 | ) -> None: 53 | """ 54 | Initialize an instance of Assigner. 55 | 56 | :param assignees: A list of objects to be assigned. 57 | :param key: The key used for assigning. 58 | """ 59 | self.assignees = assignees 60 | self.key = key 61 | self.metadata = metadata if metadata is not None else dict() 62 | 63 | def __bool__(self) -> bool: 64 | """Check if the key exists in the first assignee's vars dict.""" 65 | return self.key in self.assignees[0].vars 66 | 67 | def __enter__(self) -> Any: 68 | self.value = self.assignees[0].vars[self.key] 69 | 70 | return self.value 71 | 72 | def __exit__(self, *args: Any) -> bool: 73 | self.set(self.value) 74 | 75 | return False 76 | 77 | def set(self, value: Union[Thread, Conversation]) -> None: 78 | """Assign value to the 'vars' dictionary of all assignees.""" 79 | if isinstance(value, Conversation): 80 | for thread in value: 81 | thread.metadata |= self.metadata 82 | add_hooks(thread) 83 | elif isinstance(value, Thread): 84 | value.metadata |= self.metadata 85 | add_hooks(value) 86 | else: 87 | raise NotImplementedError( 88 | "Only a Thread or a Conversation can be assigned." 89 | ) 90 | 91 | for assignee in self.assignees: 92 | assignee.vars[self.key] = value 93 | 94 | def unset(self) -> None: 95 | """Unset the key in all assignees.""" 96 | for assignee in self.assignees: 97 | if self.key in assignee.vars: 98 | del assignee.vars[self.key] 99 | 100 | 101 | def link(instance: Any, suffix: Optional[str] = None) -> Assigner: 102 | """ 103 | Create an Assigner object linked to the given instance. 104 | 105 | :param instance: The instance to be linked. 106 | :returns: An Assigner object. 107 | :raises NotImplementedError: If instance type is unsupported. 108 | """ 109 | metadata = dict(otree_session=instance.session.code) 110 | super_names = [s.__name__ for s in instance.__class__.mro()] 111 | 112 | if "BasePlayer" in super_names: 113 | targets = [instance.participant] 114 | metadata["participant"] = instance.participant.id_in_session 115 | key_in_vars = PLAYER_KEY_FORMAT.format( 116 | instance.__module__, instance.round_number 117 | ) 118 | elif "BaseGroup" in super_names: 119 | targets = [p.participant for p in instance.get_players()] 120 | metadata["group"] = instance.id_in_subsession 121 | key_in_vars = GROUP_KEY_FORMAT.format( 122 | instance.__module__, instance.round_number 123 | ) 124 | elif "BaseSubsession" in super_names: 125 | targets = [p.participant for p in instance.get_players()] 126 | metadata["subsession"] = instance.round_number 127 | key_in_vars = ROUND_KEY_FORMAT.format( 128 | instance.__module__, instance.round_number 129 | ) 130 | elif "Participant" in super_names: 131 | targets = [instance] 132 | metadata["participant"] = instance.id_in_session 133 | key_in_vars = PARTICIPANT_KEY_FORMAT 134 | elif "Session" in super_names: 135 | targets = [instance] 136 | metadata["session"] = instance.code 137 | key_in_vars = SESSION_KEY_FORMAT 138 | else: 139 | raise NotImplementedError( 140 | f"Instance of type {type(instance).__name__} not recognized. " 141 | "Target can only be a Player, Participant, Group, Subsession, or Session." 142 | ) 143 | 144 | if suffix is not None: 145 | key_in_vars += f":{suffix}" 146 | 147 | return Assigner(targets, key_in_vars, metadata) 148 | -------------------------------------------------------------------------------- /games/pd/built.json: -------------------------------------------------------------------------------- 1 | {"threads":[[1,"CLIThread"],[2,"CLIThread"]],"treatments":["base","enemy","competition"],"rounds":10,"variables":{"both_cooperate":{"base":20,"enemy":20,"competition":20},"temptation":{"base":28,"enemy":28,"competition":28},"sucker":{"base":0,"enemy":0,"competition":0},"both_defect":{"base":8,"enemy":8,"competition":8},"p1name":{"base":"John","enemy":"John","competition":"John"},"p2name":{"base":"Michael","enemy":"Michael","competition":"Michael"}},"prompts":{"system":"{% if i == 1 %}\n {% set name = p1name %}\n {% set other_name = p2name %}\n{% else %}\n {% set name = p2name %}\n {% set other_name = p1name %}\n{% endif %}\n{% if name == \"base\" %}\n You are {{name}}.\n\n You are participating in an experiment. In this experiment, you are together with another participant called {{other_name}}. {{other_name}} is another artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other_name}}.\n\n Each participant has two options:\n\n OPTION 1: Gives {{both_cooperate}} cents to {{other_name}}.\n\n OPTION 2: Gives {{both_defect}} cents to yourself.\n\n If you both choose OPTION 1, you each earn {{both_cooperate}} cents and are better off than if you both choose OPTION 2 (in which case you both earn {{both_defect}} cents). But if you choose OPTION 2 while {{other_name}} chooses OPTION 1, you earn {{temptation}} cents while {{other_name}} earns {{sucker}}. How many cents do you give to {{other_name}}?\n\n The experiment will run for 10 rounds.\n\n Always exclusively respond with \"OPTION 1\" or \"OPTION 2\". Do not repeat the question. Do not give any explanation.\n{% elif name == \"enemy\" %}\n You are {{name}}.\n\n You are participating in an experiment. In this experiment, you and another participant called {{other_name}} have a joint enemy. {{other_name}} is another artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other_name}}.\n\n Each participant has two options:\n\n OPTION 1: Gives {{both_cooperate}} cents to {{other_name}}.\n\n OPTION 2: Gives {{both_defect}} cents to yourself.\n\n If you both choose OPTION 1, you successfully deter the enemy and you each earn {{both_cooperate}} cents. If you both choose OPTION 2, the enemy prevails and you both earn {{both_defect}} cents. But if you choose OPTION 2 while {{other_name}} chooses OPTION 1, {{other_name}} has helped deter the enemy while you have not. You benefit from {{other_name}} having made an effort, and do not have to bear the cost of effort. This is why you earn {{temptation}} cents. As the protection is only partial, {{other_name}} has to fully pay, but is only partly protected; {{other_name}} then only earns {{sucker}}. How many cents do you give to {{other_name}}?\n\n The experiment will run for 10 rounds.\n\n Always exclusively respond with \"OPTION 1\" or \"OPTION 2\". Do not repeat the question. Do not give any explanation.\n{% else %}\n You are {{name}}.\n\n You are participating in an experiment. In this experiment, you are competing against another participant called {{other_name}} in a market. {{other_name}} is another artificial intelligence. How much money you earn depends on your own choice, and on the choice of {{other_name}}.\n\n Each participant has two options:\n\n OPTION 1: Gives {{both_cooperate}} cents to {{other_name}}.\n\n OPTION 2: Gives {{both_defect}} cents to yourself.\n\n If you both choose OPTION 1, you successfully exploit customers, and share the monopoly profit and each earn {{both_cooperate}} cents. If you both choose OPTION 2, you both engage in competition, and only redeem your production cost; you both earn {{both_defect}} cents. But if you choose OPTION 2 while {{other_name}} chooses OPTION 1, you undercut the price of {{other_name}} and serve all customers; you earn {{temptation}} cents while {{other_name}} sells nothing and only earns the minimal payoff of {{sucker}}. Which option do you choose?\n\n The experiment will run for 10 rounds.\n\n Always exclusively respond with \"OPTION 1\" or \"OPTION 2\". Do not repeat the question. Do not give any explanation.\n{% endif %}","user":"{% if i == 1 %}\n {% set other_name = p2name %}\n{% else %}\n {% set other_name = p1name %}\n{% endif %}\n{% if round > 1 %}\n In round {{round - 1}}, you played OPTION {{choices[-1]}}.\n\n {% if i == 1 %}\n {{other_name}} played OPTION {{other.choices[-1]}}.\n {% else %}\n {{other_name}} played OPTION {{other.choices[-1]}}.\n {% endif %}\n\n {% if choices[-1] == 1 and other.choices[-1] == 1 %}\n {% set payoff = both_cooperate %}\n {% elif choices[-1] == 1 and other.choices[-1] == 2 %}\n {% set payoff = sucker %}\n {% elif choices[-1] == 2 and other.choices[-1] == 1 %}\n {% set payoff = temptation %}\n {% else %}\n {% set payoff = both_defect %}\n {% endif %}\n\n Your payoff in round {{round - 1}} was {{payoff}}.\n{% endif %}\n\nThis is round {{round}}.\n\n{% if i == 1 %}\n {% if treatment.name == \"base\" %}\n You are still playing with {{other_name}}.\n {% elif treatment.name == \"enemy\" %}\n You and {{other_name}} are still having a common enemy.\n {% else %}\n You are still competing against {{other_name}} in a market.\n {% endif %}\n{% else %}\n In this round, participant {{other_name}} just played OPTION {{other.choices[-1]}}.\n\n {% if treatment.name == \"base\" %}\n You are still playing with {{other_name}}.\n {% elif treatment.name == \"enemy\" %}\n You and {{other_name}} are still having a common enemy.\n {% else %}\n You are still competing against {{other_name}} in a market.\n {% endif %}\n{% endif %}\n\nWhat is your choice? Respond with OPTION 1 or OPTION 2."},"filters":[["extract_number"],["extract_number"]]} 2 | -------------------------------------------------------------------------------- /otree/ego_human/__init__.py: -------------------------------------------------------------------------------- 1 | from alter_ego.agents import * 2 | from alter_ego.structure import Conversation 3 | from alter_ego.utils import extract_number, from_file 4 | from alter_ego.exports.otree import link as ai 5 | 6 | from otree.api import * 7 | 8 | import random 9 | 10 | 11 | doc = """ 12 | EGO, with a human second mover 13 | """ 14 | 15 | 16 | class C(BaseConstants): 17 | NAME_IN_URL = "ego_human" 18 | PLAYERS_PER_GROUP = None 19 | NUM_ROUNDS = 10 20 | 21 | 22 | class Treatment: 23 | both_cooperate = 20 24 | both_defect = 8 25 | sucker = 0 26 | temptation = 28 27 | 28 | 29 | class Subsession(BaseSubsession): 30 | pass 31 | 32 | 33 | def creating_session(subsession): 34 | for player in subsession.get_players(): 35 | if player.round_number == 1: 36 | player.participant.vars["dropout"] = False 37 | player.participant.vars["clerpay_amount"] = 1.00 # for dropouts 38 | 39 | player.treatment = random.choice([1, 3, 4]) 40 | 41 | llm = GPTThread( 42 | model="gpt-4", 43 | temperature=1.0, 44 | name="AI", 45 | ) 46 | human = ExternalThread(name="Human") 47 | 48 | convo = Conversation(llm=llm, human=human) 49 | convo.all.treatment = Treatment 50 | convo.all.num_rounds = C.NUM_ROUNDS 51 | 52 | llm.system( 53 | from_file( 54 | f"ego_human/prompts/prompts00{player.treatment}/pd_seq/system_c.txt" 55 | ) 56 | ) 57 | human.system( 58 | from_file( 59 | f"ego_human/prompts/prompts00{player.treatment}/pd_seq/system_h.txt" 60 | ) 61 | ) 62 | 63 | ai(player.participant).set(convo) 64 | player.llm_id = str(llm.id) 65 | else: 66 | player.treatment = player.in_round(1).treatment 67 | 68 | 69 | class Group(BaseGroup): 70 | pass 71 | 72 | 73 | class Player(BasePlayer): 74 | llm_id = models.StringField() 75 | treatment = models.IntegerField() 76 | 77 | choice = models.IntegerField( 78 | initial=-1, choices=[[1, "OPTION 1"], [2, "OPTION 2"]], label="Your Choice" 79 | ) 80 | 81 | other_choice = models.IntegerField() 82 | 83 | 84 | def invoke_first_mover(player): 85 | with ai(player.participant) as convo: 86 | response = extract_number( 87 | convo.llm.submit( 88 | from_file( 89 | f"ego_human/prompts/prompts00{player.treatment}/pd_seq/first_mover.txt" 90 | ), 91 | player=player, 92 | ), 93 | ) 94 | 95 | player.other_choice = convo.llm.last = response 96 | 97 | convo.human.submit( 98 | from_file( 99 | f"ego_human/prompts/prompts00{player.treatment}/pd_seq/second_mover.txt" 100 | ), 101 | player=player, 102 | ) 103 | 104 | 105 | # PAGES 106 | class Start(Page): 107 | timeout_seconds = 450 108 | 109 | @staticmethod 110 | def is_displayed(player): 111 | return player.round_number == 1 112 | 113 | @staticmethod 114 | def before_next_page(player, timeout_happened): 115 | if timeout_happened: 116 | player.participant.vars["dropout"] = True 117 | 118 | return 119 | 120 | 121 | class SecondMover(Page): 122 | timeout_seconds = 300 123 | 124 | form_fields = ["choice"] 125 | form_model = "player" 126 | 127 | @staticmethod 128 | def vars_for_template(player): 129 | # should be "before_this_page"… 130 | 131 | if player.field_maybe_none("other_choice") is None: 132 | invoke_first_mover(player) 133 | 134 | @staticmethod 135 | def js_vars(player): 136 | with ai(player.participant) as convo: 137 | return dict(history=convo.human.history) 138 | 139 | @staticmethod 140 | def app_after_this_page(player, upcoming_apps): 141 | if player.participant.vars["dropout"]: 142 | return "clerpay_end" 143 | 144 | @staticmethod 145 | def before_next_page(player, timeout_happened): 146 | if timeout_happened: 147 | player.participant.vars["dropout"] = True 148 | 149 | return 150 | 151 | with ai(player.participant) as convo: 152 | convo.human.last = player.choice 153 | 154 | # Determine payoffs 155 | 156 | if player.other_choice == 1 and player.choice == 1: 157 | convo.llm.payoff = Treatment.both_cooperate 158 | player.payoff = Treatment.both_cooperate 159 | elif player.other_choice == 1 and player.choice == 2: 160 | convo.llm.payoff = Treatment.sucker 161 | player.payoff = Treatment.temptation 162 | elif player.other_choice == 2 and player.choice == 1: 163 | convo.llm.payoff = Treatment.temptation 164 | player.payoff = Treatment.sucker 165 | elif player.other_choice == 2 and player.choice == 2: 166 | convo.llm.payoff = Treatment.both_defect 167 | player.payoff = Treatment.both_defect 168 | 169 | convo.human.submit( 170 | from_file( 171 | f"ego_human/prompts/prompts00{player.treatment}/pd_seq/result.txt" 172 | ), 173 | player=player, 174 | ) # only relevant for final round! 175 | 176 | 177 | class Result(Page): 178 | timeout_seconds = 300 179 | 180 | @staticmethod 181 | def is_displayed(player): 182 | return ( 183 | player.round_number == C.NUM_ROUNDS 184 | and not player.participant.vars["dropout"] 185 | ) 186 | 187 | @staticmethod 188 | def js_vars(player): 189 | with ai(player.participant) as convo: 190 | return dict(history=convo.human.history) 191 | 192 | 193 | class End(Page): 194 | timeout_seconds = 300 195 | 196 | @staticmethod 197 | def is_displayed(player): 198 | if not player.participant.vars["dropout"]: 199 | player.participant.vars["clerpay_amount"] = float( 200 | player.participant.payoff_plus_participation_fee() 201 | ) 202 | 203 | return player.round_number == C.NUM_ROUNDS 204 | else: 205 | return False 206 | 207 | 208 | page_sequence = [Start, SecondMover, Result, End] 209 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU LESSER GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | 9 | This version of the GNU Lesser General Public License incorporates 10 | the terms and conditions of version 3 of the GNU General Public 11 | License, supplemented by the additional permissions listed below. 12 | 13 | 0. Additional Definitions. 14 | 15 | As used herein, "this License" refers to version 3 of the GNU Lesser 16 | General Public License, and the "GNU GPL" refers to version 3 of the GNU 17 | General Public License. 18 | 19 | "The Library" refers to a covered work governed by this License, 20 | other than an Application or a Combined Work as defined below. 21 | 22 | An "Application" is any work that makes use of an interface provided 23 | by the Library, but which is not otherwise based on the Library. 24 | Defining a subclass of a class defined by the Library is deemed a mode 25 | of using an interface provided by the Library. 26 | 27 | A "Combined Work" is a work produced by combining or linking an 28 | Application with the Library. The particular version of the Library 29 | with which the Combined Work was made is also called the "Linked 30 | Version". 31 | 32 | The "Minimal Corresponding Source" for a Combined Work means the 33 | Corresponding Source for the Combined Work, excluding any source code 34 | for portions of the Combined Work that, considered in isolation, are 35 | based on the Application, and not on the Linked Version. 36 | 37 | The "Corresponding Application Code" for a Combined Work means the 38 | object code and/or source code for the Application, including any data 39 | and utility programs needed for reproducing the Combined Work from the 40 | Application, but excluding the System Libraries of the Combined Work. 41 | 42 | 1. Exception to Section 3 of the GNU GPL. 43 | 44 | You may convey a covered work under sections 3 and 4 of this License 45 | without being bound by section 3 of the GNU GPL. 46 | 47 | 2. Conveying Modified Versions. 48 | 49 | If you modify a copy of the Library, and, in your modifications, a 50 | facility refers to a function or data to be supplied by an Application 51 | that uses the facility (other than as an argument passed when the 52 | facility is invoked), then you may convey a copy of the modified 53 | version: 54 | 55 | a) under this License, provided that you make a good faith effort to 56 | ensure that, in the event an Application does not supply the 57 | function or data, the facility still operates, and performs 58 | whatever part of its purpose remains meaningful, or 59 | 60 | b) under the GNU GPL, with none of the additional permissions of 61 | this License applicable to that copy. 62 | 63 | 3. Object Code Incorporating Material from Library Header Files. 64 | 65 | The object code form of an Application may incorporate material from 66 | a header file that is part of the Library. You may convey such object 67 | code under terms of your choice, provided that, if the incorporated 68 | material is not limited to numerical parameters, data structure 69 | layouts and accessors, or small macros, inline functions and templates 70 | (ten or fewer lines in length), you do both of the following: 71 | 72 | a) Give prominent notice with each copy of the object code that the 73 | Library is used in it and that the Library and its use are 74 | covered by this License. 75 | 76 | b) Accompany the object code with a copy of the GNU GPL and this license 77 | document. 78 | 79 | 4. Combined Works. 80 | 81 | You may convey a Combined Work under terms of your choice that, 82 | taken together, effectively do not restrict modification of the 83 | portions of the Library contained in the Combined Work and reverse 84 | engineering for debugging such modifications, if you also do each of 85 | the following: 86 | 87 | a) Give prominent notice with each copy of the Combined Work that 88 | the Library is used in it and that the Library and its use are 89 | covered by this License. 90 | 91 | b) Accompany the Combined Work with a copy of the GNU GPL and this license 92 | document. 93 | 94 | c) For a Combined Work that displays copyright notices during 95 | execution, include the copyright notice for the Library among 96 | these notices, as well as a reference directing the user to the 97 | copies of the GNU GPL and this license document. 98 | 99 | d) Do one of the following: 100 | 101 | 0) Convey the Minimal Corresponding Source under the terms of this 102 | License, and the Corresponding Application Code in a form 103 | suitable for, and under terms that permit, the user to 104 | recombine or relink the Application with a modified version of 105 | the Linked Version to produce a modified Combined Work, in the 106 | manner specified by section 6 of the GNU GPL for conveying 107 | Corresponding Source. 108 | 109 | 1) Use a suitable shared library mechanism for linking with the 110 | Library. A suitable mechanism is one that (a) uses at run time 111 | a copy of the Library already present on the user's computer 112 | system, and (b) will operate properly with a modified version 113 | of the Library that is interface-compatible with the Linked 114 | Version. 115 | 116 | e) Provide Installation Information, but only if you would otherwise 117 | be required to provide such information under section 6 of the 118 | GNU GPL, and only to the extent that such information is 119 | necessary to install and execute a modified version of the 120 | Combined Work produced by recombining or relinking the 121 | Application with a modified version of the Linked Version. (If 122 | you use option 4d0, the Installation Information must accompany 123 | the Minimal Corresponding Source and Corresponding Application 124 | Code. If you use option 4d1, you must provide the Installation 125 | Information in the manner specified by section 6 of the GNU GPL 126 | for conveying Corresponding Source.) 127 | 128 | 5. Combined Libraries. 129 | 130 | You may place library facilities that are a work based on the 131 | Library side by side in a single library together with other library 132 | facilities that are not Applications and are not covered by this 133 | License, and convey such a combined library under terms of your 134 | choice, if you do both of the following: 135 | 136 | a) Accompany the combined library with a copy of the same work based 137 | on the Library, uncombined with any other library facilities, 138 | conveyed under the terms of this License. 139 | 140 | b) Give prominent notice with the combined library that part of it 141 | is a work based on the Library, and explaining where to find the 142 | accompanying uncombined form of the same work. 143 | 144 | 6. Revised Versions of the GNU Lesser General Public License. 145 | 146 | The Free Software Foundation may publish revised and/or new versions 147 | of the GNU Lesser General Public License from time to time. Such new 148 | versions will be similar in spirit to the present version, but may 149 | differ in detail to address new problems or concerns. 150 | 151 | Each version is given a distinguishing version number. If the 152 | Library as you received it specifies that a certain numbered version 153 | of the GNU Lesser General Public License "or any later version" 154 | applies to it, you have the option of following the terms and 155 | conditions either of that published version or of any later version 156 | published by the Free Software Foundation. If the Library as you 157 | received it does not specify a version number of the GNU Lesser 158 | General Public License, you may choose any version of the GNU Lesser 159 | General Public License ever published by the Free Software Foundation. 160 | 161 | If the Library as you received it specifies that a proxy can decide 162 | whether future versions of the GNU Lesser General Public License shall 163 | apply, that proxy's public statement of acceptance of any version is 164 | permanent authorization for you to choose that version for the 165 | Library. 166 | -------------------------------------------------------------------------------- /alter_ego/structure/__init__.py: -------------------------------------------------------------------------------- 1 | from abc import ABC, abstractmethod 2 | from typing import Callable, Dict, Iterator, List, Set, Tuple, Union, Any 3 | from jinja2 import Environment, StrictUndefined 4 | from alter_ego.structure.Relay import Relay 5 | import copy 6 | import uuid 7 | import os 8 | import pickle 9 | import json 10 | 11 | VALID_ROLES = ["system", "user", "assistant"] 12 | 13 | 14 | class Thread(ABC): 15 | """ 16 | Abstract base class representing a Thread. 17 | """ 18 | 19 | def __init__(self, **params: Any) -> None: 20 | """ 21 | Initialize a Thread instance. 22 | 23 | :param params: Arbitrary keyword parameters. 24 | """ 25 | # Initialize instance variables 26 | self.__dict__ |= params 27 | self.id: uuid.UUID = uuid.uuid4() 28 | self.metadata: Dict[str, Any] = {} 29 | self._history: List[Dict[str, str]] = [] 30 | self.tainted: bool = False 31 | self.convo = None # Will be assigned later 32 | self.history_hooks: Set[Callable] = set() 33 | self.choices: List[Any] = [] 34 | self.env: Environment = Environment(undefined=StrictUndefined) 35 | 36 | def __repr__(self) -> str: 37 | """ 38 | :return: String representation of the Thread. 39 | """ 40 | return f"<{self.__class__.__name__}/{self.name if 'name' in self.__dict__ else str(self.id)[0:8]}>" 41 | 42 | @property 43 | def history(self) -> List[Dict[str, str]]: 44 | """ 45 | :return: Deep copy of message history. 46 | """ 47 | return copy.deepcopy(self._history) 48 | 49 | def memorize(self, role: str, message: str) -> None: 50 | """ 51 | Memorizes a message and its associated role. 52 | 53 | :param role: Role of the message ('system', 'user', or 'assistant'). 54 | :param message: Message content. 55 | :raises ValueError: If the role is invalid or if a system message already exists. 56 | """ 57 | if role not in VALID_ROLES: 58 | raise ValueError(f'Invalid role "{role}".') 59 | 60 | if role == "system" and any(item["role"] == "system" for item in self.history): 61 | raise ValueError(f"System message already set.") 62 | 63 | self._history.append({"role": role, "content": message}) 64 | for f in self.history_hooks: 65 | f(self) 66 | 67 | def prepare(self, template: str, **extra: Any) -> str: 68 | """ 69 | Prepare a template with additional parameters. 70 | 71 | :param template: Template string. 72 | :param extra: Additional parameters to inject into the template. 73 | :return: Rendered template string. 74 | """ 75 | template = self.env.from_string(template) 76 | return template.render(**extra, **self.__dict__) 77 | 78 | def save( 79 | self, subdir: str = ".", outdir: str = "out", full_save: bool = True 80 | ) -> None: 81 | """ 82 | Saves the current Thread into a file. 83 | 84 | :param subdir: The sub-directory to save the file in. 85 | :param outdir: The main directory to save the file in. 86 | :param full_save: Whether to save as pickle (True) or JSON (False). 87 | :raises ValueError: If the Thread is not part of a Conversation. 88 | """ 89 | if self.convo is None: 90 | target_dir = f"{outdir}/{subdir}" 91 | else: 92 | target_dir = f"{outdir}/{subdir}/{self.convo.id}" 93 | 94 | os.makedirs(target_dir, exist_ok=True) 95 | 96 | outfile = ( 97 | f"{target_dir}/{self.id}.pkl" 98 | if full_save 99 | else f"{target_dir}/{self.id}.json" 100 | ) 101 | mode = "wb" if full_save else "w" 102 | 103 | with open(outfile, mode) as fp: 104 | if full_save: 105 | pickle.dump(self, fp) 106 | else: 107 | json.dump(dict(history=self.history, metadata=self.metadata), fp) 108 | 109 | def cost(self) -> float: 110 | """ 111 | Computes and returns the cost associated with the Thread. 112 | 113 | :returns: The cost, 0.0 for this base implementation. Adjust in subclasses. 114 | """ 115 | raise NotImplementedError 116 | 117 | def system(self, message: str, **kwargs: Any) -> Any: 118 | """ 119 | Sends a system-level message. 120 | 121 | :param message: The message to send. 122 | :param kwargs: Additional keyword arguments for message preparation. 123 | :returns: Return value from the send method. 124 | """ 125 | self.memorize("system", m := self.prepare(message, **kwargs)) 126 | 127 | retval = self.send("system", m, **kwargs) 128 | 129 | return retval 130 | 131 | def user(self, message: str, **kwargs: Any) -> Any: 132 | """ 133 | Sends a user-level message. 134 | 135 | :param message: The message to send. 136 | :param kwargs: Additional keyword arguments for message preparation. 137 | :returns: Return value from the send method. 138 | """ 139 | self.memorize("user", m := self.prepare(message, **kwargs)) 140 | 141 | retval = self.send("user", m, **kwargs) 142 | 143 | return retval 144 | 145 | def assistant(self, message: str, **kwargs: Any) -> Any: 146 | """ 147 | Sends an assistant-level message. 148 | 149 | :param message: The message to send. 150 | :param kwargs: Additional keyword arguments for message preparation. 151 | :returns: Return value from the send method. 152 | """ 153 | self.memorize("assistant", m := self.prepare(message, **kwargs)) 154 | 155 | retval = self.send("assistant", m, **kwargs) 156 | 157 | return retval 158 | 159 | def submit(self, message: str, **kwargs: Any) -> Any: 160 | """ 161 | Submits a message as a user and sends it after preparation. 162 | 163 | :param message: The message to submit. 164 | :param kwargs: Additional keyword arguments for message preparation. 165 | :returns: Return value from the send method. 166 | """ 167 | self.memorize("user", m := self.prepare(message, **kwargs)) 168 | 169 | retval = self.send("user", m, **kwargs) 170 | 171 | return retval 172 | 173 | @abstractmethod 174 | def send(self, role: str, message: str, **kwargs: Any) -> Any: 175 | """ 176 | Abstract method that must be implemented by subclasses to send messages. 177 | 178 | :param role: Role of the sender, can be 'system', 'user', or 'assistant'. 179 | :param message: The message to be sent. 180 | :param kwargs: Additional keyword arguments. 181 | :returns: Implementation dependent. 182 | """ 183 | pass 184 | 185 | 186 | class Conversation: 187 | """ 188 | Class encapsulating a Conversation consisting of multiple Threads. 189 | """ 190 | 191 | def __init__(self, *threads: Thread, **named_threads: Thread) -> None: 192 | """ 193 | Initialize a Conversation object. 194 | 195 | :param threads: Thread instances as positional arguments. 196 | :param named_threads: Thread instances as named arguments. 197 | :raises ValueError: If both 'threads' and 'named_threads' are used, or none of them are used. 198 | """ 199 | if (len(threads) > 0 and len(named_threads) > 0) or ( 200 | len(threads) == 0 and len(named_threads) == 0 201 | ): 202 | raise ValueError( 203 | "You may only use 'threads' or 'named_threads', not both, not neither." 204 | ) 205 | 206 | if len(threads) > 0: 207 | self.threads = tuple(threads) 208 | else: 209 | self.threads = tuple(named_threads.values()) 210 | self.__dict__ |= named_threads 211 | 212 | self.now = 1 213 | self.all = Relay(self, self.threads) 214 | self.id = uuid.uuid4() 215 | 216 | for thread in self.threads: 217 | thread.convo = self 218 | 219 | if len(self.threads) == 2: 220 | thread.other = ( 221 | self.threads[1] if thread == self.threads[0] else self.threads[0] 222 | ) 223 | elif len(self.threads) > 2: 224 | thread.others = [t for t in self.threads if t != thread] 225 | 226 | def __iter__(self) -> Iterator[Thread]: 227 | """ 228 | Create an iterator for traversing through the Threads in this Conversation. 229 | 230 | :yields: Each Thread in the Conversation. 231 | """ 232 | for j, thread in enumerate(self.threads, 1): 233 | self.now = j 234 | yield thread 235 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # alter\_ego 2 | 3 |

4 | Code style: black 5 |

6 | 7 | `alter_ego` is a library that allows you to run experiments with LLMs. `ego` is a command-line helper included with this package. 8 | 9 | `alter_ego` allows you to run microexperiments using a simple shorthand. You can also create more advanced experiments. For turn-based interactive experiments, a [builder](https://ego.mg.sb/builder/) is available. 10 | 11 | Read our [paper](https://q.mg.sb/ego). 12 | 13 | You can browse our source code in this repository. [Here](https://ego.mg.sb/docs/) are autogenerated docs that can make it easier to find what you're looking for. 14 | 15 | # Getting started 16 | 17 | ## Prerequisites 18 | 19 | 1. Install [Python](https://python.org), at least version 3.8. *If you are on Windows, make sure to install Python into the PATH.* 20 | 2. Create a *virtual environment* and activate it. On Linux and macOS, this is very simple. Just open a terminal and execute the following commands: 21 | 22 | ```console 23 | user@host:~$ python -m venv env 24 | user@host:~$ source env/bin/activate 25 | ``` 26 | 27 | *Note*: In this document, you may have to replace `python` by `python3` and `pip` by `pip3`. This depends on your system's settings. 28 | 29 | On Windows, consider using [this tutorial](https://realpython.com/python-virtual-environments-a-primer/#create-it) to create and activate a virtual environment. 30 | 31 | Some editors also do this for you. 32 | 33 | 3. Install `alter_ego` using 34 | 35 | ```console 36 | (env) user@host:~$ pip install -U alter_ego_llm 37 | ``` 38 | 39 | Note how `(env)` signals that we are in the virtual environment created earlier. 40 | 41 | **For the remainder of this document, we assume that your editor's current directory is also your terminal's present working directory.** From within your terminal, you can find out the present working directory using `pwd` — this should show the very same directory as opened in your editor. 42 | 43 | ## Using `alter_ego` with GPT 44 | 45 | *Note*: If you do not want to use GPT for now, simply change `GPTThread` to `CLIThread` in the examples below and skip this section. 46 | 47 | 1. **Obtain an API key from OpenAI.** [Here is more information.](https://help.openai.com/en/articles/4936850-where-do-i-find-my-api-key) Your API key looks as follows: `sk-***` Copy this to your clipboard. 48 | 1. Create a new file in your editor. 49 | 1. Put the content of your clipboard into the file `openai_key` in **your current directory**. The file must not have a file extension—it is literally just called `openai_key`. 50 | 51 | ## Developing a simple microexperiment 52 | 53 | *New*: [**📺 WATCH VIDEO TUTORIAL**](https://youtu.be/GPc0a-Fg1bY) 54 | 55 | Let's create a minimal experiment using `alter_ego`'s shorthand feature. 56 | 57 | 1. Create a new file in your editor, `first_experiment.py`. Here's its code: 58 | 59 | ```python 60 | import alter_ego.agents 61 | from alter_ego.utils import extract_number 62 | from alter_ego.experiment import factorial 63 | 64 | def agent(): 65 | return alter_ego.agents.GPTThread(model="gpt-3.5-turbo", temperature=1.0) 66 | 67 | prompt = "Estimate the public approval rating of {{politician}} during the {{time}} of their presidency. Only return a single percentage from 0 to 100." 68 | 69 | data = factorial( 70 | prompt, 71 | politician=["George W. Bush", "Barack Obama"], 72 | time=["1st year", "8th year"] 73 | ).run(agent, extract_number, times=1) 74 | 75 | for row in data: 76 | print(row) 77 | ``` 78 | 79 | Note how we use variables within the prompt. The crucial feature of `alter_ego` is how these variables are automatically replaced based on treatment. 80 | 81 | 2. In the terminal, run 82 | 83 | ```console 84 | (env) user@host:~$ python first_experiment.py 85 | ``` 86 | 87 | 3. This will take a few seconds and give you output similar to this: 88 | 89 | ``` 90 | {'politician': 'George W. Bush', 'time': '1st year', 'result': None} 91 | {'politician': 'George W. Bush', 'time': '8th year', 'result': None} 92 | {'politician': 'Barack Obama', 'time': '1st year', 'result': 63} 93 | {'politician': 'Barack Obama', 'time': '8th year', 'result': 8} 94 | ``` 95 | 96 | As you see, GPT did not give a valid response for George W. Bush. Let's debug by changing the line with `run` to: 97 | 98 | ```python 99 | ).run(agent, extract_number, times=1, keep_retval=True) 100 | ``` 101 | 102 | Rerunning our script gives: 103 | 104 | ```python 105 | {'politician': 'George W. Bush', 'time': '1st year', 'result': 62, 'retval': 'Approximately 62%.'} 106 | {'politician': 'George W. Bush', 'time': '8th year', 'result': None, 'retval': "It is difficult to provide an accurate estimate without conducting a specific poll or analysis. However, based on historical data and trends, it is common for a president's approval rating to decline over the course of their second term. Taking into account various factors such as the economic recession and the ongoing Iraq War during George W. Bush's final year in office (2008), it is reasonable to estimate his public approval rating to be around 25-35%. Please note that this estimation is subjective and might not perfectly reflect the actual public sentiment at that time."} 107 | ``` 108 | 109 | (Here, I have shown only two rows of the output.) 110 | 111 | As you see, in the cases where GPT returned only a single number, `alter_ego` was able to correctly extract it. Unfortunately, GPT 3.5 tends to refuse requests to just return a single number. GPT 4 works better. If you have access to GPT 4 over the API, you can change `model="gpt-3.5-turbo"` to `model="gpt-4"`. This is the resulting output (where I have omitted the `retval` once again): 112 | 113 | ```python 114 | {'politician': 'George W. Bush', 'time': '1st year', 'result': 57} 115 | {'politician': 'George W. Bush', 'time': '8th year', 'result': 34} 116 | {'politician': 'Barack Obama', 'time': '1st year', 'result': 57} 117 | {'politician': 'Barack Obama', 'time': '8th year', 'result': 55} 118 | ``` 119 | 120 | [Here](https://ego.mg.sb/docs/source/alter_ego.experiment.html#alter_ego.experiment.Experiment.run) you can view the documentation for `run`. `run` allows you to quickly execute an experiment defined by what highfalutin scientists call a “factorial design.” This is because the possibilities of `politician` (George W. Bush, Barack Obama) were “multiplied” by the possibilities for `time` (1st year, 8th year). 121 | 122 | The nice thing about these microexperiments is that you can easily carry the output forward to Pandas, Polars, etc.—this is because `data` is only a “list of dicts,” and as such it is trivial to convert to a DataFrame. This allows you to analyze data received straight from an LLM. 123 | 124 | Of course, you will often want to set the temperature to `0.0` or another low value. This depends on the nature of your use-case. 125 | 126 | ## Using the builder to construct a turn-based experiment 127 | 128 | *New*: [**📺 WATCH VIDEO TUTORIAL**](https://youtu.be/tV5xACU-abw) 129 | 130 | We offer a web app to build *simple* experiments between multiple LLMs. The builder can be found [here](https://ego.mg.sb/builder/), with its source code being available [here](https://github.com/mrpg/ego_builder). 131 | 132 | 1. For now, just read through the app (it showcases an example of a framed ultimatum game) and scroll down. 133 | 134 | 2. Copy the code shown below “Export or import scenario” on the web app into a new file. Call that file `built.json` in your current project directory. **The file must be called `built.json`.** 135 | 136 | 3. Open a terminal and execute 137 | 138 | ```console 139 | (env) user@host:~$ ego run built 140 | ``` 141 | 142 | 4. This will show “System instructions” for two separate players. Note how they vary: One player (the first one) is the proposer and the second player is the responder. 143 | 144 | 5. The proposer is now asked to put in a proposal in JSON. Let's do it: 145 | 146 | ```json 147 | {"keep": 4.2} 148 | ``` 149 | 150 | 6. As you see, the responder is notified and can now `ACCEPT` or `REJECT`. Let's accept: 151 | 152 | ``` 153 | ACCEPT 154 | ``` 155 | 156 | 7. This completes the experiment. You will see something like: 157 | 158 | ``` 159 | Experiment c6627c4e-f17f-4cdc-ba47-462eced3e489 OK 160 | ``` 161 | 162 | 8. Let's look at the data that was generated. We can get it in CSV format by executing: 163 | 164 | ```console 165 | (env) user@host:~$ ego data built c6627c4e-f17f-4cdc-ba47-462eced3e489 > data.csv 166 | ``` 167 | 168 | (You need to replace `c6627c4e-f17f-4cdc-ba47-462eced3e489` with your actual experiment ID) 169 | 170 | This should tell you that 2 lines were written. If you open `data.csv` in your preferred spreadsheet calculator, you will see the following output: 171 | 172 | | choice | convo | experiment | i | round | tainted | thread | thread\_type | treatment | 173 | |---------------|--------------------------------------|--------------------------------------|---|-------|---------|--------------------------------------|-------------|-----------| 174 | | {"keep": 4.2} | 03ba9edf-99c0-46d6-8c26-42c26683197c | c6627c4e-f17f-4cdc-ba47-462eced3e489 | 1 | 1 | False | 1ce5faaf-cc1f-438f-8231-8b7e0d96fb07 | CLIThread | take | 175 | | "ACCEPT" | 03ba9edf-99c0-46d6-8c26-42c26683197c | c6627c4e-f17f-4cdc-ba47-462eced3e489 | 2 | 1 | False | 9043cd57-3e83-42d2-8d62-c783725e05e7 | CLIThread | take | 176 | 177 | This is obviously easy to post-process in whatever statistics software you use. 178 | 179 | If you re-run the experiment, enter garbage instead of the expected inputs and re-export the data, you will see that the `tainted` column becomes `True`. You can check for `tainted` to verify that inputs were received and processed as expected. Note that once *any* Thread responds invalidly, the Conversation will be stopped and all Threads will have `tainted` set to `True`. Thus, `ego data`'s output may be partial. 180 | 181 | You can run your scenario five times by doing 182 | 183 | ```console 184 | (env) user@host:~$ ego run -n 5 built 185 | ``` 186 | 187 | Needless to say, but you can replace `5` by any integer whatsoever. 188 | 189 | Feel free to experiment with our builder. 190 | 191 | *Note*: Experts can set the environment variable `BUILT_FILE` to have `ego` use a different file name from `built.json`. 192 | 193 | ## Using `alter_ego` with oTree 194 | 195 | *New*: [**📺 WATCH VIDEO TUTORIAL**](https://youtu.be/ouxRFdKOGEw) 196 | 197 | [oTree](https://www.otree.org) is a relatively popular framework for web-based experiments. 198 | 199 | ### Checking out our apps 200 | 201 | [Here](https://otree.readthedocs.io/en/latest/install-nostudio.html) is oTree's own guide to installing it on your computer, and [here](https://gitlab.com/gr0ssmann/otree_course#installing-otree) is another one by us. 202 | 203 | In general, after installing Python (and having the installer put it in your PATH), run 204 | 205 | ```console 206 | user@host:~$ python -m venv env 207 | user@host:~$ source env/bin/activate 208 | (env) user@host:~$ pip install -U alter_ego_llm # run this first 209 | (env) user@host:~$ pip install -U otree # then run this 210 | (env) user@host:~$ otree startproject my_project # this creates an oTree “project”, say no to sample games 211 | (env) user@host:~$ cd my_project # this enters the oTree “project” 212 | ``` 213 | 214 | Put your OpenAI API key into the file `openai_key` – note how this file has no extension. 215 | 216 | [Go to `otree/` in this repository.](https://github.com/mrpg/ego/tree/master/otree) Copy any of the apps `ego_human` or `ego_chat` into your project directory. The app directory must be on the same level of `settings.py`. In other words, here is how the folder structure looks if you decide to check out `ego_chat` and you faithfully followed all previous instructions: 217 | 218 | ``` 219 | . 220 | ├── ego_chat 221 | │   ├── Chat.html 222 | │   ├── __init__.py 223 | │   ├── prompts 224 | │   │   └── system.txt 225 | │   └── Welcome.html 226 | ├── openai_key 227 | ├── Procfile 228 | ├── requirements.txt 229 | ├── settings.py 230 | ├── _static 231 | │   └── global 232 | │   └── empty.css 233 | └── _templates 234 | └── global 235 | └── Page.html 236 | ``` 237 | 238 | Amend `settings.py` as follows: 239 | 240 | ```python 241 | SESSION_CONFIGS = [ 242 | dict( 243 | name="ego_chat", 244 | app_sequence=["ego_chat"], 245 | num_demo_participants=1, 246 | ), 247 | ] 248 | ``` 249 | 250 | Run `otree devserver` and open your browser to [localhost:8000](http://localhost:8000). From there, you can click `ego_chat` to invoke the app. Enjoy! 251 | 252 | By the way, if you wish to use another model, you can change `gpt-4` in line 31 in `ego_chat/__init__.py` to `gpt-3.5-turbo` or [any other supported](https://platform.openai.com/docs/models) value. 253 | 254 | *Note*: `alter_ego` saves message histories automatically in `.ego_output` in your oTree project folder. We did this so that nothing ever gets lost. 255 | 256 | ### In general 257 | 258 | You can attach Threads (i.e., LLMs) and Conversations (i.e., bundles of LLMs) to oTree objects (participants, players, subsessions or sessions). This basically works as follows (in your app's `__init__.py`: 259 | 260 | ```python 261 | from alter_ego.agents import * 262 | from alter_ego.utils import from_file 263 | from alter_ego.exports.otree import link as ai 264 | 265 | ... 266 | 267 | def creating_session(subsession): 268 | for player in subsession.get_players(): 269 | ai(player).set(GPTThread(model="gpt-3.5-turbo", temperature=1.0)) 270 | ``` 271 | 272 | Here, each `player` would get their own personal GPT agent. If you want to assign such an agent to a `group`, just do this: 273 | 274 | ```python 275 | def creating_session(subsession): 276 | for group in subsession.get_groups(): 277 | ai(group).set(GPTThread(model="gpt-3.5-turbo", temperature=1.0)) 278 | ``` 279 | 280 | Then, within your code, you can access the agent using a *context manager*. Here's an example of a simple `live_method`-based chat if we attached the Thread to the `player` object: 281 | 282 | ```python 283 | class Chat(Page): 284 | def live_method(player, data): 285 | if isinstance(data, str): 286 | with ai(player) as llm: 287 | # this submits the player's message and gets the response 288 | response = llm.submit(data, max_tokens=500) 289 | 290 | # note: if you put the AI on the "group" object or somewhere other than 291 | # the player, you may want to change this 292 | return {player.id_in_group: response} 293 | ``` 294 | 295 | If you want to set the system prompt, you can do: 296 | 297 | ``` 298 | def before_next_page(player, timeout_happened): 299 | with ai(player) as llm: 300 | # this sets the "system" prompt 301 | llm.system("You are participating in an experiment.") 302 | ``` 303 | 304 | You can do whatever you want, but always remember to open the LLM's context (using `with ai(...) as llm`) before performing any action. (This extra step is necessary because of oTree's ORM, which otherwise couldn't notice changes deep down in the Thread.) 305 | 306 | You can also attach a whole Conversation to the aforementioned oTree objects. 307 | 308 | We provide a simple Chat in this repository, see the directory `otree/ego_chat`. 309 | 310 | Remember to put your API key into your oTree project folder. `alter_ego` saves message histories automatically in `.ego_output` in your oTree project folder. 311 | 312 | ## Developing full-fledged experiments 313 | 314 | *New*: [**📺 WATCH VIDEO TUTORIAL**](https://youtu.be/WHW0gkT-oHE) 315 | 316 | You can use the primitives exposed by this library to develop full-fledged experiments that go beyond the capabilities of our builder. The directory `scenarios/` contains a bunch of examples, including the code for our paper's machine--machine interaction example (`ego_prereg.py`). Watch the video tutorial to get a feeling for what's possible. 317 | 318 | # Citation 319 | 320 | When using any part of `alter_ego` in a scientific context, cite the following work: 321 | 322 | ```tex 323 | @article{ego, 324 | title={Integrating Machine Behavior into Human Subject Experiments: A User-friendly Toolkit and Illustrations}, 325 | author={Engel, Christoph and Grossmann, Max R. P. and Ockenfels, Axel}, 326 | year={2023}, 327 | } 328 | ``` 329 | 330 | # License 331 | 332 | `alter_ego` is © Max R. P. Grossmann *et al.*, 2023. It is licensed under LGPLv3+. Please see `LICENSE` for details. 333 | 334 | This program is distributed in the hope that it will be useful, but **without any warranty**; without even the implied warranty of **merchantability** or **fitness for a particular purpose**. See the GNU Lesser General Public License for more details. 335 | --------------------------------------------------------------------------------