Get Started with OASIS-LLM

Install OASIS-LLM, configure your provider API key, and launch a 100-trial pilot rating run against a vision-language model in minutes.

OASIS-LLM is a research harness that submits the 900 OASIS affective images to vision-language models and records their valence and arousal ratings for comparison against human norms. This guide takes you from a fresh clone to a running pilot in a few minutes.

Prerequisites

Before you start, make sure you have:

• Python 3.12 or later — check with python --version
• uv — the package manager used to install and run OASIS-LLM
• An API key for at least one supported provider (OpenRouter, OpenAI, Anthropic, or Google), or a local Ollama installation

1. Clone the repository

   Clone the repository and change into the project directory:

   git clone https://github.com/DCPMA/AI-Psy.git
   cd AI-Psy
   
2. Create your environment file

   Copy the example environment file:

   cp .env.example .env
   

   Open .env and fill in the key for the provider you want to use. You only need to set one.

   OpenRouter

   OPENROUTER_API_KEY=your_key_here
   

   OpenRouter gives you access to hundreds of models — including free-tier models — under a single key. It is the easiest way to get started if you do not already have a direct provider key.

3. Install dependencies

   Run uv sync to install all dependencies from the lockfile:

   uv sync
   
4. Download the OASIS images

   The 900 OASIS images are licensed under CC BY-NC-SA 4.0 by the original authors and are not bundled with this repository. Download them from osf.io/6pnd7 and unpack the archive into the OASIS/images/ directory:

   OASIS/
   └── images/
       ├── OASIS_1.jpg
       ├── OASIS_2.jpg
       └── ...
   

   The harness cannot submit images to a model if the OASIS/images/ directory is missing or empty. The smoke test and pilot commands will fail until the images are in place.

5. Smoke test your setup

   Before committing to a full run, verify that your API key and image path are working correctly:

   uv run oasis-llm smoke configs/runs/smoke-openrouter-gemma4.yaml
   

   This sends 3 images through the pipeline with a single valence rating each and prints the results to the terminal. It takes under a minute and costs a fraction of a cent.

   Always run oasis-llm smoke with your chosen config before launching a pilot. It catches misconfigured keys, missing images, and model connectivity problems early.

6. Launch the dashboard

   Open the Streamlit dashboard to browse the image set, design experiments, and monitor runs:

   uv run oasis-llm dashboard
   

   The dashboard opens at http://localhost:8501. From there you can inspect the OASIS image set, create datasets, and preview cost estimates before running anything from the CLI.

7. Run a pilot from the CLI

   Launch a 30-image pilot run using one of the example configs. Pick the config that matches your provider:

   OpenRouter — Gemma 4

   uv run oasis-llm run configs/runs/pilot-gemma4-31b.yaml
   

   Each pilot config targets 30 stratified images × 5 samples per image (150 trials) on both valence and arousal dimensions. The run is idempotent — if it is interrupted you can re-run the same command and it will resume from where it left off.

8. Check run status

   While a run is in progress, or after it completes, check status across all runs:

   uv run oasis-llm status
   

   To inspect a specific run by its ID:

   uv run oasis-llm status <run_id>
   

   The table shows counts of done, pending, and failed trials, along with cumulative cost in USD.

9. Export results

   Export the completed trials for a run to a CSV file:

   uv run oasis-llm export <run_id> outputs/<run_id>.csv
   

   Replace <run_id> with the identifier shown in oasis-llm status — for example, pilot-gemma4-31b. The outputs/ directory is created automatically if it does not exist.

Next steps

With your first pilot complete, you can open the dashboard’s Analysis page to compare the model’s ratings against the human norms, or read Configuration to learn how to customise run parameters like samples_per_image, max_concurrency, and prompt settings.