Interactive AI Text Adventure

# Image-Based Text Adventure Generator A Python application that transforms a collection of images into an interactive text adventure with branching storylines. ## Overview This tool uses AI vision and language models to: 1. Analyze images from your collection 2. Generate engaging story segments based on each image 3. Create thematic connections between story segments 4. Build an interactive adventure with multiple paths and endings The result is a set of markdown files that can be viewed as an interactive story where readers can make choices that lead to different narrative branches. ## Features - **Image Analysis**: Uses the Gemma 2 vision model to extract detailed descriptions from images - **Story Generation**: Creates narrative segments based on image content using Llama 3.3 language model - **Thematic Coherence**: Identifies common themes across all story segments and rewrites content for consistency - **Interactive Branching**: Automatically generates meaningful connections between story segments - **Customizable Style**: Supports multiple narrative styles (adventure, mystery, fantasy, sci-fi) - **Caching System**: Saves API responses to reduce processing time and costs on subsequent runs - **Markdown Output**: Generates properly formatted markdown files with navigation links - **Memory Management**: Processes images in batches with configurable delays to prevent memory issues ## Requirements - Python 3.6+ - Ollama (version 0.1.16 or higher) - Required Python packages (see requirements.txt): - ollama - pathlib - typing - requests - tqdm - pillow - pyyaml ## Installation 1. Clone this repository 2. Install required packages: ``` pip install -r requirements.txt ``` 3. Ensure Ollama is installed and running on your system 4. Download the required models: ``` ollama pull gemma2:27b ollama pull llama3.3:latest ``` ## Usage ### Basic Usage ```bash python main.py ``` This will: - Look for images in the default `input_images` directory - Generate stories in the `_stories` directory - Use the default "adventure" narrative style ### Command Line Options ```bash python main.py --input INPUT_DIR --output OUTPUT_DIR --style STYLE --length WORD_COUNT --batch-size BATCH_SIZE --delay DELAY --start START_NUM --end END_NUM --config CONFIG_FILE --no-cache ``` - `--input`: Directory containing images (default: "input_images") - `--output`: Directory for story files (default: "_stories") - `--style`: Narrative style - "adventure", "mystery", "fantasy", or "sci-fi" (default: "adventure") - `--length`: Approximate word count per story segment (default: 300) - `--batch-size`: Number of images to process before taking a longer break (default: 10) - `--delay`: Delay in seconds between processing images (default: 5) - `--start`: Start processing from this image number (default: 1) - `--end`: End processing at this image number (optional) - `--config`: Path to JSON configuration file - `--no-cache`: Disable caching of API responses ### Configuration File You can customize the application by creating a JSON configuration file: ```json { "input_dir": "my_images", "output_dir": "my_adventure", "vision_model": "gemma2:27b", "text_model": "llama3.3:latest", "story_length": 500, "temperature": 0.8, "narrative_style": "fantasy", "retry_attempts": 3, "retry_delay": 2, "batch_size": 5, "inter_image_delay": 10, "start_image": 1, "end_image": 20 } ``` ## Output Structure The generator creates: 1. An index.md file with: - A generated title for the overall adventure - A summary of themes - Links to all starting points 2. A markdown file for each image with: - A generated title - The image - A story segment - Links to connected story segments ## Troubleshooting If you encounter issues while using the generator, please refer to the [Troubleshooting Guide](TROUBLESHOOTING.md) for solutions to common problems. ## Examples ### Example Script An example script `example.sh` is provided to demonstrate different ways to use the generator: ```bash # Make the script executable chmod +x example.sh # Run the example ./example.sh ``` The script shows how to: - Process a subset of images to avoid memory issues - Use different narrative styles - Customize story length - Use custom input and output directories - Use a configuration file ### Sample Configuration A sample configuration file `sample_config.json` is provided as a template: ```bash # Run with the sample configuration python main.py --config sample_config.json ``` ### Output After running the generator, open `_stories/index.md` to start the adventure. Each page will present a story segment with choices that lead to other segments, creating a branching narrative experience. ## Customization - Add your own images to the input directory - Modify the narrative style to change the tone and genre - Adjust the story length to create shorter or longer segments - Edit the prompts in the code to customize the story generation process ## Acknowledgments This project uses: - Ollama for local AI model hosting - Gemma 2 for vision analysis - Llama 3.3 for text generation ------- # Multimodal Story Generation System [](https://opensource.org/licenses/MIT) [](https://www.python.org/) [](https://ollama.ai/) Transform visual inputs into structured narratives using cutting-edge AI technologies. This system combines computer vision and large language models to generate dynamic, multi-chapter stories from images.  ## Features - 🖼️ **Image Analysis** - Extract narrative elements from images using LLaVA - 📖 **Adaptive Story Generation** - Generate 5-chapter stories with Gemma2-27B - 🧠 **Context Awareness** - Maintain narrative consistency with ChromaDB RAG - 📊 **Interactive Visualization** - ReactFlow-powered story graph interface - 🚀 **Production Ready** - Dockerized microservices architecture ## Table of Contents - [Quick Start](#quick-start) - [System Requirements](#system-requirements) - [Architecture](#architecture) - [Production Deployment](#production-deployment) - [Troubleshooting](#troubleshooting) - [Ethical Considerations](#ethical-consideration) - [Contributing](#contributing) - [License](#license) ## Quick Start ### Local Development Setup 1. **Clone Repository** ```bash git clone https://github.com/kliewerdaniel/ITB02 cd ITB02 Create Virtual Environment python -m venv venv source venv/bin/activate # Linux/Mac venv\Scripts\activate # Windows Install Dependencies pip install -r requirements.txt # Apple Silicon Special Setup pip install --pre torch --extra-index-url https://download.pytorch.org/whl/nightly/cpu brew install libjpeg webp Initialize AI Models ollama pull gemma2:27b ollama pull llava Start Services # Backend (FastAPI) uvicorn backend.main:app --reload # Frontend (new terminal) cd frontend npm install && npm run dev Verify Installation curl http://localhost:8000/health # Expected response: {"status":"healthy"} System Requirements Python 3.11+ Node.js 18+ Ollama runtime 16GB RAM (24GB+ recommended for GPU acceleration) 10GB+ Disk Space Architecture [Frontend] ←HTTP→ [FastAPI] ↓ ↑ [Ollama] ←→ [ChromaDB] ↓ [Redis] ↓ [Celery Workers] Key Components Component Technology Stack Function Image Analysis LLaVA, Pillow Visual narrative extraction Story Engine Gemma2-27B, LangChain Context-aware chapter generation Knowledge Base ChromaDB Narrative consistency management API Layer FastAPI REST endpoint management Visualization ReactFlow, Zustand Interactive story mapping Production Deployment Docker Setup # Build and launch all services docker-compose up --build # Initialize vector store docker exec -it backend python -c "from backend.core.rag_manager import NarrativeRAG; NarrativeRAG()" Cluster Configuration # docker-compose.yml excerpt services: ollama: deploy: resources: limits: memory: 12G cpus: '4' Troubleshooting Common Issues Missing Vector Store rm -rf chroma_db && mkdir chroma_db Out-of-Memory Errors export OLLAMA_MAX_LOADED_MODELS=2 CUDA Compatibility Issues pip uninstall torch pip install torch --extra-index-url https://download.pytorch.org/whl/cu117 Daniel Kliewer GitHub Profile AI Systems Developer --- combine these two so that the textadventure repo will be improved with a prompt to edit it to have the all of the outlined capabilities

This is an impressive and well-structured project! A multimodal text adventure generator that integrates AI-powered vision and storytelling is a unique concept. It’s great to see that you're incorporating **LLaVA for image analysis**, **Gemma2-27B for text generation**, and **ChromaDB for maintaining narrative coherence**. ### Some Thoughts & Suggestions: 1. **Interactive Story Navigation** - Markdown with navigation links is great, but you might consider an optional lightweight **web interface** for a more interactive experience. Maybe a static site with **Next.js** or **SvelteKit**? 2. **Memory & Performance Optimization** - **Reducing API Calls**: If a user uploads a batch of images, maybe use **deduplication** (hashing image embeddings) to avoid redundant processing. - **Batch Processing**: Implement an **async queue** with Celery or Redis Streams for more efficient processing. 3. **Customization via Prompt Engineering** - You could allow users to tweak **persona, tone, or even narrative complexity** in a simple config file. - Maybe allow **prewritten characters** or user-generated lore to be injected dynamically? 4. **Multi-Modal Expansion?** - If you ever want to push this further, you could integrate **audio** (TTS for storytelling) or even **gamebook mechanics** (like skill-based challenges). Let me know if you need help refining any part of the architecture or implementation—this project has **huge potential!** 🚀

let's create a site where I can run the python script on the backend to generate all of the markdown files and simply use git commits to update the site hosted on netlify --- choose something I can deploy for free to netlify --- I want to create a UI that will allow me to upload and select the photos as well as edit the prompts and characteristics that will be employed in the generation allowing the user to tweak the variables like temperature or other aspects or selecting which ollama model to use or anything else from the config ------ I want the site to be simple to build and deploy

We can use **Next.js** with **a simple API route for backend processing** to run your Python script and generate Markdown files. Here's how the setup will work: 1. **Frontend (Next.js)**: - UI for uploading/selecting photos - Input fields for editing prompts, temperature, model selection, and other parameters - A button to trigger Markdown generation 2. **Backend (API Route in Next.js + Python script)**: - The API route calls your Python script - The script generates the Markdown files and commits them to a Git repo - Netlify auto-deploys when it detects new commits 3. **Deployment on Netlify** (Free Plan): - Uses GitHub for version control - Auto-builds and deploys when new commits are pushed This way, the site remains lightweight, and everything runs within the Netlify free tier. Want me to scaffold the Next.js + API backend for you?

now write a detailed prompt which will take this repo and turn it into what you described: https://github.com/kliewerdaniel/TextAdventure

To transform the [TextAdventure](https://github.com/kliewerdaniel/TextAdventure) repository into a web application with the desired functionalities—image upload, prompt customization, and parameter adjustments—we can follow these steps: 1. **Frontend Development with Next.js**: - **Image Upload Interface**: Implement a user-friendly component allowing users to upload images. - **Prompt and Parameter Customization**: Provide input fields for users to modify prompts, set parameters like temperature, and select AI models. - **Submission Mechanism**: Include a form submission button that sends the user's inputs to the backend for processing. 2. **Backend Integration**: - **API Routes in Next.js**: Utilize Next.js API routes to handle form submissions and manage backend processes. - **Executing Python Scripts**: Configure these API routes to invoke the existing Python scripts, passing user inputs as arguments. This can be achieved using Node.js's `child_process` module to spawn Python processes. 3. **Continuous Deployment with Netlify**: - **Git Integration**: Set up a Git repository for the project. - **Netlify Deployment**: Connect the Git repository to Netlify for automatic deployments upon new commits. **Detailed Implementation Steps**: 1. **Initialize the Next.js Project**: - Create a new Next.js application using the command: ```bash npx create-next-app text-adventure ``` 2. **Develop the Frontend Components**: - **Image Upload**: Use the `next/image` component and HTML file input elements to facilitate image uploads. - **Form for Customization**: Design a form with fields corresponding to the customizable parameters (e.g., prompts, temperature, model selection). - **Form Submission**: Handle form submissions using Next.js's built-in form handling or client-side JavaScript to send data to the backend API routes. 3. **Set Up API Routes**: - In the `pages/api` directory, create API route files (e.g., `generate.js`) to handle incoming requests. - Within these routes, use Node.js's `child_process` module to execute the Python scripts: ```javascript import { exec } from 'child_process'; export default function handler(req, res) { const { prompt, temperature, model, images } = req.body; // Construct the command to run the Python script const command = `python3 main.py --prompt "${prompt}" --temperature ${temperature} --model ${model} --images ${images.join(' ')}`; exec(command, (error, stdout, stderr) => { if (error) { console.error(`Error: ${stderr}`); res.status(500).json({ error: 'Script execution failed' }); return; } res.status(200).json({ message: 'Script executed successfully', output: stdout }); }); } ``` 4. **Handle File Uploads**: - Implement middleware or utilize libraries like `multer` to process file uploads in Next.js API routes. - Ensure uploaded images are stored in a directory accessible to the Python script. 5. **Configure Netlify for Deployment**: - **Netlify Functions**: Since Netlify supports serverless functions, ensure that the Next.js API routes are compatible. Note that Netlify has limitations on function execution time and memory usage. - **Build Settings**: In Netlify, set the build command to `next build` and the publish directory to `.next`. - **Environment Variables**: Store any sensitive information or configuration parameters as environment variables in Netlify's settings. 6. **Testing and Deployment**: - **Local Testing**: Test the application locally to ensure that the frontend can communicate with the backend and that the Python scripts execute correctly. - **Push to Git Repository**: Commit and push the code to the connected Git repository. - **Netlify Deployment**: Netlify will automatically detect the new commit and initiate the deployment process. Monitor the deployment logs for any errors and resolve them as needed. **Considerations**: - **Execution Limits**: Netlify's serverless functions have execution time and memory limits. Ensure that the Python scripts are optimized to run within these constraints. - **Security**: Validate and sanitize all user inputs to prevent security vulnerabilities. - **Dependencies**: Ensure that all necessary Python dependencies are specified in a `requirements.txt` file and that Netlify is configured to install them during the build process. By following these steps, you can create a web application that allows users to upload images, customize prompts and parameters, and generate interactive text adventures, all deployed seamlessly on Netlify.