Ollama Process Terminated Error

INFO: Started server process [31020] INFO: Waiting for application startup. INFO: Application startup complete. INFO: 127.0.0.1:55886 - "POST /generate HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ self.scope, self.receive, self.send ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/applications.py", line 112, in __call__ await self.middleware_stack(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/middleware/errors.py", line 187, in __call__ raise exc File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/middleware/errors.py", line 165, in __call__ await self.app(scope, receive, _send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/middleware/cors.py", line 85, in __call__ await self.app(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/routing.py", line 715, in __call__ await self.middleware_stack(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/routing.py", line 735, in app await route.handle(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app raise exc File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/starlette/routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/fastapi/routing.py", line 301, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<3 lines>... ) ^ File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/fastapi/routing.py", line 212, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/danielkliewer/MultiRank/main.py", line 23, in generate_text return await orchestrator.parallel_inference(request.prompt) # Access via model ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/danielkliewer/MultiRank/core/llm_orchestrator.py", line 51, in parallel_inference results = await asyncio.gather( ^^^^^^^^^^^^^^^^^^^^^ *[_call_model(model) for model in self.models.values()] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/Users/danielkliewer/MultiRank/core/llm_orchestrator.py", line 49, in _call_model return await ollama.generate(model=model_name, prompt=prompt) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/ollama/_client.py", line 242, in generate return self._request( ~~~~~~~~~~~~~^ GenerateResponse, ^^^^^^^^^^^^^^^^^ ...<16 lines>... stream=stream, ^^^^^^^^^^^^^^ ) ^ File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/ollama/_client.py", line 178, in _request return cls(**self._request_raw(*args, **kwargs).json()) ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/ollama/_client.py", line 122, in _request_raw raise ResponseError(e.response.text, e.response.status_code) from None ollama._types.ResponseError: llama runner process has terminated: signal: killed (status code: 500) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Traceback: File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/streamlit/runtime/scriptrunner/exec_code.py", line 88, in exec_func_with_error_handling result = func() File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 579, in code_to_exec exec(code, module.__dict__) ~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/danielkliewer/MultiRank/streamlit_frontend.py", line 10, in <module> results = response.json() File "/Users/danielkliewer/MultiRank/venv/lib/python3.13/site-packages/requests/models.py", line 978, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

The key error is: ``` ollama._types.ResponseError: llama runner process has terminated: signal: killed (status code: 500) ``` This means that your Ollama model process was killed, likely due to one of the following reasons: ### Possible Causes and Fixes: 1. **Out of Memory (OOM) Issue** - If your system doesn't have enough RAM/VRAM, the process can be killed. - Check memory usage: ```bash free -h # Linux/macOS vm_stat # macOS ``` - Try using a smaller model or increasing swap memory. 2. **Process Conflicts or Ollama Not Running** - Check if Ollama is running properly: ```bash ps aux | grep ollama ``` - Restart the Ollama service: ```bash ollama stop && ollama serve ``` 3. **LLM Model Not Loaded Correctly** - Ensure the model name is correct in your `llm_orchestrator.py`. - Try loading the model manually: ```python import ollama response = ollama.generate(model="your-model-name", prompt="Hello?") print(response) ``` - If the model isn't available, download it again: ```bash ollama pull your-model-name ``` 4. **Async Call Issues** - Try running `parallel_inference` with just one model and see if it works. - Debug by printing `self.models.values()` before `asyncio.gather()`. 5. **FastAPI / Uvicorn Issue** - Restart FastAPI and clear any locked processes: ```bash kill -9 $(lsof -t -i:8000) uvicorn main:app --reload ``` Let me know what you find after trying these! 🚀