Stop Parameter in JSON

How does this work Use the json stop section `json { "model": "your_model_name", "prompt": "your_prompt_here", "parameters": { "temperature": 0.0, "top_p": 1.0, "max_tokens": 50, "stop": ["<think></think>"] } } ```

The `stop` parameter in the JSON configuration is used to specify a sequence of characters or tokens that will halt the response generation once encountered. This is particularly useful when you want to ensure the output stops at a specific point, preventing the model from continuing beyond a desired boundary. Here's how your example works: 1. **Model**: `"your_model_name"` specifies the model you are using to generate the text. 2. **Prompt**: `"your_prompt_here"` contains the text or query you want the model to generate a response to. 3. **Parameters**: - **`temperature`**: Controls randomness. A value of `0.0` ensures deterministic output (no randomness). - **`top_p`**: Determines the nucleus sampling size. `1.0` includes all possibilities, ensuring no tokens are excluded. - **`max_tokens`**: Limits the number of tokens in the response to 50. - **`stop`**: The list `["<think></think>"]` defines a token or sequence of tokens where generation should halt. ### Example: If the prompt is: ```text "Write a short description about artificial intelligence:<think></think>" ``` The model will generate output like: ```text "Artificial intelligence (AI) is the simulation of human intelligence processes by machines." ``` When the model encounters `"<think></think>"`, it stops generating any further text. This is useful for: - Preventing extraneous or unwanted output. - Segregating output when processing structured data (e.g., completing a single field in JSON). - Formatting results to conform to strict requirements. Let me know if you'd like a deeper explanation or more examples!