> ## Documentation Index
> Fetch the complete documentation index at: https://docs.uptrain.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Ollama

[Ollama](https://ollama.com/) is a great solution to run large language models (LLMs) on your local system.

### How will this help?

Using Ollama you can run models like Llama, Gemma locally on your system.

In this tutorial we will walk you though running evaluations on UpTrain using your local models hosted on Ollama.

### Prerequisites

1. Install Ollama to your system, you can download it from [here](https://ollama.com/download)

2. Pull the model using the command:

   ```bash theme={null}
   ollama pull <model_name>
   ```

   For the list of models supported by Ollama you can refer [here](https://ollama.com/library)

3. You can enter [http://localhost:11434/](http://localhost:11434/) in your web browser to confirm Ollama is running

### How to integrate?

**First, let's import the necessary packages**

```python theme={null}
# %pip install uptrain
```

```python theme={null}
from uptrain import EvalLLM, Evals, Settings
import json
```

**Create your data**

You can define your data as a simple dictionary with the following keys:

* `question`: The question you want to ask
* `context`: The context relevant to the question
* `response`: The response to the question

```python theme={null}
data = [
   {
        'question': 'What is the capital of France?',
        'context': 'France, context for its exquisite pastries and fashion, has a capital city called Paris. It\'s a place where people speak French and enjoy baguettes. I once heard that the Eiffel Tower was built by aliens, but don\'t quote me on that.',
        'response': 'The capital of France is Paris.'
    }
]
```

**Define the model**

We will be using Stable LM 2 1.6B for this example. You can refer the [documentation](https://ollama.com/library/stablelm2) on Ollama.

Remember to add "ollama/" at the beginning of the model name to let UpTrain know that you are using an Ollama model.

<Note>You can check if you have downloaded Stable LM 2 1.6B by running `!ollama list`
Else you can download it by `!ollama pull stablelm2`</Note>

```python theme={null}
settings = Settings(model='ollama/stablelm2')
```

**Create an EvalLLM Evaluator**

Before we can start using UpTrain, we need to create an EvalLLM Evaluator.

```python theme={null}
eval_llm = EvalLLM(settings)
```

We have used the following 3 metrics from UpTrain's library:

1. [Context Relevance](/predefined-evaluations/context-awareness/context-relevance): Evaluates how relevant the retrieved context is to the question specified.

2. [Response Conciseness](/predefined-evaluations/response-quality/response-conciseness): Evaluates how concise the generated response is or if it has any additional irrelevant information for the question asked..

3. [Response Relevance](/predefined-evaluations/response-quality/response-relevance): Evaluates how relevant the generated response was to the question specified.

You can look at the complete list of UpTrain's supported metrics [here](https://docs.uptrain.ai/predefined-evaluations/overview)

```python theme={null}
results = eval_llm.evaluate(
    project_name = 'Ollama-Demo',
    data=data,
    checks=[Evals.CONTEXT_RELEVANCE, Evals.RESPONSE_CONCISENESS, Evals.RESPONSE_RELEVANCE]
)
```

**View your results**

```python theme={null}
results = eval_llm.evaluate(
    project_name = 'Ollam-tutorial-test',
    data=data,
    checks=[Evals.CONTEXT_RELEVANCE, Evals.RESPONSE_CONCISENESS, Evals.RESPONSE_RELEVANCE]
)
```

Sample Reponse:

```json theme={null}
[  
  {  'context': 'France, context for its exquisite pastries and fashion, has '
                 "a capital city called Paris. It's a place where people speak "
                 'French and enjoy baguettes. I once heard that the Eiffel '
                 "Tower was built by aliens, but don't quote me on that.",
      'explanation_context_relevance': '{\n'
                                       '     "Reasoning": "The extracted '
                                       'context can answer the given query '
                                       'because it provides information about '
                                       "France's capital city, Paris. This "
                                       'information is relevant to answering '
                                       'the question about the capital of '
                                       'France.",\n'
                                       '     "Choice": "A"\n'
                                       '}',
      'explanation_response_conciseness': '{\n'
                                          '     "Reasoning": "The given '
                                          'response provides information about '
                                          'the capital of France, but it lacks '
                                          'the detail that could be expected '
                                          'in this context.",\n'
                                          '     "Choice": "B"\n'
                                          '}',
      'explanation_response_relevance': 'Response Precision: 0.5{\n'
                                        '     "Reasoning": "The given response '
                                        'provides information about the '
                                        'capital of France, but it lacks the '
                                        'detail that could be expected in this '
                                        'context.",\n'
                                        '     "Choice": "B"\n'
                                        '}\n'
                                        'Response Recall: 1.0{\n'
                                        '     "Reasoning": "The given response '
                                        'adequately answers the question about '
                                        'the capital of France because it '
                                        'provides the correct answer, i.e., '
                                        'Paris.",\n'
                                        '     "Choice": "A"\n'
                                        '}',
      'question': 'What is the capital of France?',
      'response': 'The capital of France is Paris.',
      'score_context_relevance': 1.0,
      'score_response_conciseness': 0.5,
      'score_response_relevance': 0.6666666666666666
  }
]

```

<CardGroup cols={2}>
  <Card title="Tutorial" href="https://github.com/uptrain-ai/uptrain/blob/main/examples/integrations/llm_providers/ollama.ipynb" icon="github" color="#808080">
    Open this tutorial in GitHub
  </Card>

  <Card title="Have Questions?" href="https://join.slack.com/t/uptraincommunity/shared_invite/zt-1yih3aojn-CEoR_gAh6PDSknhFmuaJeg" icon="slack" color="#808080">
    Join our community for any questions or requests
  </Card>
</CardGroup>