> ## Documentation Index
> Fetch the complete documentation index at: https://docs.uptrain.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Re-Ranking Evaluations

Re-ranking involves reordering nodes based on relevance to the query and choosing top n nodes. Different evaluations are performed based on the number of nodes returned after re-ranking.

1. Same Number of Nodes

   Context Reranking: Checks if the order of re-ranked nodes is more relevant to the query than the original order.
2. Different Number of Nodes:

   Context Conciseness: Examines whether the reduced number of nodes still provides all the required information.

These evaluations collectively ensure the robustness and effectiveness of the RAG query engine, SubQuestionQueryGeneration operator, and the re-ranking process in the LlamaIndex pipeline.

### How to do it?

<Steps>
  <Step title="Install UpTrain and LlamaIndex">
    ```python theme={null}
    pip install -q html2text llama-index pandas tqdm uptrain cohere
    ```
  </Step>

  <Step title="Import required libraries">
    ```python theme={null}
    from llama_index import (
        ServiceContext,
        VectorStoreIndex,
    )
    from llama_index.node_parser import SentenceSplitter
    from llama_index.readers import SimpleWebPageReader
    from llama_index.callbacks import CallbackManager, UpTrainCallbackHandler
    from llama_index.postprocessor.cohere_rerank import CohereRerank
    from llama_index.service_context import set_global_service_context
    from llama_index.query_engine.sub_question_query_engine import (
        SubQuestionQueryEngine,
    )
    from llama_index.tools.query_engine import QueryEngineTool
    from llama_index.tools.types import ToolMetadata
    ```
  </Step>

  <Step title="Setup UpTrain Open-Source Software (OSS)">
    You can use the open-source evaluation service to evaluate your model. In this case, you will need to provie an OpenAI API key. You can get yours [here](https://platform.openai.com/account/api-keys).

    Parameters:

    * `key_type`="openai"
    * `api_key`="OPENAI\_API\_KEY"
    * `project_name_prefix`="PROJECT\_NAME\_PREFIX"

    ```python theme={null}
    callback_handler = UpTrainCallbackHandler(
        key_type="openai",
        api_key="sk-...",  # Replace with your OpenAI API key
        project_name_prefix="llama",
    )
    Settings.callback_manager = CallbackManager([callback_handler])
    ```
  </Step>

  <Step title="Load and Parse Documents">
    Load documents from Paul Graham's essay "What I Worked On".

    ```python theme={null}
    documents = SimpleWebPageReader().load_data(
      [
          "https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt"
      ]
    )
    ```

    Parse the document into nodes.

    ```python theme={null}
    parser = SentenceSplitter()
    nodes = parser.get_nodes_from_documents(documents)
    ```
  </Step>

  <Step stepNumber="5" title="Re-ranking">
    Re-ranking is the process of reordering the nodes based on their relevance to the query. There are multiple classes of re-ranking algorithms offered by Llamaindex. We have used CohereRerank for this example.

    The re-ranker allows you to enter the number of top n nodes that will be returned after re-ranking. If this value remains the same as the original number of nodes, the re-ranker will only re-rank the nodes and not change the number of nodes. Otherwise, it will re-rank the nodes and return the top n nodes.

    We will perform different evaluations based on the number of nodes returned after re-ranking.

    <Step stepNumber="5.1">
      <p> <strong> Re-ranking (With same number of nodes) </strong></p>
      If the number of nodes returned after re-ranking is the same as the original number of nodes, the following evaluation will be performed:

      * Context Reranking: Check if the order of the re-ranked nodes is more relevant to the query than the original order.

      ```python theme={null}
      api_key = "**********************************"  # Insert cohere API key here
      cohere_rerank = CohereRerank(
          api_key=api_key, top_n=5
      )  # In this example, the number of nodes before re-ranking is 5 and after re-ranking is also 5.

      index = VectorStoreIndex.from_documents(
          documents=documents, service_context=service_context
      )

      query_engine = index.as_query_engine(
          similarity_top_k=10,
          node_postprocessors=[cohere_rerank],
          service_context=service_context,
      )

      response = query_engine.query(
          "What did Sam Altman do in this essay?",
      )
      ```

      ```bash theme={null}
      Question: What did Sam Altman do in this essay?
      Context Reranking Score: 0.0
      ```
    </Step>

    <Step stepNumber="5.2">
      <p> <strong> Re-ranking (With different number of nodes) </strong></p>
      If the number of nodes returned after re-ranking is the lesser as the original number of nodes, the following evaluation will be performed:

      * Context Conciseness: If the re-ranked nodes are able to provide all the information required by the query.

      ```python theme={null}
      api_key = "**********************************"  # insert cohere API key here
      cohere_rerank = CohereRerank(
          api_key=api_key, top_n=2
      )  # In this example, the number of nodes before re-ranking is 5 and after re-ranking is 2.

      index = VectorStoreIndex.from_documents(
          documents=documents, service_context=service_context
      )
      query_engine = index.as_query_engine(
          similarity_top_k=10,
          node_postprocessors=[cohere_rerank],
          service_context=service_context,
      )

      # Use your advanced RAG
      response = query_engine.query(
          "What did Sam Altman do in this essay?",
      )
      ```

      ```bash theme={null}
      Question: What did Sam Altman do in this essay?
      Context Conciseness Score: 1.0
      ```
    </Step>
  </Step>
</Steps>

<CardGroup cols={2}>
  <Card title="Tutorial" href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/callbacks/UpTrainCallback.ipynb" icon="infinity" color="#808080">
    Open this tutorial in Colab
  </Card>

  <Card title="Have Questions?" href="https://join.slack.com/t/uptraincommunity/shared_invite/zt-1yih3aojn-CEoR_gAh6PDSknhFmuaJeg" icon="slack" color="#808080">
    Join our community for any questions or requests
  </Card>
</CardGroup>