Advanced RAG series: Query Decomposition

Query Decomposition:

• Query Translation 처럼 여러개의 쿼리를 생성한다는 점은 동일함.
• Query Translation 은 여러개의 쿼리를 유사한 의미를 가진 다른 단어로 쿼리를 생성해 내거나, multi-query 접근 방식처럼 다른 관점에서 쿼리를 생성해 나가는 방법이 있음. 하지만 이런 방법들은 유저의 쿼리가 복잡한 단계적인 과정을 포함할 때 해결하기 힘듬.
• Query Decomposition 은 복잡한 쿼리를 multiple sub-problem queries 로 분리해 나가고, 이런 쿼리들을 연속적/병렬적으로 처리해 나가면서 오리지널 쿼리의 답변을 만드는 방식이다.

 

Query Decomposition 1) Least-to-Most Prompting:

Least-to-Most Prompting:

• 복잡한 문제를 해결하기 위한 프롬프트 기법으로, 문제를 더 작은 하위 문제로 나누어서 접근하고 해결하는 방법임.
• 이 기법을 Query Decomposition 에서 사용하는 방법을 보자.

 

LangChain 에서 Least-to-most prompting 사용 방법은 다음과 같다:

from langchain.prompts import ChatPromptTemplate

decompostion_prompt = ChatPromptTemplate.from_template(
    """
    You are a helpful assistant that can break down complex questions into simpler parts. \n
        Your goal is to decompose the given question into multiple sub-questions that can be answerd in isolation to answer the main question in the end. \n
        Provide these sub-questions separated by the newline character. \n
        Original question: {question}\n
        Output (3 queries): 
    """
)

query_generation_chain = (
    {"question": RunnablePassthrough()}
    | decompostion_prompt
    | ChatOpenAI(model='gpt-4', temperature=0.7)
    | StrOutputParser()
    | (lambda x: x.split("\n"))
)

questions = query_generation_chain.invoke("What are the benefits of QLoRA?")
questions

decompostion_prompt 로 생성된 쿼리들의 출력 결과:

['What is QLoRA?',
 'What are the features of QLoRA?',
 'How do these features of QLoRA provide benefits?']

다음으로 이제 이렇게 생성된 하위 문제 쿼리들을 단계적으로 해결해나가면서 최종적으로 복잡한 답변을 생성해 나가는 코드를 보자.

from operator import itemgetter


# Create the final prompt template to answer the question with provided context and background Q&A pairs
template = """Here is the question you need to answer:

\n --- \n {question} \n --- \n

Here is any available background question + answer pairs:

\n --- \n {q_a_pairs} \n --- \n

Here is additional context relevant to the question: 

\n --- \n {context} \n --- \n

Use the above context and any background question + answer pairs to answer the question: \n {question}
"""

least_to_most_prompt = ChatPromptTemplate.from_template(template) 
llm = ChatOpenAI(model='gpt-4', temperature=0)

least_to_most_chain = (
        {'context': itemgetter('question') | retriever,
        'q_a_pairs': itemgetter('q_a_pairs'),
        'question': itemgetter('question'),
        }
        | least_to_most_prompt
        | llm
        | StrOutputParser()
    )

q_a_pairs = ""
for q in questions:

    answer = least_to_most_chain.invoke({"question": q, "q_a_pairs": q_a_pairs})
    q_a_pairs+=f"Question: {q}\n\nAnswer: {answer}\n\n"

 

 

Query Decomposition 2) Step back prompting:

Step back prompting:

• 직접적인 문제 해결 대신 '한 걸음 물러서서' 더 넓은 맥락에서 문제를 바라보는 방법임. 구체적인 질문이나 문제에 대해 즉각적으로 답하기보다, 먼저 관련된 일반적인 원칙이나 개념을 고려하는 방법이다.
• 이런 broader perspective를 통해 문제의 본질을 더 잘 이해하고, 더 효과적인 해결책을 찾기 위해서 사용되는 방법임.
• Step back prompting 을 이용해서 Original Query 에 다가 추가로 추상화된 Query 를 만들고, 이를 이용해서 Vectore Store 에 Context 를 가져와서 답변을 하도록 만들 수 있음.

 

LangChain 에서 Stepback prompting 을 사용하는 방법은 다음과 같다:

from langchain_core.prompts import ChatPromptTemplate, FewShotChatMessagePromptTemplate

examples = [
    {
        'input': 'What happens to the pressure, P, of an ideal gas if the temperature is increased by a factor of 2 and the volume is increased by a factor of 8?',
        'output': 'What are the physics principles behind this question?'
    },
    {
        'input': 'Estella Leopold went to which school between Aug 1954 and Nov 1954?',
        'output': "What was Estella Leopold's education history?"
    }
]
example_prompt = ChatPromptTemplate.from_messages(
            [
                ('human', '{input}'), ('ai', '{output}')
            ]
        )
few_shot_prompt = FewShotChatMessagePromptTemplate(
    examples=examples,
            # This is a prompt template used to format each individual example.
    example_prompt=example_prompt,
)

final_prompt = ChatPromptTemplate.from_messages(
            [
                ('system', """You are an expert at world knowledge. Your task is to step back and paraphrase a question to a more generic step-back question, which is easier to answer. Here are a few examples:"""),
                few_shot_prompt,
                ('user', '{question}'),
            ]
        )

final_prompt.format(question= "What are the benefits of QLoRA?")

step_back_query_chain = (
    {'question': RunnablePassthrough()}
    | final_prompt 
    | ChatOpenAI(model='gpt-4', temperature=0.7) 
    | StrOutputParser()
    )

step_back_query_chain.invoke("What are the optimal parameters for QLoRA?")

 

 

Step back prompting 을 이용하는 예시는 다음과 같다:

response_prompt_template = """You are an expert of world knowledge. 
I am going to ask you a question. Your response should be comprehensive and not contradicted with the following context if they are relevant. 
Otherwise, ignore them if they are not relevant.

# {normal_context}
# {step_back_context}

# Original Question: {question}
# Answer:"""
response_prompt = ChatPromptTemplate.from_template(response_prompt_template)

step_back_chain = (
    {'normal_context': RunnablePassthrough() | retriever,
     'step_back_context': RunnablePassthrough() | step_back_query_chain | retriever,
     'question': RunnablePassthrough()
     }
    | response_prompt
    | ChatOpenAI(model='gpt-4', temperature=0)
    | StrOutputParser()
)

step_back_chain.invoke("What are the optimal parameters for QLoRA?")