LangChain for LLM Application Development

Introduction

LLM 기반 AI 어플리케이션을 구축할 때 마주치는 여러 코드 패턴들을 추상화해놓아서 개발을 더 편하게 만들어주는게 Langchain 임.

 

여기서는 LangChain 의 공통적인 컴포넌트 들에 대해서도 배워보자:

• Model
• Prompts
• Index
• Chain
• Parsers

 

Models, Prompts and parsers

LangChain 구성 요소인 Model 과 Prompt 그리고 Parser 에 대해서 알아보자:

• Model: 자연어 처리(NLP) 기능을 수행하는 언어 모델을 말한다. GPT 와 같은 Transformer 기반 모델이 이에 해당함.
• Prompt: 프롬프트는 모델에 입력되는 텍스트 형식의 지시임. 지시는 모델에게 특정 작업을 수행하도록 요청하는 역할을 함.
• Parser: 모델의 output 을 적절한 형식의 구조로 변환하는 역할을 한다. 이게 없다면 수동으로 직접 파싱을 해서 원하는 결과를 추출해야함.

 

LangChain 없이 ChatGPT 를 사용하는 방법은 다음과 같다:

def get_completion(prompt, model=llm_model):
    messages = [{"role": "user", "content": prompt}]
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=0, 
    )
    return response.choices[0].message["content"]

customer_email = """
Arrr, I be fuming that me blender lid \
flew off and splattered me kitchen walls \
with smoothie! And to make matters worse,\
the warranty don't cover the cost of \
cleaning up me kitchen. I need yer help \
right now, matey!
"""

style = """American English \
in a calm and respectful tone
"""

prompt = f"""Translate the text \
that is delimited by triple backticks 
into a style that is {style}.
text: ```{customer_email}```
"""

response = get_completion(prompt)

 

 

출력:

'I am quite upset that my blender lid came off and caused my smoothie to splatter all over my kitchen walls. Additionally, the warranty does not cover the cost of cleaning up the mess. Would you be able to assist me at this time, my friend?'

 

 

LangChain 을 사용하면 Prompt 를 쉽게 재사용할 수 있다.

ChatGPT 에 요청을 하는 객체 생성:

• 여기서 필요하다면 생성자에 인자를 줘서 파라미터를 수정할 수 있다.

# To control the randomness and creativity of the generated
# text by an LLM, use temperature = 0.0
chat = ChatOpenAI(temperature=0.0, model=llm_model)
chat

ChatOpenAI(verbose=False, callbacks=None, callback_manager=None, client=<class 'openai.api_resources.chat_completion.ChatCompletion'>, model_name='gpt-3.5-turbo-0301', temperature=0.0, model_kwargs={}, openai_api_key=None, openai_api_base=None, openai_organization=None, request_timeout=None, max_retries=6, streaming=False, n=1, max_tokens=None)

 

 

Prompt template 를 이용해서 Prompt 를 관리하는 방법:

template_string = """Translate the text \
that is delimited by triple backticks \
into a style that is {style}. \
text: ```{text}```
"""

from langchain.prompts import ChatPromptTemplate

prompt_template = ChatPromptTemplate.from_template(template_string)

prompt_template.messages[0].prompt

 

 

prompt_template 내부 구조:

• Prompt 에 동적으로 넣을 변수인 input_variables 를 볼 수 있음.
• template 을 통해 프롬포트를 볼 수 있음.
• 이후에 소개하겠지만 모델이 반환할 output 을 원하는 형태로 변환하기 위해서 output_parser 도 넣을 수 있음.

PromptTemplate(input_variables=['style', 'text'], output_parser=None, partial_variables={}, template='Translate the text that is delimited by triple backticks into a style that is {style}. text: ```{text}```\n', template_format='f-string', validate_template=True)

 

 

prompt_template 에서 input_variable 확인:

prompt_template.messages[0].prompt.input_variables

['style', 'text']

 

 

prompt_template 을 통해 ChatGPT 에서 결과 가지고 오는 가이드:

• format_messages() 메소드를 통해 LLM 에게 명령할 프롬포트 생성
• chat(customer_messages) 를 통해 ChatGPT 에게 명령을 내리고 결과를 가져옴

customer_style = """American English \
in a calm and respectful tone
"""

customer_email = """
Arrr, I be fuming that me blender lid
flew off and splattered me kitchen walls
with smoothie! And to make matters worse,
the warranty don't cover the cost of
cleaning up me kitchen. I need yer help
right now, matey!
"""

customer_messages = prompt_template.format_messages(
style=customer_style,
text=customer_email)

customer_response = chat(customer_messages)

print(customer_response.content)

 

 

출력:

 I'm really frustrated that my blender lid flew off and made a mess of my kitchen walls with smoothie. To add to my frustration, the warranty doesn't cover the cost of cleaning up my kitchen. Can you please help me out, friend?

 

 

이전에 만든 prompt_template 을 재사용하는 가이드:

service_reply = """Hey there customer, \
the warranty does not cover \
cleaning expenses for your kitchen \
because it's your fault that \
you misused your blender \
by forgetting to put the lid on before \
starting the blender. \
Tough luck! See ya!
"""

service_style_pirate = """\
a polite tone \
that speaks in English Pirate\
"""

service_messages = prompt_template.format_messages(
    style=service_style_pirate,
    text=service_reply)

service_response = chat(service_messages)
print(service_response.content)

 

 

출력:

Ahoy there, matey! I must kindly inform ye that the warranty be not coverin' the expenses o' cleaning yer galley, as 'tis yer own fault fer misusin' yer blender by forgettin' to put the lid on afore startin' it. Aye, tough luck! Farewell and may the winds be in yer favor!

 

 

LangChain 의 Parser 사용 가이드

output_parser 없이 LLM 에게 그냥 JSON 포맷으로 출력해달라고 말할 수도 있음. 그러나 이 경우에 결과값은 문자열이 될거임.

 

원하는 결과 포맷:

{
  "gift": False,
  "delivery_days": 5,
  "price_value": "pretty affordable!"
}

 

 

output_parsre 없이 ChatGPT 에게 위의 형식으로 결과를 달라고 하는 예시:

customer_review = """\
This leaf blower is pretty amazing.  It has four settings:\
candle blower, gentle breeze, windy city, and tornado. \
It arrived in two days, just in time for my wife's \
anniversary present. \
I think my wife liked it so much she was speechless. \
So far I've been the only one using it, and I've been \
using it every other morning to clear the leaves on our lawn. \
It's slightly more expensive than the other leaf blowers \
out there, but I think it's worth it for the extra features.
"""

review_template = """\
For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else? \
Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the product \
to arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,\
and output them as a comma separated Python list.

Format the output as JSON with the following keys:
gift
delivery_days
price_value

text: {text}
"""

prompt_template = ChatPromptTemplate.from_template(review_template)

messages = prompt_template.format_messages(text=customer_review)
chat = ChatOpenAI(temperature=0.0, model=llm_model)
response = chat(messages)
print(response.content)

 

 

출력:

• 겉으로는 JSON 형식이긴 하나, 문자열 타입임

{
  "gift": true,
  "delivery_days": 2,
  "price_value": ["It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."]
}

 

 

output_parser 가이드:

• 원하는 Schema 를 선언해주고, 설명을 넣어주고, 어떤 결과값이 오는지 알려줘야한다.
• 만든 Schema 를 StructuredOutputParser 에게 던져줘서 output_parser 를 생성한다.
• output_parser 는 get_format_instructions() 메소드를 통해 원하는 응답형식을 달라고 명령을 내리는 Prompt 를 생성할 수 있다.

from langchain.output_parsers import ResponseSchema
from langchain.output_parsers import StructuredOutputParser

gift_schema = ResponseSchema(name="gift",
                             description="Was the item purchased\
                             as a gift for someone else? \
                             Answer True if yes,\
                             False if not or unknown.")
delivery_days_schema = ResponseSchema(name="delivery_days",
                                      description="How many days\
                                      did it take for the product\
                                      to arrive? If this \
                                      information is not found,\
                                      output -1.")
price_value_schema = ResponseSchema(name="price_value",
                                    description="Extract any\
                                    sentences about the value or \
                                    price, and output them as a \
                                    comma separated Python list.")

response_schemas = [gift_schema, 
                    delivery_days_schema,
                    price_value_schema]

output_parser = StructuredOutputParser.from_response_schemas(response_schemas)

format_instructions = output_parser.get_format_instructions()

print(format_instructions)

 

 

format_instruction 출력:

he output should be a markdown code snippet formatted in the following schema, 
including the leading and trailing "\`\`\`json" and "\`\`\`":

'''json
{
    "gift": string  // Was the item purchased as a gift for someone else?                              Answer True if yes,                             False if not or unknown.
    "delivery_days": string  // How many days did it take for the product                                      to arrive? If this                                       information is not found,                                      output -1.
    "price_value": string  // Extract any sentences about the value or                                     price, and output them as a                                     comma separated Python list.
}
'''

 

 

prompt_template 에서 output_parser 를 이용하는 가이드:

• 최종적으로 ChatGPT 에게 받아온 response 를 outut_parser.pasre() 메소드로 던지면 원하는 형식의 응답을 가져올 수 있다.

review_template_2 = """\
For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else?
Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the product
to arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,
and output them as a comma separated Python list.

text: {text}

{format_instructions}
"""

prompt = ChatPromptTemplate.from_template(template=review_template_2)

messages = prompt.format_messages(text=customer_review,
format_instructions=format_instructions)

response = chat(messages)

output_dict = output_parser.parse(response.content)

print(messages[0].content)
print(response.content)

output_dict
output_dict.get('delivery_days')

 

 

최종 출력되는 프롬포트 형식:

For the following text, extract the following information:

gift: Was the item purchased as a gift for someone else? Answer True if yes, False if not or unknown.

delivery_days: How many days did it take for the productto arrive? If this information is not found, output -1.

price_value: Extract any sentences about the value or price,and output them as a comma separated Python list.

text: This leaf blower is pretty amazing.  It has four settings:candle blower, gentle breeze, windy city, and tornado. It arrived in two days, just in time for my wife's anniversary present. I think my wife liked it so much she was speechless. So far I've been the only one using it, and I've been using it every other morning to clear the leaves on our lawn. It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features.


The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "\`\`\`json" and "\`\`\`":

'''json
{
    "gift": string  // Was the item purchased                             as a gift for someone else?                              Answer True if yes,                             False if not or unknown.
    "delivery_days": string  // How many days                                      did it take for the product                                      to arrive? If this                                       information is not found,                                      output -1.
    "price_value": string  // Extract any                                    sentences about the value or                                     price, and output them as a                                     comma separated Python list.
}
'''

 

 

ChatGPT 에게 받아온 결과 출력:

{
    "gift": true,
    "delivery_days": "2",
    "price_value": ["It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."]
}

 

 

output_parser 의 parse 결과 출력:

{'gift': True,
 'delivery_days': '2',
 'price_value': ["It's slightly more expensive than the other leaf blowers out there, but I think it's worth it for the extra features."]}

 

 

Memory

LangChain 에서는 대화 내역을 기록하기 위한 Memory 를 사용한다.

 

LangChain 에서는 여러가지 메모리 타입을 지원함:

• ConservationBufferMemory: 대화 내역을 모두 메모리에 저장하는 기본적인 방식
• ConservationBufferWindowMemory: ConservationBufferMemory 의 문제점은 Token 의 수가 점점 늘어난다는 거임. 토큰 수가 증가함에 따라서 ChatGPT 비용또한 증가할거라서 이 문제를 해결해야함. 이를 위한 기법으로 ConservationBufferWindowMemory 는 최근 몇 개의 대화 내역만 기록하는 방법임.
• ConservationTokenBufferMemory: 이 메모리 타입은 명시한 토큰의 수까지만 저장함. 이 수를 초과한 오래된 대화 내역은 삭제됨.
• ConservationSummaryMemory: 이 방법 또한 Token 제한이 있다. 하지만 토큰의 수를 초과한 대화 내역은 자동적으로 요약되서 저장된다.
• VectorDatamemory: text 를 Vector database 에 저장해놓고 이를 대화할 때 가져와서 사용하는 기법임.
• EntityMemory: LLM 이 구체적인 사람이나 사물과 같은 엔터티를 기억하기 위해 사용하는 메모리 타입이라고 함.

 

ConservationBufferMemory 사용 가이드:

• ConservationBufferMemory() 를 명시적으로 만듬.
• ConservationChain 을 통해 대화를 할 수 있음. 여기에 사용할 LLM 과 Memory 타입 그리고 verbose 여부를 명시해줘야함.
• verbose 값을 false 로 주면 대화 내역은 기록되지 않음. true 로 주면 기록됨.
• conservation.predict() 메소드를 통해서 대화를 시작할 수 있음.

from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

llm = ChatOpenAI(temperature=0.0, model=llm_model)
memory = ConversationBufferMemory()
conversation = ConversationChain(
    llm=llm, 
    memory = memory,
    verbose=True
)

conversation.predict(input="Hi, my name is Andrew")
conversation.predict(input="What is 1+1?")
conversation.predict(input="What is my name?")

 

 

conservation.predict 출력:

> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
Human: Hi, my name is Andrew
AI: Hello Andrew, it's nice to meet you. My name is AI. How can I assist you today?
Human: Hi, my name is Andrew
AI:

> Finished chain.
'Yes, you already told me that Andrew. Is there something specific you need help with?'

 

 

대화 내역 보기:

• load_memory_variables() 메소드를 통해서 메모리에 저장된 대화 내역을 볼 수 있음.

memory.load_memory_variables({})

 

 

대화 내역 출력:

{'history': "Human: Hi, my name is Andrew\nAI: Hello Andrew, it's nice to meet you. My name is AI. How can I assist you today?\nHuman: Hi, my name is Andrew\nAI: Yes, you already told me that Andrew. Is there something specific you need help with?\nHuman: What is 1+1?\nAI: The answer to 1+1 is 2. Is there anything else you would like me to calculate for you?\nHuman: What is my name?\nAI: Your name is Andrew, as you mentioned earlier. Is there anything else you would like me to assist you with?"}

 

 

memory 에서 임의로 대화 내역을 기록하는 방법:

• save_context() 메소드를 호출하면 된다.

memory.save_context({"input": "Hi"}, 
                  {"output": "What's up"})

 

 

ConversationBufferWindowMemory 사용 가이드:

• ConversationBufferWindowMemory 를 생성할 때 명시한 k 값으로 윈도우 사이즈를 지정할 수 있음.

from langchain.memory import ConversationBufferWindowMemory

memory = ConversationBufferWindowMemory(k=1)               

memory.save_context({"input": "Hi"},
                    {"output": "What's up"})
memory.save_context({"input": "Not much, just hanging"},
                    {"output": "Cool"})

memory.load_memory_variables({})

 

 

출력 예시:

• 윈도우 사이즈가 1이므로 가장 최근 대화만 저장됨.

{'history': 'Human: Not much, just hanging\nAI: Cool'}

 

 

ConversationTokenBufferMemory 사용 가이드:

• ConversationTokenBufferMemory 를 생성할 때 명시한 max_token_limit 값만큼 최대 토큰 수가 지정됨. 이 토큰 수를 넘으면 오래된 대화 내역은 잘림. 그리고 llm 도 파라미터로 지정할 수 있음. 이건 LLM 마다 토큰 세는 방법이 달라서 그렇다고 함.

from langchain.memory import ConversationTokenBufferMemory
from langchain.llms import OpenAI
llm = ChatOpenAI(temperature=0.0, model=llm_model)

memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=50)
memory.save_context({"input": "AI is what?!"},
                    {"output": "Amazing!"})
memory.save_context({"input": "Backpropagation is what?"},
                    {"output": "Beautiful!"})
memory.save_context({"input": "Chatbots are what?"}, 
                    {"output": "Charming!"})

memory.load_memory_variables({})

 

 

토큰 수가 잘리게 출력됨:

{'history': 'AI: Amazing!\nHuman: Backpropagation is what?\nAI: Beautiful!\nHuman: Chatbots are what?\nAI: Charming!'}

 

 

ConversationSummaryMemory 사용 가이드:

• 이것 또한 ConversationSummaryBufferMemory 를 생성할 때 llm 과 max_token_limit 을 지정해줘야한다. 이 토큰 수를 초과할 경우 요약되서 메모리에 기록됨.

from langchain.memory import ConversationSummaryBufferMemory

# create a long string
schedule = "There is a meeting at 8am with your product team. \
You will need your powerpoint presentation prepared. \
9am-12pm have time to work on your LangChain \
project which will go quickly because Langchain is such a powerful tool. \
At Noon, lunch at the italian resturant with a customer who is driving \
from over an hour away to meet you to understand the latest in AI. \
Be sure to bring your laptop to show the latest LLM demo."

memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=100)
memory.save_context({"input": "Hello"}, {"output": "What's up"})
memory.save_context({"input": "Not much, just hanging"},
                    {"output": "Cool"})
memory.save_context({"input": "What is on the schedule today?"}, 
                    {"output": f"{schedule}"})

memory.load_memory_variables({})

 

 

요약 출력:

{'history': "System: The human and AI engage in small talk before discussing the day's schedule. The AI informs the human of a morning meeting with the product team, time to work on the LangChain project, and a lunch meeting with a customer interested in the latest AI developments."}

 

 

Chain

LangChain 에서 Chain 은 여러개의 LLM 과 Prompt 를 결합해서 응답을 내놓을 수 있도록 하는거임.

 

그러니까 하나의 언어 모델에 입력한 프롬포트의 결과를 다른 언어 모델의 입력으로 넣는 식으로 해서 최종적인 결과를 낼 수 있도록 하는거지.

 

하나의 완성된 응답을 받기 위해 Chaining Prompts 를 사용하는 걸 추상화한 거라고 보면 됨.

• Chaining Prompts 는 프롬포트를 여러번 입력해서 최종 결과를 얻는 방법임. 주로 복잡한 처리 작업에 사용되었던 거.

 

LLMChain

LangChain 에서 다른 Chain 에 사용되는 가장 기본적인 Chain 이 LLMChain 임.

 

LLMChain 을 엮어서 복잡한 체인을 만들어낸다고 알면된다.

 

LLMChain 가이드:

• LLMChain 을 만들 때도 사용할 LLM 과 Prompt 를 넣어주면 된다.
• chain.run() 메소드를 통해서 LLM 에게 질의하고 결과를 가져올 수 있다.

from langchain.chat_models import ChatOpenAI 
from langchain.prompts import ChatPromptTemplate 
from langchain.chains import LLMChain

llm = ChatOpenAI(temperature=0.9, model=llm_model)

prompt = ChatPromptTemplate.from_template(
"What is the best name to describe
a company that makes {product}?"
)

chain = LLMChain(llm=llm, prompt=prompt)

product = "Queen Size Sheet Set"
chain.run(product)

 

 

출력: 

'QueenSheet Co.'

 

 

SimpleSequentialChain

SequentialChain 은 하나의 LLMChain 의 결과값이 다음 LLMChain 의 input 으로 전달시켜주는 chain 을 말한다.

• 물론 둘 다 같은 LLM 을 이용할 수 도 있다.

 

SimpleSequentialChain 에서 주의할 건 체인을 하나씩만 연결시킬 수 있다는 거임. 그러니까 두 개의 LLMChain 에게서 받아온 결과를 다른 LLMChain 의 입력으로 전달하는게 불가능. 오로지 하나만.

 

다음 이미지를 참고하면 더 SimpleSequentialChain 을 이해하기 쉬울 것

 

 

SimpleSequentialChain 가이드:

• 두 개의 LLMChain 을 만들고 이를 SimpleSequentialChain 으로 연결시킴.
• 보면 각각의 LLMChain 마다 프롬포트와 LLM 을 다르게 지정하는 것도 가능하다.
• 두 번째 LLMChain 의 company_name 으로 첫 번째 LLMChain 의 결과값이 전달된다.

from langchain.chains import SimpleSequentialChain

llm = ChatOpenAI(temperature=0.9, model=llm_model)

# prompt template 1
first_prompt = ChatPromptTemplate.from_template(
    "What is the best name to describe \
    a company that makes {product}?"
)

# Chain 1
chain_one = LLMChain(llm=llm, prompt=first_prompt)

# prompt template 2
second_prompt = ChatPromptTemplate.from_template(
    "Write a 20 words description for the following \
    company:{company_name}"
)
# chain 2
chain_two = LLMChain(llm=llm, prompt=second_prompt)

overall_simple_chain = SimpleSequentialChain(chains=[chain_one, chain_two],verbose=True)

overall_simple_chain.run(product)

 

 

출력:

'"Royal Linens offers luxurious bedding and bath linens fit for a king or queen, with exquisite attention to quality and detail."'

 

 

SequentialChain

SimpleSeqeuntialChain 보다 좀 더 복잡한 체인임.

 

여러 LLMChain 의 결과를 다음 LLMChain 의 입력으로 전달해줄 수 있다.

 

이미지로는 다음과 같음:

 

 

SequentialChain 가이드:

• 4개의 LLMChain 이 있고 이게 SequentialChain 으로 연결된다.
• 중요한 건 하나의 LLMChain 을 만들 때마다 결과값을 output_key 로 지정한다는 것. 이 output_key 가 다음 LLMCahin 의 프롬포트에 있는 placeholder 와 매핑되는 거다. first chain 과 seconde chain 을 보면 English_Review 로 연결됨.
• SequentialChain 의 output_variables 로 최종 결과값은 무슨 키로 가져올 건지 명시할 수 있다. 중간 결과값도 가져올 수 있음.

from langchain.chains import SequentialChain

llm = ChatOpenAI(temperature=0.9, model=llm_model)

# prompt template 1: translate to english
first_prompt = ChatPromptTemplate.from_template(
    "Translate the following review to english:"
    "\n\n{Review}"
)
# chain 1: input= Review and output= English_Review
chain_one = LLMChain(llm=llm, prompt=first_prompt, output_key="English_Review")

second_prompt = ChatPromptTemplate.from_template(
    "Can you summarize the following review in 1 sentence:"
    "\n\n{English_Review}"
)

# chain 2: input= English_Review and output= summary
chain_two = LLMChain(llm=llm, prompt=second_prompt, output_key="summary")

# prompt template 3: translate to english
third_prompt = ChatPromptTemplate.from_template(
    "What language is the following review:\n\n{Review}"
)
# chain 3: input= Review and output= language
chain_three = LLMChain(llm=llm, prompt=third_prompt, output_key="language")

# prompt template 4: follow up message
fourth_prompt = ChatPromptTemplate.from_template(
    "Write a follow up response to the following "
    "summary in the specified language:"
    "\n\nSummary: {summary}\n\nLanguage: {language}"
)
# chain 4: input= summary, language and output= followup_message
chain_four = LLMChain(llm=llm, prompt=fourth_prompt, output_key="followup_message")

# overall_chain: input= Review 
# and output= English_Review,summary, followup_message
overall_chain = SequentialChain(
    chains=[chain_one, chain_two, chain_three, chain_four],
    input_variables=["Review"],
    output_variables=["English_Review", "summary","followup_message"],
    verbose=True
)

review = df.Review[5]
overall_chain(review)

 

 

출력:

{'Review': "Je trouve le goût médiocre. La mousse ne tient pas, c'est bizarre. J'achète les mêmes dans le commerce et le goût est bien meilleur...\nVieux lot ou contrefaçon !?",
 'English_Review': 'I find the taste mediocre. The foam does not hold up, which is strange. I buy the same ones in stores and the taste is much better... Old batch or counterfeit!?',
 'summary': 'The reviewer finds the taste of the product mediocre and suspects that it might be an old batch or counterfeit as the foam does not hold up.',
 'followup_message': "Réponse : Le critique trouve le goût du produit médiocre et suspecte qu'il pourrait s'agir d'un lot ancien ou contrefait car la mousse ne tient pas."}

 

 

Router Chain

Router Chain 은 LLM 의 결과값이 여러가지 종류가 될 수 있을 때 해당 결과값에 맞는 프롬포트를 실행시키기 위해 사용되는 Chain 이다.

이전보다 약간 더 복잡함.

 

RouterChain 가이드:

• 각 상황마다 실행될 수 있는 여거 프롬포트를 만들고 이 프롬포트에 해당하는 LLMChain 을 파이썬 딕셔너리 형식으로 만들어야한다. 이를 통해서 각 결과마다 다른 LLMChain 이 실행될 수 있도록 하는거임.
• 만약 원하는 결과가 없을수도 있기 때문에 Defaut LLMChain 도 만들어놔야함.
• MULTI_PROMPT_ROUTER_TEMPLATE 를 이용해서 Output 으로 destionation 과 next_input 을 JSON 형식으로 내달라고 해야한다.
  • 이 destination 에는 다음에 실행될 LLMChain 의 이름이 들어갈거고, next_input 에는 해당 LLMChain 에 들어갈 input 이 들어갈거임.
• router_prompt 를 만들 때 명시한 RouterOutputParser 가 (destination, next_input) 결과를 파싱해서 다음 LLMChain 에게 전달해줄거임.
• MultiPromptChain 을 만들 때는 라우팅을 해줄 RouterChain 과 이 Chain 의 결과에 따라 실행될 Chain 들을 생성자로 넣어서 만들어서 사용하면 된다. 이떄 기본 DefaultChain 도 넣어줘야함.

physics_template = """You are a very smart physics professor. \
You are great at answering questions about physics in a concise\
and easy to understand manner. \
When you don't know the answer to a question you admit\
that you don't know.

Here is a question:
{input}"""


math_template = """You are a very good mathematician. \
You are great at answering math questions. \
You are so good because you are able to break down \
hard problems into their component parts, 
answer the component parts, and then put them together\
to answer the broader question.

Here is a question:
{input}"""

history_template = """You are a very good historian. \
You have an excellent knowledge of and understanding of people,\
events and contexts from a range of historical periods. \
You have the ability to think, reflect, debate, discuss and \
evaluate the past. You have a respect for historical evidence\
and the ability to make use of it to support your explanations \
and judgements.

Here is a question:
{input}"""


computerscience_template = """ You are a successful computer scientist.\
You have a passion for creativity, collaboration,\
forward-thinking, confidence, strong problem-solving capabilities,\
understanding of theories and algorithms, and excellent communication \
skills. You are great at answering coding questions. \
You are so good because you know how to solve a problem by \
describing the solution in imperative steps \
that a machine can easily interpret and you know how to \
choose a solution that has a good balance between \
time complexity and space complexity. 

Here is a question:
{input}"""

prompt_infos = [
    {
        "name": "physics", 
        "description": "Good for answering questions about physics", 
        "prompt_template": physics_template
    },
    {
        "name": "math", 
        "description": "Good for answering math questions", 
        "prompt_template": math_template
    },
    {
        "name": "History", 
        "description": "Good for answering history questions", 
        "prompt_template": history_template
    },
    {
        "name": "computer science", 
        "description": "Good for answering computer science questions", 
        "prompt_template": computerscience_template
    }
]

from langchain.chains.router import MultiPromptChain
from langchain.chains.router.llm_router import LLMRouterChain,RouterOutputParser
from langchain.prompts import PromptTemplate

llm = ChatOpenAI(temperature=0, model=llm_model)


destination_chains = {}
for p_info in prompt_infos:
    name = p_info["name"]
    prompt_template = p_info["prompt_template"]
    prompt = ChatPromptTemplate.from_template(template=prompt_template)
    chain = LLMChain(llm=llm, prompt=prompt)
    destination_chains[name] = chain  

destinations = [f"{p['name']}: {p['description']}" for p in prompt_infos]
destinations_str = "\n".join(destinations)

default_prompt = ChatPromptTemplate.from_template("{input}")
default_chain = LLMChain(llm=llm, prompt=default_prompt)

MULTI_PROMPT_ROUTER_TEMPLATE = """Given a raw text input to a \
language model select the model prompt best suited for the input. \
You will be given the names of the available prompts and a \
description of what the prompt is best suited for. \
You may also revise the original input if you think that revising\
it will ultimately lead to a better response from the language model.

<< FORMATTING >>
Return a markdown code snippet with a JSON object formatted to look like:
### json
{{{{
    "destination": string \ name of the prompt to use or "DEFAULT"
    "next_inputs": string \ a potentially modified version of the original input
}}}}
### 

REMEMBER: "destination" MUST be one of the candidate prompt \
names specified below OR it can be "DEFAULT" if the input is not\
well suited for any of the candidate prompts.
REMEMBER: "next_inputs" can just be the original input \
if you don't think any modifications are needed.

<< CANDIDATE PROMPTS >>
{destinations}

<< INPUT >>
{{input}}

<< OUTPUT (remember to include the ```json)>>"""

router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(
    destinations=destinations_str
)
router_prompt = PromptTemplate(
    template=router_template,
    input_variables=["input"],
    output_parser=RouterOutputParser(),
)

router_chain = LLMRouterChain.from_llm(llm, router_prompt)

chain = MultiPromptChain(router_chain=router_chain, destination_chains=destination_chains, default_chain=default_chain, verbose=True)

chain.run("What is black body radiation?")

 

 

출력:

> Entering new MultiPromptChain chain...
physics: {'input': 'What is black body radiation?'}
> Finished chain.
"Black body radiation refers to the electromagnetic radiation emitted by a perfect black body, which is an object that absorbs all radiation that falls on it and emits radiation at all wavelengths. The radiation emitted by a black body depends only on its temperature and follows a specific distribution known as Planck's law. This type of radiation is important in understanding the behavior of stars, as well as in the development of technologies such as incandescent light bulbs and infrared cameras."

 

 

Question and Answer

LLM 에게 질문을 할 때 Prompt 와 함께 Context 를 같이 던져서 주는 방법이 있다.

 

뭐 예를 들면 PDF 파일에 있는 연관된 내용을 빠르게 찾아서 LLM 의 Context 로 던져서 질문을 하는 방법임.

 

이 방법은 PDF 파일 통째로 LLM 에게 질문으로 던지는게 아님. (이렇게 대규모로 Input 을 넣게되면 성능이 잘 나오지 않으니까 이렇게 하면 안됨)

• LLM 은 Input + Output 을 합쳐서 Token 으로 계산해서 답변을 냄. Input 이 지나치게 크면 Output 이 작게되서 성능이 안나오니까.

이 방법은 Document 내용을 Vector Store 에 임베딩해서 저장해놓고, 사용자가 질문을 하면 Document 에 연관된 내용을 Vector Store 에서 검색해서 Context 로 사용해서 LLM 에게 질문하는 방법이다.

 

엄청 파워풀한 방법임.

 

 

Q) 큰 Document 를 Vector Store 에 저장할 때 Document 에 있는 문장들은 Chunk 단위로 쪼개지고 이게 Embedding 에 의해 vector 로 변환된 후 Vector Store 에 저장되는거야?

 

맞다.

 

큰 문서를 벡터 스토어에 저장할 때의 일반적인 절차는 문서를 더 작은 단위인 'chunks'로 나누고, 이를 각각 임베딩하여 벡터 형태로 변환한 뒤 벡터 스토어에 저장된다.

 

 

Q) 만약에 Chunk 가 단어로 쪼개져서 Vector Store 에 저장된다면 완벽한 문장을 검색하는건 힘든거지?

 

Chunk 가 단어 단위로 쪼개져서 벡터로 저장되기만 하면 문장 단위로 검색하는게 힘듬.

 

이는 각 단어가 독립적으로 저장되어 문맥적 연결성이 부족하기 때문임.

 

각 단어는 그 자체로 의미를 가질 수 있지만, 문장을 구성하는 더 큰 의미의 흐름이나 문맥적 뉘앙스를 포착하기 어렵.

 

 

Q) 궁금한게 그럼 Chunk 단위로 쪼개지는데 내가 저장한 Document 에 있는 문장을 질문하면 어떻게 완벽한 문장을 가지고 올 수 있는거야? Chunk 단위로 가지고 오기 때문에 완벽만 문장을 가지고 오기 힘들거 같은데?

 

이는 Document 가 임베딩 될 때 무슨 일이 발생하는지 이해하면 된다:

• 텍스트 전처리: 먼저, 입력된 문자열은 전처리 과정을 거친다. 이 단계에서는 텍스트를 표준화하기 위해 소문자화, 불용어 제거, 문장 부호 제거, 토크나이징(단어나 구문으로 분리) 등의 작업이 이루어짐.
• 토큰화: 전처리된 텍스트는 개별 단어나 구(phrase)로 분리된다. 이 토큰들은 나중에 벡터로 변환될 기본 단위가 된다.
• 벡터 변환: 각 토큰은 사전에 훈련된 언어 모델을 사용하여 벡터로 변환된다. 이러한 모델들은 단어의 의미적, 문맥적 속성을 수치화한 고차원의 벡터 공간에 매핑된다. 예를 들어, Word2Vec, GloVe, BERT 등의 모델이 사용될 수 있다.
• 벡터 통합: 여러 토큰으로부터 생성된 벡터들은 종종 하나의 고정된 길이의 벡터로 통합된다. 이는 문장 또는 전체 문자열의 의미를 하나의 벡터로 표현하기 위함이다. 이 통합 과정은 평균화, 최대 풀링, 또는 문맥을 고려하는 어텐션 메커니즘과 같은 방법으로 이루어질 수 있다.
• 차원 축소: 생성된 벡터는 때때로 매우 고차원일 수 있기 때문에 필요에 따라 차원 축소 기법(예: PCA, t-SNE)을 적용하여 벡터의 차원을 줄이고, 계산 효율성을 높이거나 데이터의 시각화를 용이하게 할 수 있다.
• 후처리: 벡터화 과정을 거친 후, 이 벡터는 다양한 자연어 처리 작업에 사용될 수 있다. 예를 들어, 문서 분류, 감정 분석, 유사도 측정, 정보 검색 등이 해당된다.

 

벡터 통합으로 인해서 문장 단위로 검색이 가능한거다.

 

그리고 Vector Store 는 내부 기술마다 다르겠지만 문맥 연결 정보를 저장해서 각 청크에 대한 추가 메타데이터를 저장하여 청크 간의 연결성을 추적할 수도 있다.

• 예를 들어, 각 청크가 원본 문서에서 어디에 위치하는지, 인접한 청크는 무엇인지 등의 정보를 포함시키는 것.

 

또 청크 병합 알고리즘을 통해 여러 청크가 반환될 경우, 이를 병합하여 완전한 문장이나 문서를 재구성할 수 있는 알고리즘을 적용하기도 한다. 이는 각 청크의 문맥 정보와 메타데이터를 활용해서 실행되는거임.

 

 

Q) Vector Store 데이터베이스 기술마다 Chunk 를 쪼개고, 문맥 정보를 저장하고 연결하고 병합하고, 인덱싱 하는 알고리즘이 다른거야?

 

맞다.

 

 

Q) 큰 문서를 임베딩할 때 사이즈가 줄어드는 이유가 토큰화되서 벡터로 표현되는데 이때 중복 단어는 제거가 되기 때문이겠네?

 

일부 맞다.

 

다른 이유도 있는데 큰 문서를 임베딩할 때 사이즈가 줄어드는 주요 이유는 문서 내용이 고차원 벡터로 압축되기 때문임.

 

즉 단어나 구가 벡터로 변환된다는 것 자체가 사이즈가 줄어들기도 함.

 

 

Q) Vector Store 에서 검색된 Document 가 많다면 이를 LLM 에게 하나의 Context 로 질문하게 되면 성능이 안나올 것 같은데 이건 어떻게 해결함?

 

여러가지 방법이 있다:

• Map-Reduce: 각 Document 마다 LLM 에게 질문을 병렬로 날리고, 마지막 LLM 이 요약하는 방법. (실제로 요약 처리에 적합하다)
• Refine: 순자척으로 각 Document 마다 LLM 에게 질문을 날림. 이전 답변이 다음 LLM 질문에 포함되서 최종적인 답변을 얻는 구조임. (복잡한 문제나 여러 정보 출처를 통합해야 하는 상황에서 주로 쓰임.)
• Map-ReRank: Map Reduce 처럼 각 Document 마다 LLM 에게 질문을 병렬로 날리고 여기서 가장 Score 가 높은 답변을 선택해서 답변을 줌. (검색 엔진과 같은 것)

 

 

간단하게 Vector Store 를 사용하는 가이드:

• VectorstoreIndexCreator 를 통해서 Vector Store 를 만듬.
• Vector Store 를 만들 때 DocArrayInMemorySearch 를 사용했는데, 이건 간단하게 메모리 레벨에서 Vector Store 를 사용하는거임. 외부 데이터베이스 사용이 아니라.
• VectorStoreIndexCreatoe 에게 직접 query 와 LLM 를 전달해서 답변을 얻는 방법임.

from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import CSVLoader
from langchain.vectorstores import DocArrayInMemorySearch
from IPython.display import display, Markdown
from langchain.llms import OpenAI

file = 'OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file)

from langchain.indexes import VectorstoreIndexCreator

index = VectorstoreIndexCreator(
    vectorstore_cls=DocArrayInMemorySearch
).from_loaders([loader])

query ="Please list all your shirts with sun protection \
in a table in markdown and summarize each one."

llm_replacement_model = OpenAI(temperature=0, 
                               model='gpt-3.5-turbo-instruct')

response = index.query(query, llm = llm_replacement_model)

display(Markdown(response))

 

 

임베딩 사용해보는 가이드:

from langchain.document_loaders import CSVLoader
loader = CSVLoader(file_path=file)

docs = loader.load()

from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()

embed = embeddings.embed_query("Hi my name is Harrison")

print(len(embed))
print(embed[:5])

 

 

출력:

1536
[-0.02199380286037922, 0.006747527979314327, -0.018252847716212273, -0.039167046546936035, -0.013997197151184082]

 

 

Vector Store 검색 가이드:

• similarity_search() 메소드를 통해서 해당 주어진 쿼리와 유사한 결과를 검색할 수 있다.

db = DocArrayInMemorySearch.from_documents( docs, embeddings )

query = "Please suggest a shirt with sunblocking"

docs = db.similarity_search(query)

len(docs)
docs[0]

 

 

출력: 

4
Document(page_content=': 255\nname: Sun Shield Shirt by\ndescription: "Block the sun, not the fun – our high-performance sun shirt is guaranteed to protect from harmful UV rays. \n\nSize & Fit: Slightly Fitted: Softly shapes the body. Falls at hip.\n\nFabric & Care: 78% nylon, 22% Lycra Xtra Life fiber. UPF 50+ rated – the highest rated sun protection possible. Handwash, line dry.\n\nAdditional Features: Wicks moisture for quick-drying comfort. Fits comfortably over your favorite swimsuit. Abrasion resistant for season after season of wear. Imported.\n\nSun Protection That Won\'t Wear Off\nOur high-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun\'s harmful rays. This fabric is recommended by The Skin Cancer Foundation as an effective UV protectant.', metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 255})

 

 

LangChain 에서 Vector Store 와 LLM 사용 가이드:

• chain_type 은 여러개가 있겠지만 stuff 가 가장 기본적인 타입임. Vector Store 의 결과를 하나의 Context ㄹ 만들어서 조회하는 것.

retriever = db.as_retriever()

qa_stuff = RetrievalQA.from_chain_type(
llm=llm,
chain_type="stuff",
retriever=retriever,
verbose=True
)

query = "Please list all your shirts with sun protection in a table
in markdown and summarize each one."

response = qa_stuff.run(query)

 

 

LangChain Vector Store 가이드의 추상화를 풀어내면 이 과정임: 

• Vector Store 검색 내용을 하나로 합친 후, Prompt 안에 삽입한는 거임. 

retriever = db.as_retriever()

llm = ChatOpenAI(temperature = 0.0, model=llm_model)

qdocs = "".join([docs[i].page_content for i in range(len(docs))])

response = llm.call_as_llm(f"{qdocs} Question: Please list all your \
shirts with sun protection in a table in markdown and summarize each one.")

 

 

Evaluation

LangChain 어플리케이션이 잘 작동하는지 평가하는 방법을 다룸. 전통적인 소프트웨어 기준으로는 테스트 코드를 작성하는 것과 유사하다.

 

기존에 테스트 하는 방법은 질문에 대한 답변 쌍을 인간이 생성하고, LLM 에게 예측을 한 이후 LLM 의 예측이 답변과 맞는지 비교하는 방법을 하거나, 아니면 LLM 의 예측이 답변의 조건들이 충족하는지 검증하는 식으로 평가를 했었음.

 

LangChain 에서는 좀 더 쉽게 LLM 을 이용해서 질문과 답변을 생성할 수 있도록 하는 기능들을 제공해주고, 웹 UI 환경도 제공해주고, 디버깅 기능도 제공해준다.

 

다만 LLM 을 이용해서 질문과 답변을 생성할 때는 좀 주의할 점이 있어보임.

• 하나의 LLM 으로 질문과 답변을 생성했는데, 같은 LLM 으로 예측을 수행한다면 이게 의미가 있을까? 그냥 재현할 뿐이라고 생각한다. 별 의미 없다고 생각함. 그래서 최소한 답변인 그라운드 트루스 같은 건 인간이 검증을 빡세게 하는게 중요하다고 생각함.
• 다만 이런 기능 자체가 의미가 없지는 않은 것 같다. 편리한 기능이 있기 때문에.

 

그리고 LLM 을 가지고 Evaluation 할 때는 정확한 문자열 exact maching 은 어렵다고 생각해야함. 똑같은 단어들을 그대로 써서 답변을 작성하지는 않으니까.

 

 

Evaluation 가이드:

from langchain.evaluation.qa import QAGenerateChain

example_gen_chain = QAGenerateChain.from_llm(ChatOpenAI(model=llm_model))

new_examples = example_gen_chain.apply_and_parse(
    [{"doc": t} for t in data[:5]]
)

examples += new_examples

predictions = qa.apply(examples)

from langchain.evaluation.qa import QAEvalChain

llm = ChatOpenAI(temperature=0, model=llm_model)
eval_chain = QAEvalChain.from_llm(llm)

graded_outputs = eval_chain.evaluate(examples, predictions)

for i, eg in enumerate(examples):
    print(f"Example {i}:")
    print("Question: " + predictions[i]['query'])
    print("Real Answer: " + predictions[i]['answer'])
    print("Predicted Answer: " + predictions[i]['result'])
    print("Predicted Grade: " + graded_outputs[i]['text'])
    print()

graded_outputs[0]

 

 

출력:

Example 0:
Question: Do the Cozy Comfort Pullover Set        have side pockets?
Real Answer: Yes
Predicted Answer: Yes, the Cozy Comfort Pullover Set has pull-on pants with side pockets.
Predicted Grade: CORRECT

Example 1:
Question: What collection is the Ultra-Lofty         850 Stretch Down Hooded Jacket from?
Real Answer: The DownTek collection
Predicted Answer: The Ultra-Lofty 850 Stretch Down Hooded Jacket is from the DownTek collection.
Predicted Grade: CORRECT

Example 2:
Question: What is the weight of the Women's Campside Oxfords per pair?
Real Answer: The approximate weight of the Women's Campside Oxfords per pair is 1 lb.1 oz.
Predicted Answer: The Women's Campside Oxfords weigh approximately 1 lb. 1 oz. per pair.
Predicted Grade: CORRECT

Example 3:
Question: What are the dimensions of the small and medium Recycled Waterhog Dog Mats?
Real Answer: The small mat has dimensions of 18" x 28" and the medium mat has dimensions of 22.5" x 34.5".
Predicted Answer: The small Recycled Waterhog Dog Mat has dimensions of 18" x 28" and the medium size has dimensions of 22.5" x 34.5".
Predicted Grade: CORRECT

Example 4:
Question: What are some features of the Infant and Toddler Girls' Coastal Chill Swimsuit?
Real Answer: The swimsuit has bright colors, ruffles, and exclusive whimsical prints. It is made of four-way-stretch and chlorine-resistant fabric that keeps its shape and resists snags. The fabric is UPF 50+ rated, providing the highest rated sun protection possible, blocking 98% of the sun's harmful rays. The swimsuit has crossover no-slip straps and a fully lined bottom to ensure a secure fit and maximum coverage. It can be machine washed and line dried for best results. The swimsuit is imported.
Predicted Answer: The Infant and Toddler Girls' Coastal Chill Swimsuit is a two-piece swimsuit with bright colors, ruffles, and exclusive whimsical prints. It is made of four-way-stretch and chlorine-resistant fabric that keeps its shape and resists snags. The swimsuit has UPF 50+ rated fabric that provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage. It is machine washable and should be line dried for best results.
Predicted Grade: CORRECT

Example 5:
Question: What is the fabric composition of the Refresh Swimwear, V-Neck Tankini Contrasts?
Real Answer: The Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon with 18% Lycra® spandex. The lining is made of 90% recycled nylon with 10% Lycra® spandex.
Predicted Answer: The Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon with 18% Lycra® spandex for the body and 90% recycled nylon with 10% Lycra® spandex for the lining.
Predicted Grade: CORRECT

Example 6:
Question: What is TEK O2 technology and what benefits does it offer?
Real Answer: TEK O2 technology is a state-of-the-art technology that offers the most breathability the company has ever tested. It provides enhanced breathability and guarantees to keep you dry and comfortable in all activities and weather conditions.
Predicted Answer: TEK O2 technology is a waterproof/breathable technology that offers four-season versatility, perfect for both on or off the trail. It is designed to keep you active, with three layers of high-performance fabric that instantly move moisture away for maximum protection against the elements. The air-permeable TEK O2 technology keeps you dry and comfortable. The benefits of TEK O2 technology include breathability, waterproofing, and protection against the elements.
Predicted Grade: CORRECT

{'text': 'CORRECT'}

 

 

LLM 디버깅 가이드:

import langchain
from langchain.evaluation.qa import QAGenerateChain

langchain.debug = True

example_gen_chain = QAGenerateChain.from_llm(ChatOpenAI(model=llm_model))

qa.run(examples[0]["query"])

 

 

디버깅 출력:

[chain/start] [1:chain:RetrievalQA] Entering Chain run with input:
{
  "query": "Do the Cozy Comfort Pullover Set        have side pockets?"
}
[chain/start] [1:chain:RetrievalQA > 2:chain:StuffDocumentsChain] Entering Chain run with input:
[inputs]
[chain/start] [1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain] Entering Chain run with input:
{
  "question": "Do the Cozy Comfort Pullover Set        have side pockets?",
  "context": ": 73\nname: Cozy Cuddles Knit Pullover Set\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \n\nSize & Fit \nPants are Favorite Fit: Sits lower on the waist. \nRelaxed Fit: Our most generous fit sits farthest from the body. \n\nFabric & Care \nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features \nRelaxed fit top with raglan sleeves and rounded hem. \nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \nImported.<<<<>>>>>: 10\nname: Cozy Comfort Pullover Set, Stripe\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\n\nSize & Fit\n- Pants are Favorite Fit: Sits lower on the waist.\n- Relaxed Fit: Our most generous fit sits farthest from the body.\n\nFabric & Care\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features\n- Relaxed fit top with raglan sleeves and rounded hem.\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\n\nImported.<<<<>>>>>: 632\nname: Cozy Comfort Fleece Pullover\ndescription: The ultimate sweater fleece \u2013 made from superior fabric and offered at an unbeatable price. \n\nSize & Fit\nSlightly Fitted: Softly shapes the body. Falls at hip. \n\nWhy We Love It\nOur customers (and employees) love the rugged construction and heritage-inspired styling of our popular Sweater Fleece Pullover and wear it for absolutely everything. From high-intensity activities to everyday tasks, you'll find yourself reaching for it every time.\n\nFabric & Care\nRugged sweater-knit exterior and soft brushed interior for exceptional warmth and comfort. Made from soft, 100% polyester. Machine wash and dry.\n\nAdditional Features\nFeatures our classic Mount Katahdin logo. Snap placket. Front princess seams create a feminine shape. Kangaroo handwarmer pockets. Cuffs and hem reinforced with jersey binding. Imported.\n\n \u2013 Official Supplier to the U.S. Ski Team\nTHEIR WILL TO WIN, WOVEN RIGHT IN. LEARN MORE<<<<>>>>>: 151\nname: Cozy Quilted Sweatshirt\ndescription: Our sweatshirt is an instant classic with its great quilted texture and versatile weight that easily transitions between seasons. With a traditional fit that is relaxed through the chest, sleeve, and waist, this pullover is lightweight enough to be worn most months of the year. The cotton blend fabric is super soft and comfortable, making it the perfect casual layer. To make dressing easy, this sweatshirt also features a snap placket and a heritage-inspired Mt. Katahdin logo patch. For care, machine wash and dry. Imported."
}
[llm/start] [1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain > 4:llm:ChatOpenAI] Entering LLM run with input:
{
  "prompts": [
    "System: Use the following pieces of context to answer the users question. \nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\n: 73\nname: Cozy Cuddles Knit Pullover Set\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \n\nSize & Fit \nPants are Favorite Fit: Sits lower on the waist. \nRelaxed Fit: Our most generous fit sits farthest from the body. \n\nFabric & Care \nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features \nRelaxed fit top with raglan sleeves and rounded hem. \nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \nImported.<<<<>>>>>: 10\nname: Cozy Comfort Pullover Set, Stripe\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\n\nSize & Fit\n- Pants are Favorite Fit: Sits lower on the waist.\n- Relaxed Fit: Our most generous fit sits farthest from the body.\n\nFabric & Care\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\n\nAdditional Features\n- Relaxed fit top with raglan sleeves and rounded hem.\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\n\nImported.<<<<>>>>>: 632\nname: Cozy Comfort Fleece Pullover\ndescription: The ultimate sweater fleece \u2013 made from superior fabric and offered at an unbeatable price. \n\nSize & Fit\nSlightly Fitted: Softly shapes the body. Falls at hip. \n\nWhy We Love It\nOur customers (and employees) love the rugged construction and heritage-inspired styling of our popular Sweater Fleece Pullover and wear it for absolutely everything. From high-intensity activities to everyday tasks, you'll find yourself reaching for it every time.\n\nFabric & Care\nRugged sweater-knit exterior and soft brushed interior for exceptional warmth and comfort. Made from soft, 100% polyester. Machine wash and dry.\n\nAdditional Features\nFeatures our classic Mount Katahdin logo. Snap placket. Front princess seams create a feminine shape. Kangaroo handwarmer pockets. Cuffs and hem reinforced with jersey binding. Imported.\n\n \u2013 Official Supplier to the U.S. Ski Team\nTHEIR WILL TO WIN, WOVEN RIGHT IN. LEARN MORE<<<<>>>>>: 151\nname: Cozy Quilted Sweatshirt\ndescription: Our sweatshirt is an instant classic with its great quilted texture and versatile weight that easily transitions between seasons. With a traditional fit that is relaxed through the chest, sleeve, and waist, this pullover is lightweight enough to be worn most months of the year. The cotton blend fabric is super soft and comfortable, making it the perfect casual layer. To make dressing easy, this sweatshirt also features a snap placket and a heritage-inspired Mt. Katahdin logo patch. For care, machine wash and dry. Imported.\nHuman: Do the Cozy Comfort Pullover Set        have side pockets?"
  ]
}
[llm/end] [1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain > 4:llm:ChatOpenAI] [450.35999999999996ms] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "Yes, the Cozy Comfort Pullover Set has pull-on pants with side pockets.",
        "generation_info": null,
        "message": {
          "content": "Yes, the Cozy Comfort Pullover Set has pull-on pants with side pockets.",
          "additional_kwargs": {},
          "example": false
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {
      "prompt_tokens": 734,
      "completion_tokens": 17,
      "total_tokens": 751
    },
    "model_name": "gpt-3.5-turbo-0301"
  }
}
[chain/end] [1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain] [450.959ms] Exiting Chain run with output:
{
  "text": "Yes, the Cozy Comfort Pullover Set has pull-on pants with side pockets."
}
[chain/end] [1:chain:RetrievalQA > 2:chain:StuffDocumentsChain] [451.493ms] Exiting Chain run with output:
{
  "output_text": "Yes, the Cozy Comfort Pullover Set has pull-on pants with side pockets."
}
[chain/end] [1:chain:RetrievalQA] [705.62ms] Exiting Chain run with output:
{
  "result": "Yes, the Cozy Comfort Pullover Set has pull-on pants with side pockets."
}
'Yes, the Cozy Comfort Pullover Set has pull-on pants with side pockets.'

 

 

Agent

LangChain 에이전트는 기본적으로 언어 모델을 사용하여 다양한 작업을 수행하게 만들어줌.

 

언어 모델은 단순히 텍스트 응답을 생성하는 것을 넘어, 특정 상황에서 어떤 행동을 취해야 하는지 결정하는 '추론 엔진'으로 활용되는 걸 말함.

 

그러니까 LLM 을 사용할 때 지식 스토어로 접근하기 보다는 추론 엔진으로 접근하는게 더 타탕하다고 한다.

 

주로 Agents 는 Chains 와 비교되서 설명함:

• Chains'에서는 행동 시퀀스가 하드코드되어 있음. RouterChain 을 생각해보면 된다. 개발자가 올 수 있는 입력들을 생각해놓고 이게 오면 이 프롬포트를 실행하면 돼. 이렇게 명시해놓는 거.
• Agent 는 주어진 도구에 맞춰서 실시간으로 행동을 선택하고 순서를 결정할 수 있다고 함. 이게 가장 큰 차이임. Agent 에 필요한 도구들을 주고 질문을 하게되면 Agent 는 여러 도구들과의 상호작용을 포함하는 복잡한 워크플로우를 동적으로 생성하고 실행할 수 있게됨.

 

그냥 LLM 은 사용자의 입력에 따라 텍스트를 생성하는 것 말고는 할 수 없는데 Agent 는 주어진 도구를 따라서 검색 엔진을 통해 검색을 해볼 수도 있고, 데이터베이스에서 조회를 해볼수도 있음.

 

 

Q) LLM 이 어떻게 사용할 도구를 선택하는건데?

 

OpenAI 의 tool calling LLMs 에 의존함. 이건 LLM 을 도구 선택에 대해 파인 튜닝한 거임.

그래서 도구를 명시해놓으면 알아서 잘 선택한다고 함.

 

 

Q) LLM 을 Knowledge Store 라고 생각하지 말고 Reasoning Engine 이라고 생각하라는 말은 무슨 뜻이야?

 

단순히 정보를 저장하고 검색하는 도구가 아니라, 복잡한 문제를 해석하고 해결하는 데 사용할 수 있는 추론 도구로 봐야 한다는 것.

 

만약 LLM을 Knowledge Store로 본다면, LLM은 주로 저장된 데이터나 사실을 기반으로 답변을 생성할거다. 이 경우 모델은 입력된 질문에 가장 적절한 정보를 "기억"하고 그것을 반환하는 역할을 수행할 것.

 

하지만 LLM을 Reasoning Engine으로 보면, 이 모델은 단순히 정보를 반환하는 것을 넘어서 문제를 분석하고 복잡한 추론을 통해 새로운 결론이나 해답을 제공할 수 있는 것임. 예를 들어, 주어진 상황에 대한 설명을 바탕으로 그 원인을 추론하거나, 여러 가지 정보를 종합하여 새로운 아이디어를 제시할 수 있는 것.

 

 

Q) LLM 이 추론을 잘하게 만들려면 답변을 작성하기 위해서 거쳐야하는 Step 을 명확히 제시해주는게 중요하곘네?

 

맞다. 이런 프롬포트 기법도 중요함.

 

일반적인 Step 을 명시하는 가이드는 다음과 같음:

• 문제 이해하기: 질문의 핵심 요소와 맥락을 정확하게 파악하는 것. 이 단계에서는 질문의 주요 용어와 구조를 분석하고, 요구하는 답변의 유형을 이해해야한다.
• 관련 정보 수집: 필요한 정보를 수집하고 조사하는 것. 이 정보는 직접적인 지식 뿐만 아니라 관련된 사례, 이론, 데이터 등을 포함할 수 있다.
• 논리 구조 개발: 수집한 정보를 바탕으로 답변을 구성하기 위한 논리적 구조를 개발하는 것. 이 과정에서 추론 방식(연역적, 귀납적, 비교적 등)을 선택하고, 답변의 방향을 정할 수 있다.
• 답변 초안 작성: 논리 구조에 따라 답변의 초안을 작성하는 것. 이 단계에서는 각 주장을 지지하는 근거를 명확히 하고, 논리적 연결고리를 확인해야한다.
• 검토 및 수정: 초안을 검토하고 필요한 수정을 진행하는 것. 이 단계에서는 답변의 정확성, 완성도, 그리고 표현의 명확성을 확인해야한다.
• 최종 답변 제시: 수정된 답변을 최종적으로 정리하여 제시하는 것. 이때 답변은 명확하고 이해하기 쉬워야 하며, 질문의 요구사항을 충족시켜야 한다.

 

 

Q) Step 을 제시하는 것보다 더 추론을 잘하게하는 방법은?

 

Fine Tuning 이 있음.

 

그리고 여러가지 방법이 있는데 논문을 통해 봐야할듯.

 

Few-Shot 제시하는 방법도 있긴 하곘다. 근데 이건 특정 질문에 대한 예시라서, 예상된 질문 내에서만 이 방법이 가능할듯. 그리고 이 방법은 Input 토큰이 커지는 문제가 생길 수도 있어서 Router Chain 과 같이 써야함.

• 근데 Agent 는 Router Chain 의 상위호환 같아서.

 

 

LangChain Agent 사용 가이드:

from langchain.agents.agent_toolkits import create_python_agent
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.tools.python.tool import PythonREPLTool
from langchain.python import PythonREPL
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(temperature=0, model=llm_model)

tools = load_tools(["llm-math","wikipedia"], llm=llm)

agent= initialize_agent(
    tools, 
    llm, 
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    handle_parsing_errors=True,
    verbose = True)

agent("What is the 25% of 300?")

question = "Tom M. Mitchell is an American computer scientist \
and the Founders University Professor at Carnegie Mellon University (CMU)\
what book did he write?"
result = agent(question) 

 

 

수학 질문에 대한 답변:

> Entering new AgentExecutor chain...
Thought: We need to calculate 25% of 300, which means we need to multiply 300 by 0.25.

Action:

{
"action": "Calculator",
"action_input": "300*0.25"
}

Observation: Answer: 75.0
Thought:The calculator tool returned the correct answer of 75.0.

Final Answer: 25% of 300 is 75.0.

> Finished chain.
{'input': 'What is the 25% of 300?', 'output': '25% of 300 is 75.0.'}

 

 

위키백과 질문에 대한 답변:

> Entering new AgentExecutor chain...
Thought: I should use Wikipedia to find the answer to this question.

Action:
{
"action": "Wikipedia",
"action_input": "Tom M. Mitchell"
}

Observation: Page: Tom M. Mitchell
Summary: Tom Michael Mitchell (born August 9, 1951) is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU). He is a founder and former Chair of the Machine Learning Department at CMU. Mitchell is known for his contributions to the advancement of machine learning, artificial intelligence, and cognitive neuroscience and is the author of the textbook Machine Learning. He is a member of the United States National Academy of Engineering since 2010. He is also a Fellow of the American Academy of Arts and Sciences, the American Association for the Advancement of Science and a Fellow and past President of the Association for the Advancement of Artificial Intelligence. In October 2018, Mitchell was appointed as the Interim Dean of the School of Computer Science at Carnegie Mellon.

Page: Tom Mitchell (Australian footballer)
Summary: Thomas Mitchell (born 31 May 1993) is a professional Australian rules footballer playing for the Collingwood Football Club in the Australian Football League (AFL). He previously played for the Sydney Swans from 2012 to 2016, and the Hawthorn Football Club between 2017 and 2022. Mitchell won the Brownlow Medal as the league's best and fairest player in 2018 and set the record for the most disposals in a VFL/AFL match, accruing 54 in a game against Collingwood during that season. Whom he would later join in 2023, en route to winning the 2023 AFL Grand Final and his first AFL premiership.
Thought:The book that Tom M. Mitchell wrote is called "Machine Learning".

Action:
{
"action": "Wikipedia",
"action_input": "Machine Learning (book)"
}

Observation: Page: Machine learning
Summary: Machine learning (ML) is a field of study in artificial intelligence concerned with the development and study of statistical algorithms that can learn from data and generalize to unseen data, and thus perform tasks without explicit instructions. Recently, artificial neural networks have been able to surpass many previous approaches in performance.Machine learning approaches have been applied to many fields including natural language processing, computer vision, speech recognition, email filtering, agriculture, and medicine. ML is known in its application across business problems under the name predictive analytics. Although not all machine learning is statistically based, computational statistics is an important source of the field's methods.
The mathematical foundations of ML are provided by mathematical optimization (mathematical programming) methods. Data mining is a related (parallel) field of study, focusing on exploratory data analysis (EDA) through unsupervised learning.From a theoretical viewpoint, probably approximately correct (PAC) learning provides a framework for describing machine learning.

Page: Active learning (machine learning)
Summary: Active learning is a special case of machine learning in which a learning algorithm can interactively query a human user (or some other information source), to label new data points with the desired outputs. The human user must possess knowledge/expertise in the problem domain, including the ability to consult/research authoritative sources when necessary.  In statistics literature, it is sometimes also called optimal experimental design. The information source is also called teacher or oracle.
There are situations in which unlabeled data is abundant but manual labeling is expensive. In such a scenario, learning algorithms can actively query the user/teacher for labels. This type of iterative supervised learning is called active learning. Since the learner chooses the examples, the number of examples to learn a concept can often be much lower than the number required in normal supervised learning. With this approach, there is a risk that the algorithm is overwhelmed by uninformative examples.  Recent developments are dedicated to multi-label active learning, hybrid active learning and active learning in a single-pass (on-line) context, combining concepts from the field of machine learning (e.g. conflict and ignorance) with adaptive, incremental learning policies in the field of online machine learning. Using active learning allows for faster development of a machine learning algorithm, when comparative updates would require a quantum or super computer.Large-scale active learning projects may benefit from crowdsourcing frameworks such as Amazon Mechanical Turk that include many humans in the active learning loop.



Page: Quantum machine learning
Summary: Quantum machine learning is the integration of quantum algorithms within machine learning programs.The most common use of the term refers to machine learning algorithms for the analysis of classical data executed on a quantum computer, i.e. quantum-enhanced machine learning. While machine learning algorithms are used to compute immense quantities of data, quantum machine learning utilizes qubits and quantum operations or specialized quantum systems to improve computational speed and data storage done by algorithms in a program. This includes hybrid methods that involve both classical and quantum processing, where computationally difficult subroutines are outsourced to a quantum device. These routines can be more complex in nature and executed faster on a quantum computer. Furthermore, quantum algorithms can be used to analyze quantum states instead of classical data.Beyond quantum computing, the term "quantum machine learning" is also associated with classical machine learning methods applied to data generated from quantum experiments (i.e. machine learning of quantum systems), such as learning 
Thought:Tom M. Mitchell wrote the book "Machine Learning".

Final Answer: Machine Learning

> Finished chain.

 

 

LLM 에서 Custom Agent 를 만드는 가이드:

• 1) Load LLM
• 2) Define Tools
• 3) Create Prompt
  • input 은 유저의 목표일 것.
  • agent_scratchpad 는 이전에 Agent 가 대답한 출력과 도구가 포함될 거임. 이는 작업의 중간 결과를 저장하고, 이 정보를 기반으로 추가적인 작업을 결정하거나 최종 결과를 도출하는 데 사용됨.이걸 통해서 반복되는 질문은 캐싱 역할을 하기도 하고, 이전에 대답한 내용들을 이용해서 더 나은 답변을 작성하기도 할거임.
• 4) Bind tools to LLM
• 5) Create the Agent
• 6) Adding memory
  • 보면 chat_history 를 제공함. 이건 사용자와 에이전트 간의 대화 내역을 저장하는 역할함. 이 데이터는 사용자의 요청과 에이전트의 응답을 포함한 전체 대화의 맥락을 추적하는 데 사용되고 대화 내역은 에이전트가 사용자의 질문이나 요청을 더 잘 이해하고, 대화의 흐름을 유지하며, 사용자의 의도와 상황에 맞는 응답을 제공함.

# 1) Load LLM 
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)


# 2) Define Tools 
from langchain.agents import tool


@tool
def get_word_length(word: str) -> int:
    """Returns the length of a word."""
    return len(word)


get_word_length.invoke("abc")

tools = [get_word_length]


# 3) Create Prompt

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are very powerful assistant, but don't know current events",
        ),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

# 4) Bind tools to LLM
llm_with_tools = llm.bind_tools(tools)


# 5) Create the Agent

from langchain.agents.format_scratchpad.openai_tools import (
    format_to_openai_tool_messages,
)
from langchain.agents.output_parsers.openai_tools import OpenAIToolsAgentOutputParser

agent = (
    {
        "input": lambda x: x["input"],
        "agent_scratchpad": lambda x: format_to_openai_tool_messages(
            x["intermediate_steps"]
        ),
    }
    | prompt
    | llm_with_tools
    | OpenAIToolsAgentOutputParser()
)

from langchain.agents import AgentExecutor

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)


list(agent_executor.stream({"input": "How many letters in the word eudca"}))

# 6) Adding memory

from langchain.prompts import MessagesPlaceholder

MEMORY_KEY = "chat_history"
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are very powerful assistant, but bad at calculating lengths of words.",
        ),
        MessagesPlaceholder(variable_name=MEMORY_KEY),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

from langchain_core.messages import AIMessage, HumanMessage

chat_history = []

agent = (
    {
        "input": lambda x: x["input"],
        "agent_scratchpad": lambda x: format_to_openai_tool_messages(
            x["intermediate_steps"]
        ),
        "chat_history": lambda x: x["chat_history"],
    }
    | prompt
    | llm_with_tools
    | OpenAIToolsAgentOutputParser()
)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True

input1 = "how many letters in the word educa?"
result = agent_executor.invoke({"input": input1, "chat_history": chat_history})
chat_history.extend(
    [
        HumanMessage(content=input1),
        AIMessage(content=result["output"]),
    ]
)
agent_executor.invoke({"input": "is that a real word?", "chat_history": chat_history})