Skip to content

Commit eafaa98

Browse files
yorekyorek
committed
added simple samples
1 parent 407850d commit eafaa98

File tree

8 files changed

+254
-2
lines changed

8 files changed

+254
-2
lines changed

‎.env.example

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ AZURE_OPENAI_CHAT_API_VERSION="2024-08-01-preview"
55
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME="text-embedding-3-small"
66
AZURE_OPENAI_EMBEDDING_API_VERSION="2023-05-15"
77

8-
MSSQL_CONNECTION_STRING='Driver={ODBC Driver 18 for SQL Server};Server=<server>;Database=langchain;Connection Timeout=30;TrustServerCertificate=yes;LongAsMax=yes;UID=<user-id>;PWD=<password>'
8+
MSSQL_CONNECTION_STRING='Driver={ODBC Driver 18 for SQL Server};Server=<server>;Database=langchain;Connection Timeout=30;TrustServerCertificate=yes;LongAsMax=yes;UID=<user-id>;PWD=<password>;'
99

1010
# For Azure SQL use the following
11-
MSSQL_CONNECTION_STRING='Driver={ODBC Driver 18 for SQL Server};Server=<server>.database.windows.net;Database=langchain;Connection Timeout=30;LongAsMax=yes;!'
11+
MSSQL_CONNECTION_STRING='Driver={ODBC Driver 18 for SQL Server};Server=<server>.database.windows.net;Database=langchain;Connection Timeout=30;LongAsMax=yes;'

‎README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,15 @@ If you're just getting started with LangChain, take a look at this article with
1414

1515
If you already have some familiarity with LangChain, and you are looking for samples that helps you to get started using LangChain with SQL Serve or Azure SQL, you can jump directly to the samples below.
1616

17+
## LangChain Samples
18+
19+
Make sure the create an `.env` using `.env.example` as a template.
20+
21+
Samples on how to use the `langchain_sqlserver` library with SQL Server or Azure SQL as a vector store are:
22+
23+
- `test-1.py`: Basic sample to store vectors, content and metadata into SQL Server or Azure SQL and then do simple similarity searches.
24+
- `test-2.py`: Read books reviews from a file, store it in SQL Server or Azure SQL, and then do similarity searches.
25+
1726
## LangChain Tutorials
1827

1928
### Build a semantic search engine

‎example_data/book1.txt

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
Practical Azure SQL Database for Modern Developers: Building Applications in the Microsoft Cloud
2+
---
3+
Davide Mauri
4+
---
5+
Here is the expert-level, insider guidance you need on using Azure SQL Database as your back-end data store. This book highlights best practices in everything ranging from full-stack projects to mobile applications to critical, back-end APIs. The book provides instruction on accessing your data from any language and platform. And you learn how to push processing-intensive work into the database engine to be near the data and avoid undue networking traffic. Azure SQL is explained from a developer's point of view, helping you master its feature set and create applications that perform well and delight users.
6+
7+
Core to the book is showing you how Azure SQL Database provides relational and post-relational support so that any workload can be managed with easy accessibility from any platform and any language. You will learn about features ranging from lock-free tables to columnstore indexes, and about support for data formats ranging from JSON and key-values to the nodes and edges in the graph database paradigm. Reading this book prepares you to deal with almost all data management challenges, allowing you to create lean and specialized solutions having the elasticity and scalability that are needed in the modern world.
8+
9+
What You Will Learn
10+
- Master Azure SQL Database in your development projects from design to the CI/CD pipeline
11+
- Access your data from any programming language and platform
12+
- Combine key-value, JSON, and relational data in the same database
13+
- Push data-intensive compute work into the database for improved efficiency
14+
- Delight your customers by detecting and improving poorly performing queries
15+
- Enhance performance through features such as columnstore indexes and lock-free tables
16+
- Build confidence in your mastery of Azure SQL Database's feature set

‎example_data/book2.txt

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Hyperion (Hyperion Cantos)
2+
---
3+
Dan Simmons
4+
---
5+
A stunning tour de force filled with transcendent awe and wonder, Hyperion is a masterwork of science fiction that resonates with excitement and invention, the first volume in a remarkable epic by the multiple-award-winning author of The Hollow Man.
6+
7+
On the world called Hyperion, beyond the reach of galactic law, waits a creature called the Shrike. There are those who worship it. There are those who fear it. And there are those who have vowed to destroy it. In the Valley of the Time Tombs, where huge, brooding structures move backward through time, the Shrike waits for them all.
8+
9+
On the eve of Armageddon, with the entire galaxy at war, seven pilgrims set forth on a final voyage to Hyperion seeking the answers to the unsolved riddles of their lives. Each carries a desperate hope—and a terrible secret. And one may hold the fate of humanity in his hands.

‎example_data/book3.txt

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
T-SQL Fundamentals (Developer Reference) 4th Edition
2+
---
3+
Itzik Ben-Gan
4+
---
5+
Query and modify data effectively with the latest T-SQL features
6+
7+
Master Transact-SQL's fundamentals, and write correct, robust code for querying and modifying data with modern Microsoft data technologies, including SQL Server 2022, Azure SQL Database, and Azure SQL Managed Instance. Long-time Microsoft Data Platform MVP Itzik Ben-Gan explains key T-SQL concepts, helping you apply your knowledge with hands-on exercises. Ben-Gan first introduces T-SQL's theory and underlying logic, illuminating it as both a language and a way of thinking. Next, he walks through core topics, including logical query processing, single table queries, joins, subqueries, table expressions, set operators, data analysis, data modifications, temporal tables, and transactions and concurrency. Building on this foundation, you'll enhance your coding capabilities, from programmatic constructs to the powerful new SQL Graph. Throughout, Ben-Gan presents reusable T-SQL sample code that works in cloud, on-premises, and hybrid environments.
8+
9+
Microsoft Data Platform MVP Itzik Ben-Gan helps you:
10+
11+
- Understand why T-SQL works as it does, so you can write better code
12+
- Review relational theory elements and modern SQL Server architecture
13+
- Create tables and defi ne data integrity
14+
- Build single-table SELECT queries, multiple-table joins, and subqueries
15+
- Utilize derived tables, Common Table Expressions, views, inline table-valued functions, and APPLY
16+
- Make the most of UNION, INTERSECT, and EXCEPT set operators
17+
- Perform data analysis with window functions, pivoting, grouping sets, and time series
18+
- Insert, update, delete, and merge data
19+
- Get started with system-versioned temporal tables
20+
- Understand and apply transactions and concurrency, including concurrent user connections, locks, blocking, isolation levels, and more
21+
- Use SQL Graph to model, create, modify, and query graph-based data
22+
- Explore SQL Server programmable objects and programming capabilities
23+
- Set up your environment, install sample databases, and fi nd even more help
24+
25+
For IT Professionals and Consultants
26+
27+
- For T-SQL developers, DBAs, BI practitioners, data scientists, report writers, analysts, architects, and SQL Server power users
28+
- For anyone else who needs to write queries or develop T-SQL code for SQL Server, Azure SQL Database, or Azure SQL Managed Instance

‎example_data/book4.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Empty Book
2+
---
3+
John Doe
4+
---
5+
Really, nothing here!

‎test-1.py

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
import os
2+
import logging
3+
4+
from dotenv import load_dotenv
5+
from langchain_sqlserver.vectorstores import SQLServer_VectorStore
6+
from langchain_openai import AzureOpenAIEmbeddings
7+
8+
# logging.basicConfig(format='%(asctime)s - %(levelname)s - %(message)s', level=logging.INFO)
9+
# logger = logging.getLogger(__name__)
10+
# logger.setLevel(logging.INFO)
11+
12+
load_dotenv()
13+
14+
print("Setting up connection to Azure OpenAI embeddings...")
15+
16+
embeddings = AzureOpenAIEmbeddings(
17+
azure_deployment="text-embedding-3-small"
18+
)
19+
20+
connection_string = os.environ["MSSQL_CONNECTION_STRING"]
21+
22+
print("Setting up connection to SQL Server table 1...")
23+
24+
store1 = SQLServer_VectorStore(
25+
connection_string=connection_string,
26+
embedding_function=embeddings,
27+
table_name="langchain_test_table1",
28+
embedding_length=1536
29+
)
30+
31+
print("Setting up connection to SQL Server table 2...")
32+
33+
store2 = SQLServer_VectorStore(
34+
connection_string=connection_string,
35+
embedding_function=embeddings,
36+
table_name="langchain_test_table2",
37+
embedding_length=1536
38+
)
39+
40+
print("Creating payload...")
41+
42+
payload = [
43+
{'id': 1, 'text': "Apples and oranges"},
44+
{'id': 2, 'text': "Cars and airplanes"},
45+
{'id': 3, 'text': "Pineapple" },
46+
{'id': 4, 'text': "Train"},
47+
{'id': 5, 'text': "Bananas"},
48+
{'id': 6, 'text': "Boats"},
49+
{'id': 7, 'text': "Vessels"}
50+
]
51+
52+
ids = [p['id'] for p in payload]
53+
texts = [p['text'] for p in payload]
54+
metadatas = [{'len': len(p['text'])} for p in payload]
55+
56+
print("Delete text in table 1...")
57+
58+
store1.delete(ids)
59+
60+
print("Storing text in table 1...")
61+
62+
store1.add_texts(
63+
ids=ids,
64+
texts=texts,
65+
metadatas=metadatas
66+
)
67+
68+
print("Delete text in table 2...")
69+
70+
store2.delete(ids)
71+
72+
print("Storing text in table 2...")
73+
74+
store2.add_texts(
75+
ids=ids,
76+
texts=texts,
77+
metadatas=metadatas
78+
)
79+
80+
print()
81+
search_text = "ships"
82+
print(f"Similarity search for '{search_text}' in table 1...")
83+
result1 = store1.similarity_search(search_text, 3)
84+
print(result1)
85+
86+
print()
87+
search_text = "ships"
88+
print(f"Similarity search for '{search_text}' with score in table 1...")
89+
result1 = store1.similarity_search_with_score(search_text, 3)
90+
print(result1)
91+
92+
print()
93+
search_text = "ships"
94+
print(f"Similarity search for '{search_text}' as retriever with score in table 1...")
95+
retriever = store1.as_retriever(
96+
search_type="similarity",
97+
search_kwargs={"k": 3}
98+
)
99+
result1 = retriever.invoke(search_text)
100+
print(result1)
101+
102+
print()
103+
print("Done.")
104+
105+

‎test-2.py

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
import os
2+
import logging
3+
4+
from dotenv import load_dotenv
5+
from langchain_sqlserver.vectorstores import SQLServer_VectorStore
6+
from langchain_openai import AzureOpenAIEmbeddings
7+
from langchain_core.documents import Document
8+
9+
# logging.basicConfig(format='%(asctime)s - %(levelname)s - %(message)s', level=logging.INFO)
10+
# logger = logging.getLogger(__name__)
11+
# logger.setLevel(logging.INFO)
12+
13+
load_dotenv()
14+
15+
print(f"Setting up connection to Azure OpenAI embeddings ({os.environ["AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME"]})...")
16+
embeddings = AzureOpenAIEmbeddings(
17+
azure_deployment=os.environ["AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME"]
18+
)
19+
20+
print("Setting up connection to SQL Server...")
21+
vector_store = SQLServer_VectorStore(
22+
connection_string=os.environ["MSSQL_CONNECTION_STRING"],
23+
table_name="langchain_test_table",
24+
embedding_function=embeddings,
25+
embedding_length=1536
26+
)
27+
28+
print("Adding documents...")
29+
30+
def get_book(book_id):
31+
with open(f"./example_data/book{book_id}.txt", "r") as file:
32+
payload = file.read()
33+
34+
book = payload.split("---")
35+
return {
36+
"title": book[0].strip(),
37+
"author": book[1].strip(),
38+
"content": book[2].strip()
39+
}
40+
41+
for i in range(1, 4):
42+
print(f"Adding document {i}")
43+
book = get_book(i)
44+
document = Document(page_content=book["content"], metadata={"book": book["title"], "author": book["author"]})
45+
vector_store.delete(ids=[str(i)])
46+
vector_store.add_documents([document], ids=[str(i)])
47+
48+
print("Delete sample document...")
49+
50+
vector_store.delete(ids=["4"])
51+
52+
query = "learn how to write T-SQL queries"
53+
54+
print()
55+
print(f"Search: '{query}'")
56+
results = vector_store.similarity_search(query=query,k=2)
57+
for doc in results:
58+
print(f"* {doc.metadata}")
59+
60+
print()
61+
print("Search with filter")
62+
results = vector_store.similarity_search(query=query,k=2,filter={"author": "Itzik Ben-Gan"})
63+
for doc in results:
64+
print(f"* {doc.metadata}")
65+
66+
print()
67+
print("Search with score")
68+
results = vector_store.similarity_search_with_score(query=query,k=2)
69+
for doc, score in results:
70+
print(f"* [Similarity Score={score:3f}] {doc.metadata}")
71+
72+
print()
73+
print("Use as Retriever")
74+
retriever = vector_store.as_retriever(
75+
search_type="mmr",
76+
search_kwargs={"k": 2, "fetch_k": 2, "lambda_mult": 0.5},
77+
)
78+
results = retriever.invoke(query)
79+
for doc in results:
80+
print(f"* {doc.metadata}")

0 commit comments

Comments
 (0)