LangChain core

[jacoblee93]

See/comment on @hwchase17's original post here: langchain-ai/langchain#13823

TL;DR: we are splitting our core functionality to langchain-core to make LangChain more stable and reliable. This should be invisible to the eye and will happen in the background for the next two weeks, and we’d recommend not using langchain-core until then, but we’re flagging for transparency.

Hi all!

As some people may have noticed, we recently split out some code to langchain-core. We want to provide a bit of visibility into our thinking and future steps we’re going to take before announcing it more widely.

The main driver for this is stability and modularity of the package. There is a LOT in LangChain - we’ve integrated with over 500 different partners. Some parts of the package change rather rapidly - LangChain has also remained 0.0.x to reflect the fast moving nature of the field. This has contributed to a sense of sprawl and instability. We want to make it easy to build on top of LangChain and to benefit from of all the integrations and run-time logic that the community has worked on.

We want to split out core abstractions and runtime logic to a separate langchain-core package. langchain-core will contain interfaces for key abstractions (LLMs, vectorstores, retrievers, etc) as well as logic for combining them in chains (LCEL). This will be a more stable package. It will change less frequently, when there are breaking changes we will reflect that in versioning (see more on versioning below), and we will maintain a stable release. This core library will also have minimal dependencies. This will provide a stable base that not only LangChain but also other libraries and integrations can build upon.

We also want to split out key integrations (with the help of partners). This will allow us to properly version those integrations. Right now the biggest source of breaking changes is with integrations. Many integration partners are also moving rapidly, and keeping their SDKs in sync with LangChain abstractions can be tough. By splitting out those integrations into separate packages that can then be versioned independently, we hope to make these integrations more stable and cohesive.

The newest version of langchain uses langchain-core. This was done in a fully backwards compatible way - all previous files in langchain should still expose the same classes and methods as before. If you run into any errors - please let us know! We want this to be fully backwards compatible. We also want future version 0.1 to be fully backwards compatible. We will only start making breaking changes in 0.2.

Versioning for langchain-core is currently at 0.0.x. We plan to bump it to 0.1 on 12/8 and announce it more widely the following week. This two week buffer is to give us time to work with integration partners to create standalone packages for their integrations. We will also work over that time to fully update documentation (right now there is no reference to langchain-core - that is intentional, we do not want people to have to know that it exists yet).

We do not intend to split out all integrations into their own packages over the next two weeks. The remaining long tail of integrations will be kept in LangChain. Over time, we will move them out (either to their own individual packages or to a collective package) and LangChain will contain chains and agents - higher level abstractions that are more use case specific.

Overall, we are excited about this, but also recognize that given the size of LangChain this is a large undertaking. If you have feedback on this plan, PLEASE let us know. We are very open to feedback. We want this to be minimally disruptive to everyone already using LangChain, and only provide benefits in the form of more stability and communication.

[ap-1]

Could you provide more insight into how integrations will be split out into their own packages?

• Are you planning to have this based on individual features (e.g. @langchain/supabase-vector-store and @langchain/supabase-hybrid-search) or just integration partners (e.g. @langchain/supabase)?
• What about integration partners that have multiple features being used in multiple places (e.g. OpenAI with Functions and Cloudflare with Workers AI)

[jacoblee93]

• It'll be up to the partner! But I think the best way to do it is one package per integration to avoid dependency conflicts and excessive fragmentation.
• OpenAI is a special case because it's a required dependency. Current plan there is to move the core OpenAI classes (chat models, LLMs, embeddings) into the class and re-export everything required, then the main package can just pull it in.