Nc/docs 8mar (langchain-ai#252)

* Use a more reasonable syntaz highlight theme (the default one does not highlight keywords like import and await)

* Small update to Install instructions

* Add yarn/npm install instructions for any place with a npm install command

* Update contributing doc

* Lint

* Add some more cross links and add an index doc for document loaders

* Add guide on which vector store to pick

* Separate install from getting started

* Lint

* Lint

CONTRIBUTING.md

@@ -4,8 +4,20 @@ Hi there! Thank you for even being interested in contributing to LangChain.
 As an open source project in a rapidly developing field, we are extremely open
 to contributions, whether it be in the form of a new feature, improved infra, or better documentation.
 
-To contribute to this project, please follow a ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow.
-Please do not try to push directly to this repo unless you are maintainer.
+To contribute to this project, please follow a ["fork and pull request"](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) workflow. Please do not try to push directly to this repo unless you are a maintainer.
+
+## Quick Links
+
+### Not sure what to work on?
+
+If you are not sure what to work on, we have a few suggestions:
+
+- Look at the issues with the [help wanted](https://github.com/hwchase17/langchainjs/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) label. These are issues that we think are good targets for new contributors. If you are interested in working on one of these, please comment on the issue so that we can assign it to you. And any questions let us know, we're happy to guide you!
+- At the moment our main focus is reaching parity with the Python version across both [integrations](https://langchain.com/integrations.html) and [features](https://langchain.com/features.html). If you are interested in working on a specific integration or feature, just pick anything from those lists not done yet, please let us know and we can help you get started.
+
+### New abstractions
+
+We are currently trying to keep API parity between the Python and JS versions of LangChain, where possible. As such we ask that if you have an idea for a new abstraction, please open an issue first to discuss it. This will help us make sure that the API is consistent across both versions. If you're not sure what to work on, we recommend looking at the links above first.
 
 ## 🗺️Contributing Guidelines
 
@@ -46,7 +58,7 @@ good code into the codebase.
 
 ### 🏭Release process
 
-# TODO:
+> TODO:
 
 As of now, LangChain has an ad hoc release process: releases are cut with high frequency via by
 a developer and published to [npm](https://www.npmjs.com/package/langchain).

docs/.eslintrc.js

@@ -33,26 +33,6 @@ module.exports = {
       { ignore: ["^@theme", "^@docusaurus", "^@generated"] },
     ],
     "import/extensions": OFF,
-    "header/header": [
-      ERROR,
-      "block",
-
-      [
-        "*",
-        " * Copyright (c) Meta Platforms, Inc. and affiliates.",
-        " *",
-        " * This source code is licensed under the MIT license found in the",
-        " * LICENSE file in the root directory of this source tree.",
-        " *",
-        // Unfortunately eslint-plugin-header doesn't support optional lines.
-        // If you want to enforce your website JS files to have @flow or @format,
-        // modify these lines accordingly.
-        {
-          pattern: ".* @format",
-        },
-        " ",
-      ],
-    ],
     "react/jsx-filename-extension": OFF,
     "react-hooks/rules-of-hooks": ERROR,
     "react/prop-types": OFF, // PropTypes aren't used much these days.

docs/docs/getting-started.md

@@ -2,64 +2,9 @@
 
 This tutorial gives you a quick walkthrough about building an end-to-end language model application with LangChain.
 
-## Installation
+## Installation and setup
 
-To get started, install LangChain with the following command:
-
-```bash
-npm i langchain
-```
-
-We currently support LangChain on Node.js 16, 18, and 19. Go [here](https://github.com/hwchase17/langchainjs/discussions/152) to vote on the next environment we should support.
-
-### Node.js 16
-
-If you are running this on Node.js 16, either:
-
-- run your application with `NODE_OPTIONS='--experimental-fetch' node ...`, or
-- install `node-fetch` and follow the instructions [here](https://github.com/node-fetch/node-fetch#providing-global-access)
-
-If you are running this on Node.js 18 or 19, you do not need to do anything.
-
-### TypeScript
-
-We suggest updating your `tsconfig.json` to include the following:
-
-```json
-{
-  "compilerOptions": {
-    ...
-    "target": "ES2020", // or higher
-    "module": "nodenext",
-  }
-}
-```
-
-## Loading the library
-
-### ESM in Node.js
-
-LangChain is an ESM library currently targeting Node.js environments. To use it, you will need to use the `import` syntax, inside a project with `type: module` in its `package.json`.
-
-```typescript
-import { OpenAI } from "langchain";
-```
-
-### CommonJS in Node.js
-
-If your project is using CommonJS, you can use LangChain only with the dynamic `import()` syntax.
-
-```typescript
-const { OpenAI } = await import("langchain");
-```
-
-### Other environments
-
-LangChain currently supports only Node.js-based environments. This includes Vercel Serverless functions (but not Edge functions), as well as other serverless environments, like AWS Lambda and Google Cloud Functions.
-
-We currently do not support running LangChain in the browser. We are listening to the community on additional environments that we should support. Go [here](https://github.com/hwchase17/langchainjs/discussions/152) to vote and discuss the next environments we should support.
-
-Please see [Deployment](./deployment.md) for more information on deploying LangChain applications.
+To get started, follow the [installation instructions](./install) to install LangChain.
 
 ## Picking up a LLM
 
@@ -87,7 +32,7 @@ We will then need to set the environment variable for the OpenAI key. Three opti
 
 1. We can do this by setting the value in a `.env` file and use the [dotenv](https://github.com/motdotla/dotenv) package to read it.
 
-```
+```bash
 OPENAI_API_KEY="..."
 ```
 
@@ -128,6 +73,7 @@ First lets define the prompt template:
 
 ```typescript
 import { PromptTemplate } from "langchain/prompts";
+
 const template = "What is a good name for a company that makes {product}?";
 const prompt = new PromptTemplate({
   template: template,
@@ -159,6 +105,7 @@ Extending the previous example, we can construct an LLMChain which takes user in
 ```typescript
 import { OpenAI } from "langchain/llms";
 import { PromptTemplate } from "langchain/prompts";
+
 const model = new OpenAI({ temperature: 0.9 });
 const template = "What is a good name for a company that makes {product}?";
 const prompt = new PromptTemplate({
@@ -204,13 +151,13 @@ In order to load agents, you should understand the following concepts:
 
 For this example, you will also need to install the SerpAPI package for JavaScript/TypeScript.
 
-```bash
-npm i serpapi
+```bash npm2yarn
+npm install -S serpapi
 ```
 
 And set the appropriate environment variables in the `.env` file.
 
-```
+```bash
 SERPAPI_API_KEY="..."
 ```
 

docs/docs/install.md

@@ -0,0 +1,71 @@
+# Setup and Installation
+
+## Installation
+
+To get started, install LangChain with the following command:
+
+```bash npm2yarn
+npm install -S langchain
+```
+
+We currently support LangChain on Node.js 16, 18, and 19. Go [here](https://github.com/hwchase17/langchainjs/discussions/152) to vote on the next environment we should support.
+
+### Node.js 16
+
+If you are running this on Node.js 16, either:
+
+- run your application with `NODE_OPTIONS='--experimental-fetch' node ...`, or
+- install `node-fetch` and follow the instructions [here](https://github.com/node-fetch/node-fetch#providing-global-access)
+
+If you are running this on Node.js 18 or 19, you do not need to do anything.
+
+### TypeScript
+
+If you are using TypeScript we suggest updating your `tsconfig.json` to include the following:
+
+```json
+{
+  "compilerOptions": {
+    ...
+    "target": "ES2020", // or higher
+    "module": "nodenext",
+  }
+}
+```
+
+## Loading the library
+
+### ESM in Node.js
+
+LangChain is an ESM library currently targeting Node.js environments. To use it, you will need to use the `import` syntax, inside a project with `type: module` in its `package.json`.
+
+```typescript
+import { OpenAI } from "langchain";
+```
+
+### CommonJS in Node.js
+
+If your project is using CommonJS, you can use LangChain only with the dynamic `import()` syntax.
+
+```typescript
+const { OpenAI } = await import("langchain");
+```
+
+If you're using TypeScript in a CommonJS project, you'll need to add the following to your `tsconfig.json`:
+
+```json
+{
+  "compilerOptions": {
+    ...
+    "moduleResolution": "node16",
+  }
+}
+```
+
+### Other environments
+
+LangChain currently supports only Node.js-based environments. This includes Vercel Serverless functions (but not Edge functions), as well as other serverless environments, like AWS Lambda and Google Cloud Functions.
+
+We currently do not support running LangChain in the browser. We are listening to the community on additional environments that we should support. Go [here](https://github.com/hwchase17/langchainjs/discussions/152) to vote and discuss the next environments we should support.
+
+Please see [Deployment](./deployment.md) for more information on deploying LangChain applications.

docs/docs/modules/agents/load_from_hub.md

@@ -4,13 +4,13 @@
 
 For this example, you will also need to install the SerpAPI package for JavaScript/TypeScript.
 
-```bash
-npm i serpapi
+```bash npm2yarn
+npm install -S serpapi
 ```
 
 And set the appropriate environment variables in the `.env` file.
 
-```
+```bash
 SERPAPI_API_KEY="..."
 ```
 

docs/docs/modules/agents/overview.md

@@ -16,13 +16,13 @@ In order to load agents, you should understand the following concepts:
 
 For this example, you will also need to install the SerpAPI package for JavaScript/TypeScript.
 
-```bash
-npm i serpapi
+```bash npm2yarn
+npm install -S serpapi
 ```
 
 And set the appropriate environment variables in the `.env` file.
 
-```
+```bash
 SERPAPI_API_KEY="..."
 ```
 

docs/docs/modules/document_loaders/index.md

@@ -0,0 +1,8 @@
+---
+sidebar_label: Overview
+sidebar_position: 1
+---
+
+# Document Loaders
+
+Document loaders make it easy to create [Documents](../indexes/document.md) from a variety of sources. These documents can then be loaded onto [Vector Stores](../indexes/vector_stores/) to load documents from a source.

docs/docs/modules/indexes/document.md

@@ -17,3 +17,5 @@ You can create one with metadata with:
 import { Document } from "langchain/document";
 const doc = new Document({ pageContent: "foo", metadata: { source: "1" } });
 ```
+
+Also check out [Document Loaders](../document_loaders/) for a way to load documents from a variety of sources.

docs/docs/modules/indexes/text_splitter.md

@@ -60,7 +60,13 @@ const output = splitter.createDocuments([text]);
 
 Finally, `TokenTextSplitter` splits a raw text string by first converting the text into BPE tokens, then split these tokens into chunks and convert the tokens within a single chunk back into text.
 
-To utilize the `TokenTextSplitter`, first install the accompanying required library: `npm install -S @dqbd/tiktoken`.
+To utilize the `TokenTextSplitter`, first install the accompanying required library
+
+```bash npm2yarn
+npm install -S @dqbd/tiktoken
+```
+
+Then, you can use it like so:
 
 ```typescript
 import { Document } from "langchain/document";

docs/docs/modules/indexes/vector_stores/chroma.md

@@ -7,7 +7,11 @@ Use [chroma](https://github.com/chroma-core/chroma) with langchainjs.
 ## Setup
 
 1. Run chroma inside of docker on your computer [docs](https://docs.trychroma.com/api-reference)
-2. Install the chroma js client. `npm install -S chromadb`
+2. Install the chroma js client.
+
+```bash npm2yarn
+npm install -S chromadb
+```
 
 ## Usage
 

docs/docs/modules/indexes/vector_stores/hnswlib.md

@@ -1,6 +1,12 @@
 # HNSWLib
 
-HNSWLib is an in-memory vectorstore. You can install it with `npm install hnswlib-node`.
+HNSWLib is an in-memory vectorstore. You can install it with
+
+```bash npm2yarn
+npm install hnswlib-node
+```
+
+And this is how to use it:
 
 ```typescript
 import { HNSWLib } from "langchain/vectorstores";

docs/docs/modules/indexes/vector_stores/index.md

@@ -1,3 +1,13 @@
 # Vectorstores
 
 A vectorstore is a particular type of database optimized for storing documents, embeddings, and then allowing for fetching of the most relevant documents for a particular query.
+
+## Which one to pick?
+
+Here's a quick guide to help you pick the right vector store for your use case:
+
+- If you're after something that can just run inside your application, in-memory, without any other servers to stand up, then go for [HNSWLib](./hnswlib.md)
+- If you come from Python and you were looking for something similar to FAISS, pick [HNSWLib](./hnswlib.md)
+- If you're looking for an open-source full-featured vector database that you can run locally in a docker container, then go for [Chroma](./chroma.md)
+- If you're using Supabase already then look at the [Supabase](./supabase.mdx) vector store to use the same Postgres database for your embeddings too
+- If you're looking for a production-ready vector store you don't have to worry about hosting yourself, then go for [Pinecone](./pinecone.md)

docs/docs/modules/indexes/vector_stores/pinecone.md

@@ -1,6 +1,10 @@
 # Pinecone
 
-Langchain.js accepts [@pinecone-database/pinecone](https://docs.pinecone.io/docs/node-client) as the client for Pinecone vectorstore. Install the library with `npm install -S @pinecone-database/pinecone`.
+Langchain.js accepts [@pinecone-database/pinecone](https://docs.pinecone.io/docs/node-client) as the client for Pinecone vectorstore. Install the library with
+
+```bash npm2yarn
+npm install -S @pinecone-database/pinecone
+```
 
 ## Index docs
 

docs/docs/modules/indexes/vector_stores/supabase.mdx

@@ -4,20 +4,29 @@ Langchain supports using Supabase Postgres database as a vector store, using the
 
 ## Setup
 
-1. Install the library with `npm install -S @supabase/supabase-js`.
+### Install the library with
 
-2. Create a table and search function in your database
+```bash npm2yarn
+npm install -S @supabase/supabase-js
+```
+
+### Create a table and search function in your database
+
+Run this in your database:
 
 ```sql
+-- Enable the pgvector extension to work with embedding vectors
 create extension vector;
 
+-- Create a table to store your documents
 create table documents (
   id bigserial primary key,
   content text, -- corresponds to Document.pageContent
   metadata jsonb, -- corresponds to Document.metadata
   embedding vector(1536) -- 1536 works for OpenAI embeddings, change if needed
 );
 
+-- Create a function to search for documents
 create function match_documents (
   query_embedding vector(1536),
   match_count int
@@ -43,6 +52,7 @@ begin
 end;
 $$;
 
+-- Create an index to be used by the search function
 create index on documents
   using ivfflat (embedding vector_cosine_ops)
   with (lists = 100);