GPT-J-6B_NLP_Module

Open source module for Natural Language Processing based on GPT-J-6B, a language model developed by EleutherAI as an alternative to GPT-3. It allows generating text, translating, answering questions and completing code through the Banana.dev API.

• GPT-J-6B Guide
• NPM package gpt-j
• Playlist functionality test

Index 📜

See

Section 1) Description, configuration and technologies.

• 1.0) Project description.
• 1.1) Project status.
• 1.2) Technologies used.
• 1.3) Required dependencies.

Section 2) Project execution and testing

• 2.0) Project setup.
• 2.1) NPM package execution.
• 2.2) Environment variables.

Section 3) Documentation and resources

• 3.0) GPT-J documentation.
• 3.1) Video tutorials.
• 3.2) References.

Section 1) Description, configuration and technologies.

1.0) Project description.

See

Open source Natural Language Processing module. GPT-J-6B is a 6 billion parameter language model trained using Mesh Transformer JAX. It was developed by EleutherAI as an open source alternative to GPT-3. The model can perform tasks such as text generation, translation, answering questions and code completion. This module provides a simple interface to interact with GPT-J-6B through the Banana.dev API.

1.1) Project status.

See

1.2) Technologies used.

See

• Node.js - JavaScript Runtime
• Banana.dev API - AI model inference service
• GPT-J-6B - 6 billion parameter language model
• NPM - Package manager

1.3) Required dependencies.

See

{
  "@banana-dev/banana-dev": "3.0.0"
}

Section 2) Project execution and testing.

2.0) Project setup.

See

Project cloning

git clone https://github.com/andresWeitzel/Api_GPT-J_NLP_NodeJs
cd Modulo_GPT-J-6B_NLP_NodeJs

Dependencies installation

npm install @banana-dev/banana-dev@3.0.0

2.1) NPM package execution.

See

NPM package installation

npm i gpt-j

Basic usage

const modelRunner = require('gpt-j');
const apiKey = 'XXXX'
const modelKey = 'gptj'

modelRunner.run('hello', apiKey, modelKey);

2.2) Environment variables.

See

Environment variables configuration

Create config.js file:

module.exports = {
    API_KEY: process.env.API_KEY || "xxxx",
    MODEL_KEY: process.env.MODEL_KEY || "gptj"
}

Implementation with environment variables

const config = require('config.js');
const modelRunner = require('gpt-j');

//keys
const apiKey = config.API_KEY;
const modelKey = config.MODEL_KEY;

modelRunner.run('hello', apiKey, modelKey);

IMPORTANT: Create a .gitignore file to exclude the config.js file

Section 3) Documentation and resources.

3.0) GPT-J documentation.

See

Model Parameters

Input (text param)

Corresponds to the input layer that the model will analyze (Ex: generate two functions in javascript)

Text Length (length param)

The output text length is measured in tokens, these are common character sequences that are found through the model core. The higher the number, the more text and information we will get in the output.

Temperature Adjustment (temperature param)

Temperature determines the exhaustiveness of the generative model.

• Setting low temperature values leads to a safer model.
• Setting high temperature values leads to a more unstable model.

Batch Size (batchSize param)

Implemented for GPU performance.

Parameters Example

{
    "text": "i want to know the current temperature",
    "length": 250,
    "temperature": 0.9,
    "batchSize": 1
}

Model Layer Model Parameters implemented (Setup)

module.exports.set = (text, length, temp, batch) => {
    const params = {
        "text": text,
        "length": length,
        "temperature": temp,
        "batchSize": batch
    }
    return params;
}

Model Layer Runner implemented (Execution)

Input: I want to know the current temperature in Buenos Aires, Argentina

//Imports
const gptCore = require('@banana-dev/banana-dev');
const config = require('../configs/config.js');
const modelParameters = require('../models/modelParameters');

//keys
const apiKey = config.API_KEY;
const modelKey = config.MODEL_KEY;

//Params
let text = "I want to know the current temperature in Buenos Aires, Argentina"
let length = 400
let temperature = 0.7
let batchSize = 1

let params = modelParameters.set(text, length, temperature, batchSize);

let run = async (params) => {
    try {
        var out = await gptCore.run(apiKey, modelKey, params)
        console.log(out)
        return out
    } catch (error) {
        console.log(error);
    }
}

run(params)

GPT-J Core Output (Response)

{
    message: 'success',
    created: 1668961622,
    apiVersion: '26 Nov 2021',
    modelOutputs: [
        {
            output: '\n' +
                '\n' +
                'Buenos Aires has a very high temperature in summer. If you are a person who lives in Argentina and would like to know what the heat rate is in Buenos Aires, I present my method to know the current temperature in Buenos Aires.\n' +
                '\n' +
                'You don\'t need a GPS to know the current temperature\n' +
                '\n' +
                'Yes, it\'s true, you can know the current temperature at any point in the city in a few seconds. All the information you need is in the following tables.\n' +
                '\n' +
                'To know the current temperature in Buenos Aires, you need to know the temperature of the city area where you are now. If you are in the city, the heat rate in Buenos Aires is quite similar. However, if you are at a point in the city outside the center, the temperature will be higher.\n' +
                '\n' +
                'To know the temperature of the area where you are now, you simply need to know your location point. For example, if you are in the city of Buenos Aires, then you need to know your location point to know the temperature of Buenos Aires.\n' +  
                '\n' +
                'To know your location point, you don\'t need a GPS. You just need to know your address and your speed. The address and speed are the two coordinates of your position.',
            input: 'I want to know the current temperature in Buenos Aires, Argentina'
        }
    ],
    callID: 'call_4a3b9440-fcb6-4122-b269-6ac55a46c3eb'
}

3.1) Video tutorials.

See

Watch Functional test playlist

3.2) References.

See

GPT-J-6B Core API

• Base Example
• Generate Api Key
• Models
• Banana API Documentation
• Pricing and Plans

Core Original GTP-J-6B (Mesh Transformer JAX)

• (The Core with the unoptimized model weighs 6gb. With the Optimized Model 61gb).
• Repository
• Original Paper
• Blog EleutherAI
• Model on HuggingFace

Additional documentation

• GPT-J-6B Guide
• NPM package gpt-j
• Implementation Tutorial
• Comparison with other models
• JAX Documentation
• Mesh Transformer JAX Wiki