From 7b8739d5ebef015a8279589ec343ba2a1dfaa07e Mon Sep 17 00:00:00 2001 From: youjiang <254693273@qq.com> Date: Fri, 20 Sep 2024 19:07:36 +0800 Subject: [PATCH 1/2] add:Gapi on Jetson --- .../Gapi-Jetson-Getting-Started.md | 131 ++++++++++++++++++ sidebars.js | 1 + 2 files changed, 132 insertions(+) create mode 100644 docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md diff --git a/docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md b/docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md new file mode 100644 index 000000000000..7d018a73330c --- /dev/null +++ b/docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md @@ -0,0 +1,131 @@ +--- +description: This article introduces how to use Gapi on NVIDIA Jetson devices, an embeddable API gateway designed to rapidly transform the achievements of the Jetson AI Lab into real-world applications through a low-code workflow engine and microservice framework. +title: Getting Started with Gapi +tags: + - AI model optimize +image: https://files.seeedstudio.com/wiki/wiki-platform/S-tempor.png +slug: /Gapi-Getting-Started-with-Jetson +last_update: + date: 09/20/2024 + author: Youjiang +--- + +# Getting Started with Gapi on NVIDIA® Jetson Devices + +[Gapi](https://genainerds.com/#/Gapi) is an embeddable API gateway that creates streaming integrations between AI micro services and the systems that users leverage everyday. + +The project's goal is to accelerate the speed of creating pilots and demos of Jetson AI Lab achievements into real world environments. "On Device" generative AI doesn't mean it has to live on an island! + +
+ +- Workflow engine with low code UI with dozens of open integrations and customizable clients for mobile web and desktop. +- Micro service framework for wrapping Jetson containers (Ollama, Whisper, Piper TTS, etc. are done, with more coming). Or wrap your own models/code and integrate it into Workflows. +- Real-time, hybrid, binary+json messaging smoothens intra-service calls and reduced latency. +- A fast path to proving generative AI value to stakeholders in their actual environment. + +## Gapi Server + +Embeddable API gateway software that runs in the background with a low code workflow UI for testing. The server is a message hub and state machine for workflow 'nodes' that talk to Micro Services. Think of it as connective-tissue for applications. + +
+ +:::note +A Micro Service is a process that runs some wrapper python scripts that integrates custom code/models into Workflows using a streaming API. +::: + +- Gapi Server can run on any Jetson Orin or really any computer as the Micro Services connect outbound over secure web sockets. It doesn't use any GPU resources. There is also a little demo version to skip the Server install (but you'll still need to run your own Micro Services). +- [Gapi Project Page](https://genainerds.com/#/Gapi) +- [Gapi Github](https://github.com/GenAI-Nerds/Gapi/) +- [Gapi Docs](https://genainerds.com/#/Docs) +- [Gapi Hosted Demo](https://genaigapi.com/) + +## Requirements +- Any NVIDIA Jetson Orin +- Docker +- Sufficient Storage Space: 1.3GB + +## Install Gapi on Jetson + +**Run These Steps to install Gapi on Jetson:** +```bash +mkdir ~/gapiData && cd ~/gapiData +curl -L https://raw.githubusercontent.com/genai-nerds/Gapi/main/gapiConfigs.zip -o gapiConfigs.zip +unzip -q gapiConfigs.zip +docker run -d --name gapi --network host -v ~/gapiData:/opt/gapi/vdata genainerds/gapi:arm64 /bin/bash -c "cd /opt/gapi/bin && ./startGapi.sh" +echo "You may need to hit Enter now. Afterwards the Docker container 'gapi' s +``` + +**Log In** +- Browse in: http://[host-device-ip]:8090 +- User: root +- Pass: !gapi2024 +- Change password in Settings! Docs shows how to add SSL cert. + +## Workflows + +Workflows visually connect the execution and flow of data between Nodes. + +
+ +A Transaction (or single firing) has "Rolling Input" data it accumulates as Node to Node steps each talk to Micro Services and APIs. All Nodes enjoy variables and flow control using familiar json and javascript concepts. + +
+ +Each Node can append or reference the rolling data in the Transaction while making decisions along the way. + +
+ +Watch live Transactions as they start from clients, webhooks and published messages from Micro Services with visual feedback and debugging. + +## Micro Services + +A Micro Service is a process that runs a wrapper python script that integrates your custom code/models so they can integrate into Gapi Workflows. + +
+ +You can run a Micro Service wherever you like and connect it to a Gapi Server via the streaming, hybrid binary+json message protocol. + +There are some out-of-the-box “Community Micro Services" that we integrate, test and pack into Docker images. When you run them, they auto integrate, load NVIDIA layers correctly and offer logging to the host system. + +
+ +Even better, create your own! Just implement an on_message Python handler to process requests and respond. The rest is handled for you. + +
+ +Think of a Micro Service as a simple wrapper to some code or model you have. It works the same as any other Node inside a Workflow. When it's your Micro Service Node's turn your on_message function will be called. Your script gets the rolling Transaction data for context and you then publish your data directly back into the flow. + +
+ +## Running the Community Micro Services + +**Requirements for Community Micro Services** +1. Any NVIDIA Jetson Orin +2. Sufficient Storage Space: + - Literally 4KB for your own Micro Service + - Anywhere from ~4GB to ~10GB for each Community Micro Service + +**Steps:** +1. Login and go to the Micro Services tab +2. Follow the instructions in the blue box on that page to download your custom configuration +3. Then follow the instructions below that for installing the Micro Service you want + +## Congrats! You Can Go Through the Workflow Tips Now + +
+ + +## Tech Support & Product Discussion + +Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs. + +
+ + +
+ +
+ + +
+ diff --git a/sidebars.js b/sidebars.js index 793fd5acf6ab..895d9ab47d15 100644 --- a/sidebars.js +++ b/sidebars.js @@ -3480,6 +3480,7 @@ const sidebars = { // id: 'Edge/NVIDIA_Jetson/Application/Managed_Services', // }, items: [ + 'Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started', 'Edge/NVIDIA_Jetson/Application/Developer_Tools/alwaysAI-Jetson-Getting-Started', 'Edge/NVIDIA_Jetson/Application/Developer_Tools/Cochl.Sense-Jetson-Getting-Started', 'Edge/NVIDIA_Jetson/Application/Developer_Tools/CVEDIA-Jetson-Getting-Started', From 26cccb8ebf19e9fae8c5b8d64cee3db13164267e Mon Sep 17 00:00:00 2001 From: youjiang <254693273@qq.com> Date: Mon, 23 Sep 2024 09:41:09 +0800 Subject: [PATCH 2/2] update:slug --- .../Application/Developer_Tools/Gapi-Jetson-Getting-Started.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md b/docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md index 7d018a73330c..479575985169 100644 --- a/docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md +++ b/docs/Edge/NVIDIA_Jetson/Application/Developer_Tools/Gapi-Jetson-Getting-Started.md @@ -4,7 +4,7 @@ title: Getting Started with Gapi tags: - AI model optimize image: https://files.seeedstudio.com/wiki/wiki-platform/S-tempor.png -slug: /Gapi-Getting-Started-with-Jetson +slug: /gapi_getting_started-with_jetson last_update: date: 09/20/2024 author: Youjiang