Skip to content

Workers API #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 123 additions & 39 deletions cloudflare-workers/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Cloudflare Workers API for Couchbase

This project demonstrates how to build a Cloudflare Workers-based API using the **Hono framework** that interfaces with Couchbase's Data API to manage airport data from the travel-sample dataset.
This project demonstrates how to build a Cloudflare Workers-based API using the [**Hono framework**](https://developers.cloudflare.com/workers/frameworks/framework-guides/hono/) that interfaces with Couchbase's Data API to manage airport data from the travel-sample dataset.

## Overview

Expand All @@ -13,12 +13,11 @@ The API provides a comprehensive Airport Information System that manages airport
- `DELETE /airports/{document_key}` - Delete an airport document

### Airport Information Queries
- `POST /airports/routes` - Find routes for a specific airport
- `POST /airports/airlines` - Find airlines that service a specific airport
- `GET /airports/{airport_code}/routes` - Find routes for a specific airport
- `GET /airports/{airport_code}/airlines` - Find airlines that service a specific airport

### Full Text Search (FTS) Features
- `POST /fts/index/create` - Create FTS index for hotel geo-spatial search
- `POST /airports/hotels/nearby` - Find hotels near a specific airport using geo-spatial FTS
- `GET /airports/{airport_id}/hotels/nearby/{distance}` - Find hotels near a specific airport within a specific distance

**Note:** The FTS features require:
1. A Full Text Search index with geo-spatial mapping on hotel documents
Expand All @@ -27,10 +26,11 @@ The API provides a comprehensive Airport Information System that manages airport

## Prerequisites

- [Node.js](https://nodejs.org/) (v18 or later)
- [Node.js](https://nodejs.org/) (v20 or later)
- [Wrangler CLI](https://developers.cloudflare.com/workers/wrangler/install-and-update/)
- Couchbase Server with Data API enabled
- Couchbase travel-sample bucket loaded
- [Cloudflare account](https://dash.cloudflare.com/) with verified email
- [Couchbase Capella](https://www.couchbase.com/products/capella/) cluster with Data API enabled
- Couchbase [travel-sample](https://docs.couchbase.com/dotnet-sdk/current/ref/travel-app-data-model.html) bucket loaded

## Setup

Expand All @@ -43,27 +43,110 @@ cd cloudflare-workers
```bash
npm install
```
4. Configure your environment variables in `wrangler.jsonc` or through Cloudflare dashboard:
```json
{
"vars": {
"DATA_API_USERNAME": "your_username",
"DATA_API_PASSWORD": "your_password",
"DATA_API_ENDPOINT": "your_endpoint"
}
}
4. Configure your database (see Database Configuration section)
5. Configure your environment variables (see Deployment section for details)

## Database Configuration

Setup Database Configuration
To know more about connecting to your Capella cluster, please follow the [instructions](https://docs.couchbase.com/cloud/get-started/connect.html).

Specifically, you need to do the following:

1. Create [database credentials](https://docs.couchbase.com/cloud/clusters/manage-database-users.html) to access the travel-sample bucket (Read and Write) used in the application.
2. [Allow access](https://docs.couchbase.com/cloud/clusters/allow-ip-address.html) to the Cluster from the IP on which the application is running.

## FTS Index Setup

Before using the hotel search functionality, you need to create a Full Text Search index. A script is provided to create the required geo-spatial FTS index:

### Using the FTS Index Creation Script

1. Set your environment variables:
```bash
export DATA_API_USERNAME="your_username"
export DATA_API_PASSWORD="your_password"
export DATA_API_ENDPOINT="your_endpoint"
```

2. Run the script to create the FTS index:
```bash
./scripts/create-fts-index.sh
```

This script creates a geo-spatial FTS index called `hotel-geo-index` that enables proximity searches on hotel documents. The index must be created before using the hotel search functionality.

**Note:** The index creation is a one-time setup process. Once created, the index will be built in the background and will be ready for use with the hotel search endpoint.

## Development

Start the development server:
```bash
npm run dev
```

## Testing

This project includes comprehensive unit tests for all handler functions using [Vitest](https://vitest.dev/) and the [@cloudflare/vitest-pool-workers](https://developers.cloudflare.com/workers/testing/vitest-integration/) testing framework.

### Running Tests

Run all tests:
```bash
npm run test
```

Run specific test categories:
```bash
# Run only handler tests
npm run test:handlers

# Run a specific test file
npm test test/handlers/getHotelsNearAirport.spec.ts
```

## Deployment

Deploy to Cloudflare Workers:
### Prerequisites

- [Cloudflare account](https://dash.cloudflare.com/) with verified email
- [Wrangler CLI](https://developers.cloudflare.com/workers/wrangler/install-and-update/) installed

### Authentication

```bash
wrangler login
```

### Environment Variables

Set your Couchbase Data API credentials using one of these methods:

**Option 1: Cloudflare Dashboard (Recommended)**
1. Deploy first: `npm run deploy`
2. Go to [Workers Dashboard](https://dash.cloudflare.com/) → Your Worker → Settings → Environment Variables
3. Add: `DATA_API_USERNAME`, `DATA_API_PASSWORD`, `DATA_API_ENDPOINT` as secrets

**Option 2: CLI Secrets**
```bash
wrangler secret put DATA_API_USERNAME
wrangler secret put DATA_API_PASSWORD
wrangler secret put DATA_API_ENDPOINT
```

**Option 3: wrangler.jsonc (Development only)**
```json
{
"vars": {
"DATA_API_USERNAME": "your_username",
"DATA_API_PASSWORD": "your_password",
"DATA_API_ENDPOINT": "your_endpoint"
}
}
```

### Deploy

```bash
npm run deploy
```
Expand Down Expand Up @@ -96,40 +179,30 @@ curl -X DELETE https://your-worker.your-subdomain.workers.dev/airports/airport_1

### Find routes for an airport
```bash
curl -X POST https://your-worker.your-subdomain.workers.dev/airports/routes \
-H "Content-Type: application/json" \
-d '{"airportCode": "LAX"}'
curl https://your-worker.your-subdomain.workers.dev/airports/LAX/routes
```

### Find airlines for an airport
```bash
curl -X POST https://your-worker.your-subdomain.workers.dev/airports/airlines \
-H "Content-Type: application/json" \
-d '{"airportCode": "LAX"}'
curl https://your-worker.your-subdomain.workers.dev/airports/LAX/airlines
```

### Create FTS index for hotel search
### Find hotels near an airport with specific distance
```bash
curl -X POST https://your-worker.your-subdomain.workers.dev/fts/index/create \
-H "Content-Type: application/json"
curl "https://your-worker.your-subdomain.workers.dev/airports/airport_1254/hotels/nearby/50km"
```

**Note:** This endpoint creates a geo-spatial FTS index called `hotel-geo-index` that enables proximity searches on hotel documents. The index must be created before using the hotel search functionality.

### Find hotels near an airport
```bash
curl -X POST https://your-worker.your-subdomain.workers.dev/airports/hotels/nearby \
-H "Content-Type: application/json" \
-d '{"airportCode": "SFO", "distance": "10km"}'
```
**Path Parameters:**
- `airport_id`: Airport document ID (required) - e.g., airport_1254, airport_1255
- `distance`: Search radius (required) - e.g., 50km, 10km

**Parameters:**
- `airportCode`: FAA or ICAO code for the airport (required)
- `distance`: Search radius (optional, default: "5km")
**Prerequisites:** Make sure you have created the FTS index using the provided script before using this endpoint.

## Project Structure

```
scripts/ # Utility scripts
└── create-fts-index.sh # Script to create FTS index for hotel search
src/
├── handlers/ # API route handlers
│ ├── createAirport.ts
Expand All @@ -138,14 +211,25 @@ src/
│ ├── deleteAirport.ts
│ ├── getAirportRoutes.ts
│ ├── getAirportAirlines.ts
│ ├── createFTSIndex.ts
│ └── getHotelsNearAirport.ts
├── types/ # TypeScript type definitions
│ ├── airport.ts
│ ├── hotel.ts
│ └── env.ts
├── utils/ # Utility functions
└── index.ts # Main application entry point
test/
├── handlers/ # Handler unit tests
│ ├── createAirport.spec.ts
│ ├── getAirport.spec.ts
│ ├── updateAirport.spec.ts
│ ├── deleteAirport.spec.ts
│ ├── getAirportRoutes.spec.ts
│ ├── getAirportAirlines.spec.ts
│ └── getHotelsNearAirport.spec.ts
├── utils/ # Test utilities and helpers
│ └── testHelpers.ts
└── setup.ts # Test setup configuration
```

## License
Expand Down
4 changes: 4 additions & 0 deletions cloudflare-workers/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
"dev": "wrangler dev",
"start": "wrangler dev",
"test": "vitest",
"test:unit": "vitest run --reporter=verbose",
"test:watch": "vitest --watch",
"test:coverage": "vitest run --coverage",
"test:handlers": "vitest run test/handlers/",
"cf-typegen": "wrangler types"
},
"devDependencies": {
Expand Down
136 changes: 136 additions & 0 deletions cloudflare-workers/scripts/create-fts-index.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
#!/bin/bash

# Script to create FTS index for hotel geo-spatial search
# This creates a Full Text Search index called 'hotel-geo-index' that enables proximity searches on hotel documents.
#
# Prerequisites:
# - Couchbase Server with Data API enabled
# - travel-sample bucket loaded with hotel documents in inventory.hotel collection
# - Hotels must have geo coordinates (geo.lat and geo.lon fields) for proximity search
#
# Usage:
# 1. Set your environment variables:
# export DATA_API_USERNAME="your_username"
# export DATA_API_PASSWORD="your_password"
# export DATA_API_ENDPOINT="your_endpoint"
# 2. Run this script: ./scripts/create-fts-index.sh

# Check if environment variables are set
if [ -z "$DATA_API_USERNAME" ] || [ -z "$DATA_API_PASSWORD" ] || [ -z "$DATA_API_ENDPOINT" ]; then
echo "Error: Please set the following environment variables:"
echo " DATA_API_USERNAME"
echo " DATA_API_PASSWORD"
echo " DATA_API_ENDPOINT"
exit 1
fi

# Index configuration
INDEX_NAME="hotel-geo-index"
BUCKET_NAME="travel-sample"
SCOPE_NAME="inventory"

# Construct the URL
URL="https://${DATA_API_ENDPOINT}/_p/fts/api/bucket/${BUCKET_NAME}/scope/${SCOPE_NAME}/index/${INDEX_NAME}"

echo "Creating FTS index '${INDEX_NAME}' for geo-spatial hotel search..."
echo "URL: ${URL}"

# Create the FTS index with geo-spatial mapping
curl -X PUT "${URL}" \
-u "${DATA_API_USERNAME}:${DATA_API_PASSWORD}" \
-H "Content-Type: application/json" \
-d '{
"type": "fulltext-index",
"name": "hotel-geo-index",
"sourceType": "gocbcore",
"sourceName": "travel-sample",
"sourceParams": {},
"planParams": {
"maxPartitionsPerPIndex": 1024,
"indexPartitions": 1
},
"params": {
"doc_config": {
"docid_prefix_delim": "",
"docid_regexp": "",
"mode": "scope.collection.type_field",
"type_field": "type"
},
"mapping": {
"analysis": {},
"default_analyzer": "standard",
"default_datetime_parser": "dateTimeOptional",
"default_field": "_all",
"default_mapping": {
"dynamic": true,
"enabled": false
},
"default_type": "_default",
"docvalues_dynamic": false,
"index_dynamic": true,
"store_dynamic": false,
"type_field": "_type",
"types": {
"inventory.hotel": {
"dynamic": true,
"enabled": true,
"properties": {
"geo": {
"dynamic": false,
"enabled": true,
"fields": [
{
"analyzer": "keyword",
"include_in_all": true,
"include_term_vectors": true,
"index": true,
"name": "geo",
"store": true,
"type": "geopoint"
}
]
},
"name": {
"dynamic": false,
"enabled": true,
"fields": [
{
"analyzer": "standard",
"include_in_all": true,
"include_term_vectors": true,
"index": true,
"name": "name",
"store": true,
"type": "text"
}
]
},
"city": {
"dynamic": false,
"enabled": true,
"fields": [
{
"analyzer": "standard",
"include_in_all": true,
"include_term_vectors": true,
"index": true,
"name": "city",
"store": true,
"type": "text"
}
]
}
}
}
}
},
"store": {
"indexType": "scorch",
"segmentVersion": 15
}
}
}'

echo ""
echo "FTS index creation completed!"
echo "Note: The index is being built in the background. Use the hotel search endpoint once the index is ready."
Comment on lines +39 to +136

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The script currently executes the curl command to create the FTS index but doesn't explicitly check for or handle potential failures from curl (e.g., network errors, HTTP 4xx/5xx errors from the API). If the curl command fails, the script will still proceed to print the success messages (lines 135-136), which can be misleading.

Could you enhance the script to check the exit status of the curl command and report an error if it fails? Using curl -f (or --fail) makes curl exit with an error code on HTTP server errors, which can then be checked.

This would make the script more robust and provide clearer feedback to the user if the index creation doesn't succeed.

if ! curl -f -sS -X PUT "${URL}" \
  -u "${DATA_API_USERNAME}:${DATA_API_PASSWORD}" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "fulltext-index",
    "name": "hotel-geo-index",
    "sourceType": "gocbcore",
    "sourceName": "travel-sample",
    "sourceParams": {},
    "planParams": {
      "maxPartitionsPerPIndex": 1024,
      "indexPartitions": 1
    },
    "params": {
      "doc_config": {
        "docid_prefix_delim": "",
        "docid_regexp": "",
        "mode": "scope.collection.type_field",
        "type_field": "type"
      },
      "mapping": {
        "analysis": {},
        "default_analyzer": "standard",
        "default_datetime_parser": "dateTimeOptional",
        "default_field": "_all",
        "default_mapping": {
          "dynamic": true,
          "enabled": false
        },
        "default_type": "_default",
        "docvalues_dynamic": false,
        "index_dynamic": true,
        "store_dynamic": false,
        "type_field": "_type",
        "types": {
          "inventory.hotel": {
            "dynamic": true,
            "enabled": true,
            "properties": {
              "geo": {
                "dynamic": false,
                "enabled": true,
                "fields": [
                  {
                    "analyzer": "keyword",
                    "include_in_all": true,
                    "include_term_vectors": true,
                    "index": true,
                    "name": "geo",
                    "store": true,
                    "type": "geopoint"
                  }
                ]
              },
              "name": {
                "dynamic": false,
                "enabled": true,
                "fields": [
                  {
                    "analyzer": "standard",
                    "include_in_all": true,
                    "include_term_vectors": true,
                    "index": true,
                    "name": "name",
                    "store": true,
                    "type": "text"
                  }
                ]
              },
              "city": {
                "dynamic": false,
                "enabled": true,
                "fields": [
                  {
                    "analyzer": "standard",
                    "include_in_all": true,
                    "include_term_vectors": true,
                    "index": true,
                    "name": "city",
                    "store": true,
                    "type": "text"
                  }
                ]
              }
            }
          }
        }
      },
      "store": {
        "indexType": "scorch",
        "segmentVersion": 15
      }
    }
  }'; then
  echo "" >&2 # Maintain blank line before error message for readability
  echo "Error: FTS index creation failed. Please check curl output, credentials, endpoint, and FTS service status." >&2
  exit 1
fi

echo ""
echo "FTS index creation request submitted successfully!"
echo "Note: The index is being built in the background. Use the hotel search endpoint once the index is ready."

Loading