|
| 1 | +# E-Commerce Demo with Apify Integration |
| 2 | + |
| 3 | +This Next.js application demonstrates how to integrate the Apify E-Commerce Scraping Tool to scrape real product data from e-commerce websites. |
| 4 | + |
| 5 | +## Features |
| 6 | + |
| 7 | +- 🔍 Search for products using keywords |
| 8 | +- 📊 Display product data in multiple formats (cards, tables, statistics) |
| 9 | +- 🛒 Real-time scraping from Amazon using Apify E-Commerce Scraping Tool |
| 10 | +- 💰 Automatic calculation of average prices |
| 11 | +- ⚡ Built with Next.js 16, TypeScript, and Tailwind CSS |
| 12 | + |
| 13 | +## Prerequisites |
| 14 | + |
| 15 | +Before running this application, you need: |
| 16 | + |
| 17 | +1. **Node.js** (version 18 or higher) |
| 18 | +2. **Apify Account** - Create a free account at [https://console.apify.com](https://console.apify.com) |
| 19 | +3. **Apify API Token** - Get your token from [https://console.apify.com/account#/integrations](https://console.apify.com/account#/integrations) |
| 20 | + |
| 21 | +## Setup |
| 22 | + |
| 23 | +1. **Install dependencies:** |
| 24 | + |
| 25 | +```bash |
| 26 | +npm install |
| 27 | +``` |
| 28 | + |
| 29 | +2. **Set up environment variables:** |
| 30 | + |
| 31 | +Create a `.env.local` file in the root directory: |
| 32 | + |
| 33 | +```bash |
| 34 | +APIFY_TOKEN=your_apify_token_here |
| 35 | +``` |
| 36 | + |
| 37 | +Replace `your_apify_token_here` with your actual Apify API token. |
| 38 | + |
| 39 | +## Running the Application |
| 40 | + |
| 41 | +### Development Mode |
| 42 | + |
| 43 | +```bash |
| 44 | +npm run dev |
| 45 | +``` |
| 46 | + |
| 47 | +Open [http://localhost:3000](http://localhost:3000) in your browser. |
| 48 | + |
| 49 | +### Production Build |
| 50 | + |
| 51 | +```bash |
| 52 | +npm run build |
| 53 | +npm start |
| 54 | +``` |
| 55 | + |
| 56 | +## How to Use |
| 57 | + |
| 58 | +1. Enter a product keyword in the search bar (e.g., "wireless headphones", "laptop", "coffee maker") |
| 59 | +2. Click the **Submit** button |
| 60 | +3. Wait for the Apify E-Commerce Scraping Tool to fetch real product data from Amazon |
| 61 | +4. View the scraped products in the: |
| 62 | + - **Statistics Cards** - Total products, average price, and data source |
| 63 | + - **Product Table** - Detailed tabular view of all products |
| 64 | + - **Product Cards** - Visual card-based layout |
| 65 | + |
| 66 | +## How It Works |
| 67 | + |
| 68 | +1. **User Input**: User enters a search query in the SearchBar component |
| 69 | +2. **API Call**: The app sends a POST request to `/api/scrape` with the query |
| 70 | +3. **Apify Integration**: The API route calls the Apify E-Commerce Scraping Tool Actor with: |
| 71 | + - `keyword`: The user's search query |
| 72 | + - `marketplaces`: `['www.amazon.com']` (currently limited to Amazon) |
| 73 | + - `scrapeMode`: `'AUTO'` (automatically chooses best scraping method) |
| 74 | +4. **Data Processing**: The Actor scrapes Amazon and returns product data |
| 75 | +5. **Display**: The app transforms and displays the data in the UI |
| 76 | + |
| 77 | +## Apify E-Commerce Scraping Tool |
| 78 | + |
| 79 | +This application uses the [E-commerce Scraping Tool](https://apify.com/apify/e-commerce-scraping-tool) Actor, which can: |
| 80 | + |
| 81 | +- Extract product data from almost any e-commerce website |
| 82 | +- Search by keywords across major marketplaces (Amazon, Walmart, eBay, etc.) |
| 83 | +- Return structured data including: |
| 84 | + - Product name and description |
| 85 | + - Price and currency |
| 86 | + - Product images |
| 87 | + - Product URLs |
| 88 | + - Brand information |
| 89 | + |
| 90 | +## Pricing |
| 91 | + |
| 92 | +The E-commerce Scraping Tool uses a pay-per-event model: |
| 93 | + |
| 94 | +- **Free Plan**: $5 credit allows ~800 product URLs to be scraped |
| 95 | +- **Starter Plan**: ~13,000 URLs can be scraped with included credits |
| 96 | +- See [pricing details](https://apify.com/apify/e-commerce-scraping-tool/pricing) for more information |
| 97 | + |
| 98 | +## Project Structure |
| 99 | + |
| 100 | +``` |
| 101 | +ecomm-demo/ |
| 102 | +├── app/ |
| 103 | +│ ├── api/ |
| 104 | +│ │ └── scrape/ |
| 105 | +│ │ └── route.ts # API route for Apify integration |
| 106 | +│ ├── layout.tsx |
| 107 | +│ └── page.tsx # Main page component |
| 108 | +├── components/ |
| 109 | +│ ├── SearchBar.tsx # Search input component |
| 110 | +│ ├── StatsCards.tsx # Statistics display |
| 111 | +│ ├── ProductTable.tsx # Table view of products |
| 112 | +│ ├── ProductCards.tsx # Card view of products |
| 113 | +│ └── ui/ # shadcn/ui components |
| 114 | +├── data/ |
| 115 | +│ └── products.ts # Mock data (fallback) |
| 116 | +└── lib/ |
| 117 | + └── types.ts # TypeScript types |
| 118 | +``` |
| 119 | + |
| 120 | +## Technologies Used |
| 121 | + |
| 122 | +- **Next.js 16** - React framework with App Router |
| 123 | +- **TypeScript** - Type-safe JavaScript |
| 124 | +- **Tailwind CSS** - Utility-first CSS framework |
| 125 | +- **shadcn/ui** - High-quality UI components |
| 126 | +- **Apify Client** - Official Apify API client for Node.js |
| 127 | + |
| 128 | +## Troubleshooting |
| 129 | + |
| 130 | +### "APIFY_TOKEN environment variable is not set" |
| 131 | + |
| 132 | +Make sure you've created a `.env.local` file with your Apify token: |
| 133 | + |
| 134 | +```bash |
| 135 | +APIFY_TOKEN=your_actual_token_here |
| 136 | +``` |
| 137 | + |
| 138 | +### Slow scraping |
| 139 | + |
| 140 | +The first scrape may take 30-60 seconds as the Actor needs to: |
| 141 | +1. Start up |
| 142 | +2. Search Amazon |
| 143 | +3. Extract product data |
| 144 | +4. Return results |
| 145 | + |
| 146 | +This is normal behavior for web scraping. |
| 147 | + |
| 148 | +### No results returned |
| 149 | + |
| 150 | +Try: |
| 151 | +- Using a more specific search query |
| 152 | +- Checking that your APIFY_TOKEN is valid |
| 153 | +- Verifying you have sufficient Apify credits |
| 154 | + |
| 155 | +## License |
| 156 | + |
| 157 | +This project is for demonstration purposes. |
0 commit comments