#

crawler4j

Here are 24 public repositories matching this topic...

brianmadden / krawler

A web crawling framework written in Kotlin

kotlin link-checker framework web-crawler webcrawler web-crawling crawler4j

Updated Jun 29, 2021
Kotlin

javagaorui5944 / ProxyIpPool

🚄The Crawler Proxy IP Pool Component

redis quartz jsoup crawler4j delayqueue

Updated Sep 1, 2022
Java

soberqian / Java-Carwler-Technology

网络数据采集技术—Java网络爬虫 (书稿完整代码，涉及网络爬虫的各种技术和知识点)

mysql java crawler thread jsoup httpclient okhttp thread-pool crawler4j webmagic webcollector urlconnection

Updated Sep 1, 2022
Java

HHN / crawler4j

Open Source Web Crawler for Java - A fork of yasserg/crawler4j

java crawler spider web-crawler crawler4j web-spider

Updated Nov 4, 2024
Java

Keerthivasan13 / CSCI572-Information_Retrieval_And_Web_Search_Engines

Search Engine projects

Updated May 20, 2020
Java

chanddu / Book-Search-Engine

Search Engine for Books (Java, Apache Lucene, crawler4j, Apache Spark)

search-engine apache-spark apache pagerank indexing inverted-index tf-idf crawler4j graphx apache-lucene phrase-query

Updated Jul 25, 2018
Java

manjeersrujan / SimpleEcommerceCrawler

Simple Ecommerce website crawler, search using ElasticSearch and Crawler4j

java search-engine elasticsearch crawler spellcheck language-detection indexing crawler4j

Updated Jul 11, 2016
Java

AMOOOMA / StockDataCrawler

Stock Data Crawler made with crawler4j, data from wsj.com

java csv stock-data crawler4j wall-street-journal

Updated Nov 21, 2019
Java

Muhammad-Elgendi / Distributed-crawler4j

Distributed crawler4j using java agent development environment (jade framework)

java distributed-systems crawler crawler4j jade-framework

Updated Apr 29, 2018
Java

cf-toolsuite / sanford

Sanford utilizes LLMs, a storage bucket, and a Vector store to search for and/or summarize documents that you upload.

java gradle cloud-foundry minio crawler4j open-ai redis-stack pgvector chromadb ollama genai-usecase spring-ai dell-ecs-s3

Updated Nov 4, 2024
Java

yasirerkam / YSRsearch

Search Engine

search search-engine crawler crawling vaadin crawl vaadin-framework search-algorithm search-interface crawlers searching-algorithms crawler4j searching vaadin7 vaadin8

Updated Sep 1, 2022
CSS

xchengyu / Web_Crawler

Simple web crawler

Updated Feb 14, 2017
Java

wwyqianqian / information-retrieval

Information retrieval.

java information-retrieval crawler4j

Updated Dec 28, 2019
Java

LUMR / crawler-job

分布式网络爬虫

spring-boot vertx crawler4j

Updated Nov 1, 2018
Java

peterchenhdu / future-framework

future-framework project. https://issues.sonatype.org/browse/OSSRH-41434

jwt spring maven quartz cas codec springboot crawler4j swagger2 digester3

Updated Aug 24, 2018
Java

mukeshkdangi / nypost_crawler

crawler webcrawler crawler4j csci572

Updated Sep 7, 2018
Java

tirthmehta / Google-Cloud-Platform-based-Hadoop-Map-Reduce

Determination of which words occur in a dataset of textbooks along with each word's occurrence count identification with the help of Google Cloud Platform based Dataproc cluster formation.

java dataproc-cluster crawler4j googlecloud dataprocessing googlecloudplatform dataproc

Updated Jul 28, 2017
Java

asifzubair / information_retrieval

Information Retrieval and Web Search Engines

autocomplete spellcheck hadoop solr pagerank map-reduce lucene inverted-index crawler4j

Updated Apr 27, 2017
PHP

guillevc / eli5-crawling

Crawling and searching reddit.com/r/explainlikeimfive

elasticsearch spring-boot vue crawling crawler4j

Updated Jan 9, 2020
Java

shalipoto / crawler4j-sunset

crawler4j with additional page saving features for offline content browsing

java mirroring crawler4j

Updated May 4, 2019
Java

Improve this page

Add a description, image, and links to the crawler4j topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the crawler4j topic, visit your repo's landing page and select "manage topics."