Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,22 @@ All notable changes to LocalPulse AI. The format follows

## 1.1.0 — Unreleased

The next feature release expands LocalPulse into a reading, study and follow-up workspace. No new
extension permissions or dependencies. Email read activity is optional and uses a separately
- Opt-in saved research collections with local full-text search, rename/delete and validated JSON backups; fresh cloud consent on reopening/import.
- Navigable verified source excerpts with surrounding text and extracted PDF page numbers.

The next feature release expands LocalPulse into a reading, study and follow-up workspace. Tab grouping, background scheduling and notifications use optional permissions. Local OCR and
audio tools use bundled open-source runtimes and opt-in model downloads. Email read activity is optional and uses a separately
connected public open-source service, or the user's own deployment.

### Added

- **Automatic Gmail and Outlook tracking:** one onboarding choice connects and prepares the first image; separate images per compose draft, per-draft toggles and local estimated-read badges near tracked messages.
- **Tracking controls:** local-only image names, optional generic notifications, 15-minute browser collection and a synthetic encrypted pipeline test with cleanup. Direct owner pixel loads are blocked where identifiable.
- **Private tab organizer:** explicit tab review, exact-URL duplicate selection, protected close confirmation, local session save/restore and optional native domain groups in Chrome.

- **Local media:** bundled Tesseract recognizes images/scanned PDFs with seven optional language packs; bundled Whisper tiny English transcribes chosen audio. Explicit model-data downloads, review/correction, text export and workspace handoff.
- **Explicit web research:** typed-query Wikipedia search, optional SearXNG JSON endpoint and selected public-source fetching; no private context is attached automatically.
- **Platform shortcut hints:** browser-reported OS and configured extension command; Windows/Linux modifier names and macOS symbols.
- **Discoverable email tracking:** first-run setup offers explicit opt-in or Skip for now; a
labeled Email tracking button opens the feature directly from the panel. The free public
service is selected with a real URL value, while custom hosting remains available.
Expand Down Expand Up @@ -50,6 +60,12 @@ connected public open-source service, or the user's own deployment.

### Improved

- Fixed fresh-install background startup when optional alarm/notification APIs are unavailable;
onboarding and toolbar handlers now register before tracking is enabled. Added a production-manifest
regression check so pre-granted test permissions cannot hide this failure.
- Fixed media dialog model readiness across React effect cleanup and shortcut guidance when the
browser command is unassigned. Added real PDF OCR, offline model-cache reuse and optional notification checks.

- Source names stay attached when long workspace documents are split for retrieval or summaries.
- Cloud rules consider every included document; local files need consent for each request.
- Retry and hand-off use the original workspace sources while available; missing sources produce
Expand Down
50 changes: 42 additions & 8 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,20 @@ isn't there, and the in-browser models don't run under `dev` (their worker would
server, which Chrome doesn't allow). To try on-device AI, run `corepack pnpm build` and load
`local/build/chrome-mv3` in your everyday Chrome.

| Command | What it does |
| --------------------------------------- | ------------------------------------------------------------- |
| `corepack pnpm test:tracker` | Encrypted read service tests (native Node) |
| `corepack pnpm test` | Unit tests (Vitest) |
| `corepack pnpm test:e2e` | Builds a test version and runs the browser tests (Playwright) |
| `corepack pnpm lint` / `compile` | ESLint / TypeScript |
| `corepack pnpm build` / `build:firefox` | Production builds in `local/build/` |
| `corepack pnpm zip` / `zip:firefox` | Store packages in `local/build/` |
| Command | What it does |
| --------------------------------------- | --------------------------------------------------------------- |
| `corepack pnpm test:tracker` | Encrypted read service tests (native Node) |
| `corepack pnpm test` | Unit tests (Vitest) |
| `corepack pnpm test:e2e` | Builds a test version and runs the browser tests (Playwright) |
| `corepack pnpm lint` / `compile` | ESLint / TypeScript |
| `corepack pnpm build` / `build:firefox` | Production builds in `local/build/` |
| `corepack pnpm zip` / `zip:firefox` | Store packages in `local/build/` |
| `corepack pnpm crx` | Sign and verify a local CRX with the existing local signing key |

Before the first `corepack pnpm test:e2e`, run `corepack pnpm exec playwright install chromium`.
The command builds both production and test manifests. A production startup test verifies
onboarding/toolbar initialization with no optional permissions; other flows use the test manifest
to bypass interactive browser permission prompts.
A slow test downloads a real 0.7 GB in-browser model and answers with it:

```sh
Expand All @@ -41,6 +45,12 @@ The Chrome build bundles the in-browser models' WebAssembly files, which the bui
checks against `scripts/webllm-libs.sha256`. After updating `@mlc-ai/web-llm` or the model list,
run `corepack pnpm webllm-libs --update-hashes` and include the new hashes in your pull request.

Store-preparation source and verification limits are public in [docs/store/v1.1.0.md](docs/store/v1.1.0.md).
Run `node scripts/capture-store.mjs`, the screenshot-producing browser tests, and
`node scripts/prepare-store.mjs` for synthetic-data UI screenshots. The signed CRX uses local keys
which must never be committed; standard browser-store uploads use the production ZIP unless the
item has opted into Verified CRX Uploads.

## Good first contributions

### Add a provider (JSON only)
Expand Down Expand Up @@ -123,3 +133,27 @@ only after the maintainer explicitly asks. The current feature release is v1.1.0

By contributing, you agree that your work is released under the [MIT license](LICENSE) and that you
follow the [code of conduct](CODE_OF_CONDUCT.md).

## Feature issues and completion

Use the public feature issues linked in [ROADMAP.md](ROADMAP.md) to discuss implementation and
coordinate contributions. PR descriptions use `Closes #number` for implemented acceptance criteria.
GitHub closes the issue automatically when the PR merges into the default branch. Before merge,
the issue stays open with an implementation/test status; do not close it just because code was
pushed. Store publication is a separate milestone.

Local media executables are copied from pnpm-pinned packages by `scripts/media-assets.mjs`.
Model data is fetched only after user consent, from pinned revisions in `src/core/media.ts`.
Native dependency build scripts are deliberately disabled in `pnpm-workspace.yaml`; include
that file in source-review packages. Real OCR/Whisper smoke tests are optional:

```sh
mkdir -p local/fixtures
say -o local/fixtures/voice-test.wav --data-format=LEI16@16000 'Private research stays on this device.'
corepack pnpm build:e2e
LOCALPULSE_REAL_MEDIA=1 corepack pnpm exec playwright test media-real
```

The example creates audio with macOS's installed `say` tool. On other systems use your own
short local WAV containing “private research stays on this device” at the same fixture path.
No personal recordings belong in tests or source control.
27 changes: 19 additions & 8 deletions PRIVACY.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ Hosting providers handle network metadata under their policies, as explained bel
a quick action or the right-click menu), it reads that page's text, title and address, and any
text you selected. While the side panel is open, it reads the current tab to show what it will
use. It doesn't read tabs you don't use it on.
- **Files you choose.** PDFs or text files you drop into the panel or open with its file button.
- **Files you choose.** PDFs, text files, images or audio recordings you drop into the panel or open with its file button.
- **Tab organizer.** On explicit access, lists public web-tab titles and addresses in the current window, previews exact duplicates, groups by domain in Chrome and saves selected sessions locally. No content or tab metadata is sent to a service. Restoring explicitly opens saved websites with ordinary browser requests. Saved URLs may contain private query details.
- **Other tabs you choose.** Only when you add them to a question. The optional "tabs" permission
is used to list their titles and addresses so you can pick them.
- **Document workspace.** Files you add and snapshots of pages you explicitly capture remain in
Expand Down Expand Up @@ -54,11 +55,13 @@ The extension stores these items only in your browser on this computer:
- **Chat history.** Your questions and the answers, with the title and address of the page each
was about, but not the page text. Favorites and names you choose also stay here. History searches
run locally. You can turn history off and delete it in Settings or History.
- **Research library.** Complete text, titles and addresses of sources you explicitly save as named collections. Searches run locally. Delete collections separately from chat history. Library JSON backups contain full source content; keep them private. Reopening and importing collections requires fresh cloud consent and preserves original site restrictions. No pages are automatically saved.
- **Saved tab sessions.** User-chosen names, titles and full web addresses; delete them from Tab organizer separately from chat history.
- **Downloaded in-browser models.** The built-in model is managed by your browser.
- **Follow-ups.** Titles you review, optional web-page addresses, due dates and completion status.
This independent local list stays available when chat history is disabled. No email bodies or
recipient addresses are scraped. Delete tasks or clear the board from Follow-ups.
- **Email read activity.** The chosen server URL, private encryption and signing keys, random
- **Email read activity.** The chosen server URL, private encryption and signing keys, private names, random local draft UUIDs, random
tracking-image capabilities, request timestamps and pending acknowledgements. These persist
locally across browser restarts without expiry. They contain no email subjects, bodies,
recipient addresses or destination links. Disconnecting forgets the chosen URL but preserves
Expand All @@ -83,14 +86,16 @@ voices the browser reports as installed locally, with no remote voice fallback.
## Optional email read activity

The first-run welcome flow offers tracking with an explicit enable button and Skip for now.
No tracking requests happen merely by installing, viewing setup or opening the tracking panel.
No tracking requests happen merely by installing or viewing setup. Opening an already-connected dashboard only reads local data; opted-in scheduling can collect activity while the browser runs. Enabling automatic tracking prepares a first random image; further images are created for supported compose drafts.
Tracking is off until you explicitly connect a server. The optional service is open source under
MIT and can run on Vercel or your own host. Read its [public source and deployment policy](tools/email-tracker/README.md).
The server's `/` and `/transparency` disclose the implementation, GitHub deployment commit, queue
schema, deletion, limits and hosting providers. Connecting grants access only to its address.
Optional Gmail and Outlook access permits a bundled, site-specific script to find compose editor structures and tracking-image URLs. It inserts a different random image per draft, offers a per-draft toggle and shows locally decrypted counts/times next to tracked message images. It does not read subjects, recipients, message text or message IDs for tracking, and never sends email. Content scripts receive public image capabilities and display counts only; private keys stay in trusted extension contexts. Chrome restricts extension-local storage to trusted contexts.

Creating a tracking image sends only public P-256 encryption and signature-verification keys.
The extension never accesses inboxes, email subjects, bodies, contacts or recipients for this
feature and never sends email. You paste a tracking image into an email yourself.
feature and never sends email. Automatic insertion applies only to supported email sites you explicitly allow; manual images remain available for other apps.

Normal page reading can still read a webmail page or selected message when you ask the AI about it;
the page-provider consent and privacy rules above apply to that separate action.
Expand All @@ -111,12 +116,14 @@ key can decrypt queued payloads. Deletion requires a signature from the user's p
key, which is also never sent. Sharing an image URL lets others generate activity and inspect its
ciphertext, so it cannot prove authenticity of a human read.

**Check reads** fetches at most 100 events per image per collection. The
**Check reads**, an explicit message-badge refresh and optional 15-minute browser scheduling fetch at most 100 events per image per collection. The
extension decrypts and saves them locally before signing acknowledgement of those exact event IDs.
The server then deletes those values, removing empty queues. Failed acknowledgements retry on
collection without counting the same event twice; newly arriving events are retained. Deletion
applies to the active queue, not a promise to erase provider logs or historical infrastructure
snapshots. There is **no time-based expiry** for image capabilities, queued events or local results.
snapshots. Notifications are off by default and require separate optional permission. They display generic local counts. Automatic insertion and notifications can be disabled in Email tracking; reload existing mail tabs afterward. Direct owner-side image loads are blocked when browser rules can recognize them, but server-side proxies/preloads can still count.

There is **no time-based expiry** for image capabilities, queued events or local results.
Up to 1,000 events can wait per image; a full queue or provider quota can miss requests. Local
results hold up to 20,000 timestamps per image and 100 images per server.

Expand Down Expand Up @@ -146,8 +153,12 @@ providers; this independently configured feature can still contact its server.
request's changes from GitHub; for a YouTube video, it downloads the transcript from YouTube.
These requests go only to the site you're reading.

These and explicit optional tracker requests are the only network requests LocalPulse makes
besides requests to AI providers you set up.
- **OCR language data** downloads from a pinned tessdata_fast commit on raw.githubusercontent.com only after you enable recognition. Tesseract code and WebAssembly ship in the package. Images/scanned PDF pages and recognized text are processed locally.
- **Audio model data** downloads from a pinned Whisper tiny English repository on Hugging Face and its CDN only after you enable transcription. Runtime code/WebAssembly ship in the package. Audio and transcripts are processed locally; no microphone is accessed.
- **Explicit web research** sends only a typed query to Wikipedia or your chosen SearXNG endpoint, or opens a typed-query DuckDuckGo link. Public source text is fetched only after selection, without cookies. Services see the query, network address and timing. No private source context is attached automatically; AI Local-only mode does not disable explicit web search.
- **Restoring a saved tab session** opens its chosen websites with ordinary browser requests after confirmation.

These disclosed downloads and explicit optional features are the network requests LocalPulse makes besides requests to AI providers you set up.

## What it does NOT do

Expand Down
Loading
Loading