Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft implementation of nannou_webcam #808

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ target/
**/*.rs.bk
Cargo.lock
.DS_Store
.idea/
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ members = [
"nannou_osc",
"nannou_package",
"nannou_timeline",
"nannou_nokhwa",
"nannou_wgpu",
"nature_of_code",
"scripts/run_all_examples",
Expand Down
115 changes: 115 additions & 0 deletions guide/src/tutorials/nokhwa/nokhwa-capture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# An Intro to Nokhwa

**Tutorial Info**

- Author: [l1npengtul](https://l1n.pengtul.net)
- Required Knowledge:
- [Anatomy of a nannou App](/tutorials/basics/anatomy-of-a-nannou-app.md)
- [Nokhwa Introduction](/tutorials/nokhwa/nokhwa-introduction.md)
- Reading Time: 5 minutes
---

# Preface
Keep in mind that camera capture is complicated and stands on top of the steaming stack of complexity that is modern
Operating Systems and the like. Many functions return a form of `Result<T>` containing a `NokhwaError`, you **must** deal
with these to avoid `panic!`s.

# Initializing Nokhwa
First, you must call `nokhwa_initialize()`
```rust,no_run
fn on_initialized(x: bool) {
println!("Initialized")
}

nokhwa_initialize(on_initialized)
```
(You can actually omit this on non-MacOS platforms)

# Querying List of Cameras
Querying the list of available cameras is easily done by calling `query_devices()`:
```rust,no_run
let devices = query_devices().unwrap();
for device in &devices {
println!("{:?}", device)
}
```
This returns a list of `CameraInfo`s, which contain the human-readable name as well as backend-specific data and the
Camera's Index, so you can open the device.

# Opening a device
If you want more information about a device, you need to first open the device:
```rust,no_run
let mut camera = NannouCamera::new(0, None).unwrap();
```

# Querying Available Camera Formats
### What is a camera format?
-> It is a specific Resolution + Framerate + Frame Format
- Resolutions are in format (width, height)
- Framerate is a u32 integer.
- Frame formats are either YUYV(YUY2) or MJPG.

### Querying Camera Formats
Call `NannouCamera::compatible_camera_formats`:
```rust,no_run
let frame_foramts = camera.compatible_camera_formats().unwrap();
```

# Capturing Frames from the Camera
First, you must open the camera stream:
```rust,no_run
camera.open_stream().unwrap();
```
You can check if the camera is open by using `NannouCamera::is_sream_open`
```rust,no_run
let is_open = camera.is_stream_open();
```
#### A note on `ImageTexture`:
But before we get into frame capture, the `NannouCamera` returns an `ImageTexture` every time it captures a frame. This
is a thin wrapper around `ImageBuffer<Rgb<u8>, Vec<u8>>` with utility functions that allow for easy integration with
`nannou_wgpu`. To convert an `ImageTexture` to a `Texture`, use either `into_texture` or `loaded_texture_with_device_and_queue`.

The `NannouCamera` implementation is analogous to `nokhwa::ThreadedCamera`, which means
you can get frames from the camera in three ways:
### Using `NannouCamera::set_callback`:
`NannouCamera::set_callback` is a method that allows you to set a callback function, this function is called every time
a new frame is captured. The function takes a `ImageTexture`, which will be explained ahead.
Remember that this function is executed within the context of the `NannouCamera`'s thread, and that if it takes too long
you will start to drop frames.
```rust,no_run
fn frame_callback(image: ImageTexture) {
/*
- snip! -
Put things that you would do with your image here.
*/
}

fn main() {
/* -snip- */
camera.set_callback(frame_callback);
/* -snip- */
}
```
### Using `NannouCamera::last_frame`:
`NannouCamera::poll_frame` is a method that allows you to get the last captured frame. This function returns instantly
as long as the stream is open. You can also use either `last_frame_texture` or `last_frame_texture_with_device_queue_usage` to poll
a `Texture`.
```rust,no_run
let poll_frame = camera.last_frame().unwrap();
```
### Using `NannouCamera::poll_frame`:
`NannouCamera::poll_frame` is a method that allows you to **wait** for the next frame. This function is blocking. Keep
in mind that this function competes for the frame with `set_callback` **and** `last_frame`, so it is not recommended to use.
You can also use either `poll_texture` or `poll_texture_with_device_queue_usage` to poll
a `Texture`.
```rust,no_run
let poll_frame = camera.poll_frame().unwrap();
```

# Closing the camera
You can close the camera with
```rust,no_run
camera.stop_stream().unwrap();
```
Alternatively, it will automatically close when dropped.

35 changes: 35 additions & 0 deletions guide/src/tutorials/nokhwa/nokhwa-introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# An Intro to Nokhwa

**Tutorial Info**

- Author: [l1npengtul](https://l1n.pengtul.net)
- Required Knowledge:
- [Anatomy of a nannou App](/tutorials/basics/anatomy-of-a-nannou-app.md)
- Reading Time: 5 minutes
---

## What is Nokhwa?

[Nokhwa](https://crates.io/crates/nokhwa) is a crate that allows for easy use of webcams that works on Linux and Windows (with some support for macOS)

## Setting up Nokhwa

To use Nokhwa in nannou, it is necessary to add the `nannou_nokhwa` crate as a dependency in your nannou project.

Open up your `Cargo.toml` file at the root of your nannou project and add the following line under the `[dependencies]` tag:

```toml
nannou_nokhwa = "0.1.0"
```

The value in the quotes is the version of the Nolhwa package. At the time of writing this, `"0.1.0"` is the latest version.

To get the latest version of the osc library, execute `cargo search nannou_nokhwa` on the command line and read the resulting version from there.

To use the crate in your nannou-projects you can add a use-statement at the top of your `main.rs` file.

```rust,no_run
# #![allow(unused_imports)]
use nannou_nokhwa as nokhwa;
# fn main() {}
```
27 changes: 27 additions & 0 deletions nannou_nokhwa/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[package]
name = "nannou_nokhwa"
version = "0.1.0"
authors = ["l1npengtul <l1npengtul@protonmail.com>"]
edition = "2018"
description = "The API for Nannou, the creative coding framework."
readme = "README.md"
keywords = ["webcam", "capture", "cross-platform", "nokhwa"]
license = "MIT"
homepage = "https://nannou.cc"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[features]
wgpu = ["nannou_wgpu"]

[dependencies]
nokhwa = { version = "^0.9", features = ["input-avfoundation", "input-msmf", "input-v4l", "output-threaded"] }
parking_lot = "^0.11"
nannou_wgpu = {version = "^0.18", path="../nannou_wgpu", features = ["image"], optional = true }

[dependencies.image]
version = "^0.23"
no-default-features = true

[package.metadata.docs.rs]
features = ["nokhwa/docs-only", "nokhwa/docs-nolink", "nokhwa/docs-features"]
13 changes: 13 additions & 0 deletions nannou_nokhwa/LICENSE-APACHE
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Copyright 2021 nannou-org.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
16 changes: 16 additions & 0 deletions nannou_nokhwa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# `nannou_nokhwa`

**A Simplified Webcam Capture API for** [**nannou**](https://nannou.cc)**, the creative coding
framework.**

Please see [**the nannou guide**](https://guide.nannou.cc) for more information
on how to get started with nannou!

`nannou_nokhwa` uses [`nokhwa`](https://crates.io/crates/nokhwa): a crate that allows for easy webcam capture across many different platforms.

## Plaforms
- Windows via MediaFoundation
- Linux via Video4Linux2
- MacOS via AVFoundation **(buggy!)**
- Web/JS/WASM via the Browser

Loading