V4l2 programming

About V4L2

Video4Linux2 (V4L2) is a Linux kernel API for streaming video/audio devices.

For accessing devices, program codes only use ioctl() and mmap() which params are the structs and the constants defined in linux/videodev2.h (and included headers).

For example, to get a captured frame is like:

-open device files

• setup device with ioctl
• prepare mmaped areas on deriver for captured frames with ioctl
• share mmaped areas by querying with ioctl
• poll (or select) device to capture
• query the current frame mmaped area with ioctl
• copy the frame pixel data from mmaped area to user memories

About UVC

UVC is common class of USB Video. e.g.

• USB Webcam
• Face cam upon the laptop PC display.

The UVC driver supports YUYV formats for frame pixels at least.

About YUYV pixel format

YUYV represents each horizontal two pixels to 4-byte as [Y0, U, Y1, V, ...].

• The left side pixel RGB calculated by Y0, U, V
• The right side pixel RGB calculated by Y1, U, V

Note: Pixel formats are not included info of width and height. For building bitmap from a frame, it also required width and height information.

YUYV is known as YUV422.

YUV is a color model used analog video devices and cables known as YCbCr.

Y represents luminance value of the picture. Only using Y values become a monochrome picture.

U and V represents 2-dimensional colors (blue and red). [U, V] becomes as:

Basic ioctl() flow for capturing

• [1s] open a device file
• [2s] check the device is capturing type
  • VIDIOC_QUERYCAP
  • VIDIOC_CROPCAP, VIDIOC_S_CAP
• [2s] set capture settings
  • VIDIOC_S_FMT: format and width/height
  • VIDIOC_S_PARM: capture interval
• [2s] prepare buffer on mmap
  • VIDIOC_REQBUF (count > 2): prepare memories of frame count that required more than 2
  • loop of prepared count
    • VIDIOC_QUERYBUF: acquire info for mmap of each frame
    • mmap: share captured buffer with device and user program
• [3s] start capturing
  • loop for mmap list
    • VIDIOC_QBUF: enqueue a mmap frame memory, then the memory is used for storing captured frame pixels
  • VIDIOC_STREAMON: start capturing to enqueued memories
• [4] loop of capturing each frame
  • get captured frame pixel data
    • poll or select the device
    • VIDIOC_DQBUF: dequeue current captured memory and acquire index of mmap list
    • copy the mmap memory to user memory
    • VIDIOC_QBUF: enqueue dequeued mmap memory
• [3e] stop capturing
  • VIDIOC_STREAMOFF: stop capturing
• [2e] release buffers
  • munmap: release mmap memories
  • VIDIOC_REQBUF (count = 0): release device memory by preparing memories of frame count as "0" (IMPORTANT)
• [1e] close the device

for looping this process as:

• [1s] open
  • [2s] setup
    • [3s] start
      • [4] copy frame
    • [3e] stop
  • [2e] release
• [1e] close