125 lines
5.5 KiB
ReStructuredText
125 lines
5.5 KiB
ReStructuredText
Overview
|
|
========
|
|
|
|
Libmegapixels is a library for accessing V4L2 cameras in Linux. It's the camera
|
|
set-up code originally written for Megapixels but split off and improved as a
|
|
library to make it easier to debug and integrate.
|
|
|
|
The way you'd normally open a webcam in Linux and capture frames is roughly:
|
|
|
|
- Open a :code:`/dev/video*` file descriptor
|
|
- Run the :code:`VIDIOC_QUERYCAP` ioctl to check the device has the :code:`V4L2_CAP_VIDEO_CAPTURE` flag and check which methods of capturing frames are valid.
|
|
- Run the :code:`VIDIOC_S_FMT` and :code:`VIDIOC_G_FMT` ioctls a few times to narrow down a format to use.
|
|
- Set-up mmapped video capture on the device and run :code:`VIDIOC_STREAMON` to start the capture
|
|
- Get frames from the kernel
|
|
|
|
This works for UVC webcams you usually encounter on Linux devices since these are
|
|
relatively simple devices.
|
|
|
|
If you try to use the camera on a modern phone the whole workflow is completely different.
|
|
Instead of dealing with just a video device you also have to deal with
|
|
a :code:`/dev/media*` device for setting up hardware pipelines and then deal with
|
|
various :code:`/dev/v4l-subdev*` devices to configure all the nodes in that media pipeline.
|
|
|
|
One (or more) of the nodes in that media pipeline will be the regular old :code:`/dev/video*`
|
|
device again that will provide your application with the frames, but due to the way
|
|
the pipelines work a lot of the normal auto-configuration things applications do no longer work.
|
|
|
|
In the new pipelines the video device only deals with getting the video frames into
|
|
userspace. Due to this the video device does not actually know what formats and
|
|
resolutions are valid so using the old ioctls to query this information is useless.
|
|
|
|
One of the v4l-subdev devices will be representing the sensor in the device, this does know what
|
|
modes are available but to know that mode will work you need to make sure all the nodes in the
|
|
pipeline can run at that mode and all these nodes need to be manually configured. But in
|
|
reality it's even more complicated becaues this only describes what resolutions and modes
|
|
the drivers for these components support and it does not account for limitations in
|
|
the actual hardware like not all the MIPI lanes being connected between the sensor
|
|
and the SoC limiting the possible bandwidth, or even more basic bandwidth limitations
|
|
due to the length of PCB traces.
|
|
|
|
Using libmegapixels
|
|
-------------------
|
|
|
|
The solution for this mess in Megapixels which is now transferred to a library
|
|
is using config files that provide the pipeline information. This hardcodes a
|
|
series of known working modes and also provides the metadata for producing
|
|
high quality pictuers from the data coming from the pipeline.
|
|
|
|
The general use of libmegapixels is this:
|
|
|
|
.. code-block:: c
|
|
|
|
#include <libmegapixels.h>
|
|
|
|
|
|
// Create the empty device config object
|
|
libmegapixels_devconfig *config = {0};
|
|
libmegapixels_init(&config);
|
|
|
|
// Find the config file path for this device
|
|
char configpath[PATH_MAX];
|
|
int ret = libmegapixels_find_config(configpath);
|
|
|
|
// Load the config
|
|
libmegapixels_load_file(config, configpath);
|
|
|
|
// Additionally load UVC cameras with autodetection
|
|
libmegapixels_load_uvc(config);
|
|
|
|
printf("Found %d cameras\n", config->count);
|
|
|
|
// Open the first camera
|
|
libmegapixels_camera *camera = config->cameras[camera_id];
|
|
libmegapixels_open(camera);
|
|
|
|
printf("Camera has %d modes\n", camera->num_modes);
|
|
|
|
// Select the first mode
|
|
libmegapixels_mode *mode = camera->modes[mode_idx];
|
|
struct v4l2_format format = {0};
|
|
libmegapixels_select_mode(camera, mode, &format);
|
|
|
|
// Now you can do regular V4L2 things on the camera with the
|
|
// supplied FDs. video_fd for the /dev/video device.
|
|
ioctl(camera->video_fd, VIDIOC_QUERYCAP, &cap)
|
|
|
|
Libmegapixels replaces most of the original V4L2 setup code and provides a
|
|
filled in :code:`struct v4l2_format` to get all the exact mode information.
|
|
|
|
To pick a camera and mode you'd iterate over the found cameras and then over
|
|
the mode structs to find something that matches the needs of the application.
|
|
|
|
The camera struct provides the FDs for the various devices you might need:
|
|
|
|
- The :code:`video_fd` field has the FD for :code:`/dev/video*` for actually
|
|
getting the frames.
|
|
- The :code:`sensor_fd` field is for the :code:`/dev/v4l-subdev*` device that
|
|
represents the sensor. This is for setting camera V4L2 controls while capturing.
|
|
- The :code:`media_fd` field is for the :code:`/dev/media*` device that has the
|
|
pipeline for the current camera. This is mostly for the libmegapixels internals.
|
|
|
|
The config files
|
|
----------------
|
|
|
|
The libmegapixels library ships with configuration files for some devices. The
|
|
device configuration can be stored in multiple locations so it can be overridden
|
|
by packagers and end users.
|
|
|
|
/usr/share/megapixels/config/%model.conf
|
|
This is the location where the config files from packages are stored. This is
|
|
also where the build script from libmegapixels will place the internal config
|
|
files here.
|
|
|
|
/etc/megapixels/config/%model.conf
|
|
This is the place for extra and/or override config files
|
|
|
|
$cwd/config/%model.conf
|
|
The config is loaded from the current working directory to make testing and
|
|
debugging the code easier and to run it from the root of the git repository.
|
|
|
|
The %model argument in the path is referring to the device-tree compatible names
|
|
of the device. This can be found in :code:`/proc/device-tree/compatible`. This
|
|
stores the name of the hardware seperated by null-bytes in decreasing specificity
|
|
order. Libmegapixels will check all of them in order for every location above.
|