35. The Virtual Media Controller Driver (vimc)

The vimc driver emulates complex video hardware using the V4L2 API and the Media API. It has a capture device and three subdevices: sensor, debayer and scaler.

35.1. Topology

The topology is hardcoded, although you could modify it in vimc-core and recompile the driver to achieve your own topology. This is the default topology:

Media pipeline graph on vimc

35.1.1. Configuring the topology

Each subdevice will come with its default configuration (pixelformat, height, width, …). One needs to configure the topology in order to match the configuration on each linked subdevice to stream frames through the pipeline. If the configuration doesn’t match, the stream will fail. The v4l-utils package is a bundle of user-space applications, that comes with media-ctl and v4l2-ctl that can be used to configure the vimc configuration. This sequence of commands fits for the default topology:

media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81

35.2. Subdevices

Subdevices define the behavior of an entity in the topology. Depending on the subdevice, the entity can have multiple pads of type source or sink.

vimc-sensor:

Generates images in several formats using video test pattern generator. Exposes:

• 1 Pad source

vimc-debayer:

Transforms images in bayer format into a non-bayer format. Exposes:

• 1 Pad sink

• 1 Pad source

vimc-scaler:

Scale up the image by a factor of 3. E.g.: a 640x480 image becomes a 1920x1440 image. (this value can be configured, see at Module options). Exposes:

• 1 Pad sink

• 1 Pad source

vimc-capture:

Exposes node /dev/videoX to allow userspace to capture the stream. Exposes:

• 1 Pad sink

• 1 Pad source

35.3. Module options

Vimc has a module parameter to configure the driver.

• sca_mult=<unsigned int>

  Image size multiplier factor to be used to multiply both width and height, so the image size will be sca_mult^2 bigger than the original one. Currently, only supports scaling up (the default value is 3).

35.4. Source code documentation

35.4.1. vimc-streamer

struct vimc_stream

struct that represents a stream in the pipeline

Definition

struct vimc_stream {
  struct media_pipeline pipe;
  struct vimc_ent_device *ved_pipeline[VIMC_STREAMER_PIPELINE_MAX_SIZE];
  unsigned int pipe_size;
  struct task_struct *kthread;
};

Members

pipe

the media pipeline object associated with this stream

ved_pipeline

array containing all the entities participating in the stream. The order is from a video device (usually a capture device) where stream_on was called, to the entity generating the first base image to be processed in the pipeline.

pipe_size

size of ved_pipeline

kthread

thread that generates the frames of the stream.

Description

When the user call stream_on in a video device, struct vimc_stream is used to keep track of all entities and subdevices that generates and process frames for the stream.

struct media_entity *vimc_get_source_entity(struct media_entity *ent)

get the entity connected with the first sink pad

Parameters

struct media_entity * ent

reference media_entity

Description

Helper function that returns the media entity containing the source pad linked with the first sink pad from the given media entity pad list.

Return

The source pad or NULL, if it wasn’t found.

void vimc_streamer_pipeline_terminate(struct vimc_stream *stream)

Disable stream in all ved in stream

Parameters

struct vimc_stream * stream

the pointer to the stream structure with the pipeline to be disabled.

Description

Calls s_stream to disable the stream in each entity of the pipeline

int vimc_streamer_pipeline_init(struct vimc_stream *stream, struct vimc_ent_device *ved)

Initializes the stream structure

Parameters

struct vimc_stream * stream

the pointer to the stream structure to be initialized

struct vimc_ent_device * ved

the pointer to the vimc entity initializing the stream

Description

Initializes the stream structure. Walks through the entity graph to construct the pipeline used later on the streamer thread. Calls vimc_streamer_s_stream() to enable stream in all entities of the pipeline.

Return

0 if success, error code otherwise.

int vimc_streamer_thread(void *data)

Process frames through the pipeline

Parameters

void * data

vimc_stream struct of the current stream

Description

From the source to the sink, gets a frame from each subdevice and send to the next one of the pipeline at a fixed framerate.

Return

Always zero (created as int instead of void to comply with kthread API).

int vimc_streamer_s_stream(struct vimc_stream *stream, struct vimc_ent_device *ved, int enable)

Start/stop the streaming on the media pipeline

Parameters

struct vimc_stream * stream

the pointer to the stream structure of the current stream

struct vimc_ent_device * ved

pointer to the vimc entity of the entity of the stream

int enable

flag to determine if stream should start/stop

Description

When starting, check if there is no stream->kthread allocated. This should indicate that a stream is already running. Then, it initializes the pipeline, creates and runs a kthread to consume buffers through the pipeline. When stopping, analogously check if there is a stream running, stop the thread and terminates the pipeline.

Return

0 if success, error code otherwise.