ffmpeg Filtergraph description

A filtergraph is a directed graph of connected filters. It can contain cycles, and there can be multiple links between a pair of filters. Each link has one input pad on one side connecting it to one filter from which it takes its input, and one output pad on the other side connecting it to one filter accepting its output.

Each filter in a filtergraph is an instance of a filter class registered in the application, which defines the features and the number of input and output pads of the filter.

A filter with no input pads is called a "source", and a filter with no output pads is called a "sink".

4.1 Filtergraph syntax
A filtergraph has a textual representation, which is recognized by the -filter/-vf/-af and -filter_complex options in ffmpeg and -vf/-af in ffplay, and by the avfilter_graph_parse_ptr() function defined in libavfilter/avfilter.h.

A filterchain consists of a sequence of connected filters, each one connected to the previous one in the sequence. A filterchain is represented by a list of ","-separated filter descriptions.

A filtergraph consists of a sequence of filterchains. A sequence of filterchains is represented by a list of ";"-separated filterchain descriptions.

A filter is represented by a string of the form: [in_link_1]...[in_link_N]filter_name@id=arguments[out_link_1]...[out_link_M]

filter_name is the name of the filter class of which the described filter is an instance of, and has to be the name of one of the filter classes registered in the program optionally followed by "@id". The name of the filter class is optionally followed by a string "=arguments".

arguments is a string which contains the parameters used to initialize the filter instance. It may have one of two forms:

A ’:’-separated list of key=value pairs.
A ’:’-separated list of value. In this case, the keys are assumed to be the option names in the order they are declared. E.g. the fade filter declares three options in this order – type, start_frame and nb_frames. Then the parameter list in:0:30 means that the value in is assigned to the option type, 0 to start_frame and 30 to nb_frames.
A ’:’-separated list of mixed direct value and long key=value pairs. The direct value must precede the key=value pairs, and follow the same constraints order of the previous point. The following key=value pairs can be set in any preferred order.
If the option value itself is a list of items (e.g. the format filter takes a list of pixel formats), the items in the list are usually separated by ‘|’.

The list of arguments can be quoted using the character ‘'’ as initial and ending mark, and the character ‘\’ for escaping the characters within the quoted text; otherwise the argument string is considered terminated when the next special character (belonging to the set ‘[]=;,’) is encountered.

A special syntax implemented in the ffmpeg CLI tool allows loading option values from files. This is done be prepending a slash ’/’ to the option name, then the supplied value is interpreted as a path from which the actual value is loaded. E.g.

ffmpeg -i <INPUT> -vf drawtext=/text=/tmp/some_text <OUTPUT>
will load the text to be drawn from /tmp/some_text. API users wishing to implement a similar feature should use the avfilter_graph_segment_*() functions together with custom IO code.

The name and arguments of the filter are optionally preceded and followed by a list of link labels. A link label allows one to name a link and associate it to a filter output or input pad. The preceding labels in_link_1 ... in_link_N, are associated to the filter input pads, the following labels out_link_1 ... out_link_M, are associated to the output pads.

When two link labels with the same name are found in the filtergraph, a link between the corresponding input and output pad is created.

If an output pad is not labelled, it is linked by default to the first unlabelled input pad of the next filter in the filterchain. For example in the filterchain

nullsrc, split[L1], [L2]overlay, nullsink
the split filter instance has two output pads, and the overlay filter instance two input pads. The first output pad of split is labelled "L1", the first input pad of overlay is labelled "L2", and the second output pad of split is linked to the second input pad of overlay, which are both unlabelled.

In a filter description, if the input label of the first filter is not specified, "in" is assumed; if the output label of the last filter is not specified, "out" is assumed.

In a complete filterchain all the unlabelled filter input and output pads must be connected. A filtergraph is considered valid if all the filter input and output pads of all the filterchains are connected.

Leading and trailing whitespaces (space, tabs, or line feeds) separating tokens in the filtergraph specification are ignored. This means that the filtergraph can be expressed using empty lines and spaces to improve redability.

For example, the filtergraph:

testsrc,split[L1],hflip[L2];[L1][L2] hstack
can be represented as:

testsrc,
split [L1], hflip [L2];

[L1][L2] hstack
Libavfilter will automatically insert scale filters where format conversion is required. It is possible to specify swscale flags for those automatically inserted scalers by prepending sws_flags=flags; to the filtergraph description.

Here is a BNF description of the filtergraph syntax:

NAME ::= sequence of alphanumeric characters and '_'
FILTER_NAME ::= NAME["@"NAME]
LINKLABEL ::= "[" NAME "]"
LINKLABELS ::= LINKLABEL [LINKLABELS]
FILTER_ARGUMENTS ::= sequence of chars (possibly quoted)
FILTER ::= [LINKLABELS] FILTER_NAME ["=" FILTER_ARGUMENTS] [LINKLABELS]
FILTERCHAIN ::= FILTER [,FILTERCHAIN]
FILTERGRAPH ::= [sws_flags=flags;] FILTERCHAIN [;FILTERGRAPH]
4.2 Notes on filtergraph escaping
Filtergraph description composition entails several levels of escaping. See (ffmpeg-utils)the "Quoting and escaping" section in the ffmpeg-utils(1) manual for more information about the employed escaping procedure.

A first level escaping affects the content of each filter option value, which may contain the special character : used to separate values, or one of the escaping characters \'.

A second level escaping affects the whole filter description, which may contain the escaping characters \' or the special characters [],; used by the filtergraph description.

Finally, when you specify a filtergraph on a shell commandline, you need to perform a third level escaping for the shell special characters contained within it.

For example, consider the following string to be embedded in the drawtext filter description text value:

this is a 'string': may contain one, or more, special characters
This string contains the ' special escaping character, and the : special character, so it needs to be escaped in this way:

text=this is a \'string\'\: may contain one, or more, special characters
A second level of escaping is required when embedding the filter description in a filtergraph description, in order to escape all the filtergraph special characters. Thus the example above becomes:

drawtext=text=this is a \\\'string\\\'\\: may contain one\, or more\, special characters
(note that in addition to the \' escaping special characters, also , needs to be escaped).

Finally an additional level of escaping is needed when writing the filtergraph description in a shell command, which depends on the escaping rules of the adopted shell. For example, assuming that \ is special and needs to be escaped with another \, the previous string will finally result in:

-vf "drawtext=text=this is a \\\\\\'string\\\\\\'\\\\: may contain one\\, or more\\, special characters"
In order to avoid cumbersome escaping when using a commandline tool accepting a filter specification as input, it is advisable to avoid direct inclusion of the filter or options specification in the shell.

For example, in case of the drawtext filter, you might prefer to use the textfile option in place of text to specify the text to render.

https://www.octfgroup.com/