SUT Calls: package fuzzinator.call
¶
- class fuzzinator.call.AdaptiveTimeoutDecorator(**kwargs)¶
Decorator for SUT calls used in validate or reduce jobs. It helps dynamically optimizing the timeout of SUT calls. It works with SUTs that receive and respect a
timeout
kwarg both in their__init__
and__call__
methods. Further requirement is that the issue dict provided by the decorated SUT call must containtime
andid
fields. Having these requirements fulfilled, the decorator adapts the timeout of the next call by taking the geometric mean of the previous two execution times that reproduced the expected issue.The decorator takes no parameter.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.SubprocessCall reduce_call=${call} reduce_call.decorate(0)=fuzzinator.call.AdaptiveTimeoutDecorator [sut.foo.call] command=./bin/foo {test} cwd=/home/alice/foo timeout=3
- class fuzzinator.call.AnonymizeDecorator(*, old_text, new_text=None, properties=None, **kwargs)¶
Decorator for SUT calls to anonymize issue properties.
Mandatory parameter of the decorator:
old_text
: text to replace in issue properties.
Optional parameters of the decorator:
new_text
: text to replace ‘old_text’ with (empty string by default).properties
: array of properties to anonymize (anonymize all properties by default).
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall call.decorate(0)=fuzzinator.call.AnonymizeDecorator [sut.foo.call] command=/home/alice/foo/bin/foo - [sut.foo.call.decorate(0)] old_text=/home/alice/foo new_text=FOO_ROOT properties=["stdout", "stderr"]
- class fuzzinator.call.Call¶
Abstract base class to represent how a software-under-test (SUT) can be called/executed.
- __call__(*, test, **kwargs)¶
Call SUT with
test
as input. Return a dictionary object if the input triggered an issue in the SUT, or a value considered false otherwise. (NonIssue
, a dictionary that always evaluates to false, can be returned to communicate details about an input that did not trigger an issue.) The returned issue dictionary (if any) should contain an'id'
key that equals for issues that are not considered unique.Raises
NotImplementedError
by default.
- __enter__()¶
Set up steps before calling the SUT potentially multiple times.
No-op by default.
- __exit__(*exc)¶
Tear down steps after calling the SUT.
No-op by default.
- class fuzzinator.call.CallDecorator¶
Base class for SUT call (i.e.,
Call
) decorators.- __call__(call_class)¶
Return a decorated version of
call_class
. Create a subclass ofcall_class
that transfers control toinit
,enter
,exit
, orcall
when its__init__
,__enter__
,__exit__
, or__call__
methods are invoked.- Parameters:
call_class – The SUT call class to decorate.
- Returns:
The decorated version of the SUT call class.
- call(cls, obj, *, test, **kwargs)¶
Call
obj
of typecls
. The default operation is to call the__call__
method of the original version of the SUT call class and return its result.Subclasses of
CallDecorator
may override this method if customization of calling the SUT is needed. Usually, the overridden method has to call the original__call__
at some point, which can be performed either bysuper().call(cls, obj, test=test, **kwargs)
(which will call this method, and then transitively the original__call__
) or bysuper(cls, obj).__call__(test=test, **kwargs)
(which calls the original__call__
directly).- Parameters:
cls – The decorated version of the SUT call class, as returned by
__call__
.obj – The SUT call instance to invoke.
test – Input or test case for the SUT call, as defined by
Call.__call__
.
- Returns:
The result of the SUT call, as defined by
Call.__call__
.
- enter(cls, obj)¶
Enter the context managed by
obj
of typecls
. The default operation is to call the__enter__
method of the original version of the SUT call class and return its result.Subclasses of
CallDecorator
may override this method if customization of entering the context is needed. Usually, the overridden method has to call the original__enter__
at some point, which can be performed either bysuper().enter(cls, obj)
(which will call this method, and then transitively the original__enter__
) or bysuper(cls, obj).__enter__()
(which calls the original__enter__
directly).- Parameters:
cls – The decorated version of the SUT call class, as returned by
__call__
.obj – The SUT call instance managing the context.
- Returns:
A value as defined by the context management protocol (usually,
obj
).
- exit(cls, obj, *exc)¶
Exit the context managed by
obj
of typecls
. The default operation is to call the__exit__
method of the original version of the SUT call class and return its result.Subclasses of
CallDecorator
may override this method if customization of exiting the context is needed. Usually, the overridden method has to call the original__exit__
at some point, which can be performed either bysuper().exit(cls, obj, *exc)
(which will call this method, and then transitively the original__exit__
) or bysuper(cls, obj).__exit__(*exc)
(which calls the original__exit__
directly).- Parameters:
cls – The decorated version of the SUT call class, as returned by
__call__
.obj – The SUT call instance managing the context.
exc – The exception that caused the context to be exited (if any), as defined by the context management protocol.
- Returns:
Whether to suppress the exception in
exc
, as defined by the context management protocol.
- init(cls, obj, **kwargs)¶
Initialize
obj
of typecls
. The default operation is to call the__init__
method of the original version of the SUT call class.Subclasses of
CallDecorator
may override this method if customization of the initialization is needed. Usually, the overridden method has to call the original__init__
at some point, which can be performed either bysuper().init(cls, obj, **kwargs)
(which will call this method, and then transitively the original__init__
) or bysuper(cls, obj).__init__(**kwargs)
(which calls the original__init__
directly).- Parameters:
cls – The decorated version of the SUT call class, as returned by
__call__
.obj – The SUT call instance to initialize.
- class fuzzinator.call.ExitCodeFilter(*, exit_codes, invert=False, **kwargs)¶
Decorator filter for SUT calls that return issues with
'exit_code'
property.Mandatory parameter of the decorator:
exit_codes
: ifissue['exit_code']
is not in the array ofexit_codes
, the issue is filtered out; this behavior can be inverted by setting the'invert'
property to True, thus keeping issues with exit code not listed in the array.
Optional parameter of the decorator:
invert
: if it’s true then exit code filtering mechanism is inverted (boolean value, False by default).
The issues that are not filtered out are not changed in any way.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall call.decorate(0)=fuzzinator.call.ExitCodeFilter [sut.foo.call] command=/home/alice/foo/bin/foo - [sut.foo.call.decorate(0)] exit_codes=[139] invert=false
- class fuzzinator.call.FileReaderDecorator(*, cleanup=False, **kwargs)¶
Decorator for SUTs that take input as a file path: saves the content of the failing test case and optionally removes the file.
Moreover, the issue (if any) is also extended with the new
'filename'
property containing the name of the test case (as received in thetest
argument).Optional parameter of the decorator:
cleanup
: Boolean to enable the removal of the loaded file (default: False).
Example configuration snippet:
[sut.foo] call=fuzzinator.call.SubprocessCall call.decorate(0)=fuzzinator.call.FileReaderDecorator [sut.foo.call] # assuming that foo takes one file as input specified on command line command=/home/alice/foo/bin/foo {test} [sut.foo.decorate(0)] cleanup=True
- class fuzzinator.call.FileWriterDecorator(*, filename, work_dir, **kwargs)¶
Decorator for SUTs that take input from a file: writes the test input to a temporary file (relative to an implicit temporary working directory) and replaces the test input with the name of that file.
Mandatory parameter of the decorator:
filename
: name pattern for the temporary file, which may contain the substring{uid}
as a placeholder for a unique string (replaced by the decorator).
The issue returned by the decorated SUT (if any) is extended with the new
'filename'
property containing the name of the generated file (although the file itself is removed).Example configuration snippet:
[sut.foo] call=fuzzinator.call.SubprocessCall call.decorate(0)=fuzzinator.call.FileWriterDecorator [sut.foo.call] # assuming that foo takes one file as input specified on command line command=/home/alice/foo/bin/foo {test} [sut.foo.call.decorate(0)] filename=test-{uid}.txt
- class fuzzinator.call.GdbBacktraceDecorator(*, command, cwd=None, env=None, encoding=None, **kwargs)¶
Decorator for subprocess-based SUT calls with file input to extend issues with
'backtrace'
property.Mandatory parameter of the decorator:
command
: string to pass to GDB as a command to run (all occurrences of{test}
in the string are replaced by the actual name of the test file).
Optional parameters of the decorator:
cwd
: if notNone
, change working directory before GDB/command invocation.env
: if notNone
, a dictionary of variable names-values to update the environment with.encoding
: stdout and stderr encoding (default: autodetect).
The new
'backtrace'
issue property will contain the result of GDB’sbt
command after the halt of the SUT.Example configuration snippet:
[sut.foo] call=fuzzinator.call.SubprocessCall call.decorate(0)=fuzzinator.call.GdbBacktraceDecorator [sut.foo.call] # assuming that {test} is something that can be interpreted by foo as # command line argument command=./bin/foo {test} cwd=/home/alice/foo env={"BAR": "1"} [sut.foo.call.decorate(0)] command=${sut.foo.call:command} cwd=${sut.foo.call:cwd} env={"BAR": "1", "BAZ": "1"}
- class fuzzinator.call.LldbBacktraceDecorator(*, command, cwd=None, env=None, timeout=None, encoding=None, **kwargs)¶
Decorator for subprocess-based SUT calls with file input to extend issues with
'backtrace'
property.Mandatory parameter of the decorator:
command
: string to pass to Lldb as a command to run (all occurrences of{test}
in the string are replaced by the actual name of the test file).
Optional parameters of the decorator:
cwd
: if notNone
, change working directory before Lldb/command invocation.env
: if notNone
, a dictionary of variable names-values to update the environment with.timeout
: timeout (in seconds) to wait between two lldb commands (integer number, 1 by default).encoding
: stdout and stderr encoding (default: autodetect).
The new
'backtrace'
issue property will contain the result of Lldb’sbt
command after the halt of the SUT.Example configuration snippet:
[sut.foo] call=fuzzinator.call.SubprocessCall call.decorate(0)=fuzzinator.call.LldbBacktraceDecorator [sut.foo.call] # assuming that {test} is something that can be interpreted by foo as # command line argument command=./bin/foo {test} cwd=/home/alice/foo env={"BAR": "1"} [sut.foo.call.decorate(0)] command=${sut.foo.call:command} cwd=${sut.foo.call:cwd} env={"BAR": "1", "BAZ": "1"}
- class fuzzinator.call.NonIssue¶
Wrapper class for issue dictionaries to mark them as non-issue.
The
NonIssue
class is a custom dictionary with a truth value forced toFalse
. With this change, it’s capable to express that an issue dictionary doesn’t encode a failure, without emptying it or converting toNone
. The content of aNonIssue
can be taken as a SUT response to a given test case and can be useful in feedback-driven fuzzing scenarios.
- class fuzzinator.call.PlatformInfoDecorator(**kwargs)¶
Decorator for SUT calls to extend issues with
'platform'
and'node'
properties.The new
'platform'
issue property will contain the result of Python’splatform.platform
and the'node'
property will contain the result ofplatform.node
.Example configuration snippet:
[sut.foo] #call=... call.decorate(0)=fuzzinator.call.PlatformInfoDecorator
- class fuzzinator.call.RegexAutomaton(instructions, existing_fields=None)¶
Auxiliary class to recognize patterns in textual data and process them with user-defined automaton-like instructions.
Two kinds of instructions can be assigned to every pattern which are encoded as a prefix to the regex pattern in the form of:
m<inst1><inst2> /<pattern>/
. The first instruction defines what to do with the groups of a matching pattern:s: Save every group into the issue dictionary even if it exists.
a: Add only new fields to the issue dictionary.
c: Clear the whole issue dictionary.
n: Do not add any groups to the issue dictionary.
The second instruction defines how to continue the processing of the input after a match:
c: Continue the processing of the current line with the next pattern.
s: Stop the processing of the current line and continue with the next line.
t: Terminate the processing of the whole input and return with the current version of the issue dictionary.
If a pattern lacks the instructions and the slashes then it is interpreted as an
msc /<pattern>/
.
- class fuzzinator.call.RegexAutomatonFilter(**kwargs)¶
Decorator filter for SUT calls to recognise patterns in the returned issue dictionaries and process them according to user-defined automaton-like instructions.
Optional parameters of the decorator:
key: array of patterns to match against the lines of
issue[key]
(note that ‘key’ can be arbitrary, and multiple different keys can be given to the decorator). The patterns and instructions are interpreted as defined infuzzinator.call.RegexAutomaton
.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall call.decorate(0)=fuzzinator.call.RegexAutomatonFilter [sut.foo.call] command=/home/alice/foo/bin/foo - [sut.foo.call.decorate(0)] stderr=["mss /(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)/"] backtrace=["mas /#[0-9]+ +0x[0-9a-f]+ in (?P<path>[^ ]+) .*? at (?P<file>[^:]+):(?P<line>[0-9]+)/"]
- class fuzzinator.call.RegexFilter(**kwargs)¶
Decorator filter for SUT calls to recognise patterns in the returned issue dictionaries.
Optional parameters of the decorator:
key: array of patterns to match against
issue[key]
(note that ‘key’ can be arbitrary, and multiple different keys can be given to the decorator).
If none of the patterns matches on any of the fields, the issue is filtered out. The issues that are not filtered out are extended with keys-values from the named groups of the matching regex pattern.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall call.decorate(0)=fuzzinator.call.RegexFilter [sut.foo.call] command=/home/alice/foo/bin/foo - [sut.foo.call.decorate(0)] stderr=["(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)"] backtrace=["#[0-9]+ +0x[0-9a-f]+ in (?P<path>[^ ]+) .*? at (?P<file>[^:]+):(?P<line>[0-9]+)"]
- class fuzzinator.call.SanitizedStreamMonitoredSubprocessCall(**kwargs)¶
SanitizedStreamMonitoredSubprocessCall is a specialized
fuzzinator.call.StreamMonitoredSubprocessCall
that contains hard-wired regexes for ASAN and UBSAN error messages. Otherwise, it accepts parameters the same way asfuzzinator.call.StreamMonitoredSubprocessCall
. If custom end_patterns are specified, they take precedence over the predefined patterns.Note
Not available on platforms without fcntl support (e.g., Windows).
- class fuzzinator.call.SanitizerAnalyzerDecorator(**kwargs)¶
Decorator for SUT calls to beautify the sanitizer-specific error messages and extend issues with the
'security'
property which describes the severity of the issue. This decorator has to be used withfuzzinator.call.SanitizerAutomatonFilter
.Example configuration snippet:
[sut.foo] #call=... call.decorate(0)=fuzzinator.call.SanitizerAutomatonFilter call.decorate(1)=fuzzinator.call.SanitizerAnalyzerDecorator
- class fuzzinator.call.SanitizerAutomatonFilter(sanitized_properties=None, **kwargs)¶
SanitizerAutomatonFilter is a specialized
fuzzinator.call.RegexAutomatonFilter
that contains hard-wired regexes for ASAN and UBSAN error messages. The streams where these regexes are looked for can be defined with thesanitized_properties
parameter. Otherwise, it accepts optional parameters the same way asfuzzinator.call.RegexAutomatonFilter
. If they are specified, they take precedence over the predefined patterns. If an ASAN or UBSAN specific pattern is recognized, then issue dictionary will be extended with theerror_type
field.Optional parameter of the decorator:
sanitized_properties
: array of field names in the issue dictionary where the patterns are looked for (default:["stderr"]
).
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall call.decorate(0)=fuzzinator.call.SanitizerAutomatonFilter [sut.foo.call] command=/home/alice/foo/bin/foo - [sut.foo.call.decorate(0)] stdout=["mss /(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)/"] backtrace=["mas /#[0-9]+ +0x[0-9a-f]+ in (?P<path>[^ ]+) .*? at (?P<file>[^:]+):(?P<line>[0-9]+)/"] sanitized_properties=["stdout"]
- class fuzzinator.call.StdinSubprocessCall(*, command, cwd=None, env=None, no_exit_code=None, timeout=None, encoding=None, **kwargs)¶
Subprocess invocation-based call of a SUT that takes a test input on its stdin stream.
Mandatory parameter of the SUT call:
command
: string to pass to the child shell as a command to run.
Optional parameters of the SUT call:
cwd
: if notNone
, change working directory before the command invocation.env
: if notNone
, a dictionary of variable names-values to update the environment with.no_exit_code
: makes possible to force issue creation regardless of the exit code.timeout
: run subprocess with timeout.encoding
: stdout and stderr encoding (default: autodetect).
Result of the SUT call:
If the child process exits with 0 exit code, no issue is returned.
Otherwise, an issue with
'exit_code'
,'stdout'
,'stderr'
and'time'
properties is returned.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall [sut.foo.call] command=./bin/foo - cwd=/home/alice/foo env={"BAR": "1"}
- class fuzzinator.call.StreamMonitoredSubprocessCall(*, command, cwd=None, env=None, end_patterns=None, timeout=None, encoding=None, **kwargs)¶
Subprocess invocation-based call of a SUT that takes test input on its command line. The main difference from
fuzzinator.call.SubprocessCall
is that it continuously monitors the stdout and stderr streams of the SUT and forces it to terminate if some predefined patterns are appearing.Mandatory parameter of the SUT call:
command
: string to pass to the child shell as a command to run (all occurrences of{test}
in the string are replaced by the actual test input).
Optional parameters of the SUT call:
cwd
: if notNone
, change working directory before the command invocation.env
: if notNone
, a dictionary of variable names-values to update the environment with.end_patterns
: array of patterns to match against the lines of stdout and stderr streams. The patterns and instructions are interpreted as defined infuzzinator.call.RegexAutomaton
.timeout
: run subprocess with timeout.encoding
: stdout and stderr encoding (default: autodetect).
Result of the SUT call:
If processing stdout and stderr with
end_patterns
doesn’t produce any result, no issue is returned.Otherwise, an issue with keys from the matching patterns of
end_pattern
extended with the'exit_code'
,'stdout'
,'stderr'
and'time'
properties is returned.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StreamMonitoredSubprocessCall [sut.foo.call] # assuming that {test} is something that can be interpreted by foo as # command line argument command=./bin/foo {test} cwd=/home/alice/foo env={"BAR": "1"} end_patterns=["mss /(?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)/"] timeout=30
Note
Not available on platforms without fcntl support (e.g., Windows).
- class fuzzinator.call.SubprocessCall(*, command, cwd=None, env=None, no_exit_code=None, timeout=None, encoding=None, **kwargs)¶
Subprocess invocation-based call of a SUT that takes test input on its command line. (See
fuzzinator.call.FileWriterDecorator
for SUTs that take input from a file.)Mandatory parameter of the SUT call:
command
: string to pass to the child shell as a command to run (all occurrences of{test}
in the string are replaced by the actual test input).
Optional parameters of the SUT call:
cwd
: if notNone
, change working directory before the command invocation.env
: if notNone
, a dictionary of variable names-values to update the environment with.no_exit_code
: makes possible to force issue creation regardless of the exit code.timeout
: run subprocess with timeout.encoding
: stdout and stderr encoding (default: autodetect).
Result of the SUT call:
If the child process exits with 0 exit code, no issue is returned.
Otherwise, an issue with
'exit_code'
,'stdout'
,'stderr'
and'time'
properties is returned.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.SubprocessCall [sut.foo.call] # assuming that {test} is something that can be interpreted by foo as # command line argument command=./bin/foo {test} cwd=/home/alice/foo env={"BAR": "1"}
- class fuzzinator.call.SubprocessPropertyDecorator(*, property, command, cwd=None, env=None, timeout=None, encoding=None, **kwargs)¶
Decorator for SUT calls to extend issues with an arbitrary property where the value is the output of a shell subprocess.
Mandatory parameters of the decorator:
property
: name of the property to extend the issue with.command
: string to pass to the child shell as a command to run.
Optional parameters of the decorator:
cwd
: if notNone
, change working directory before the command invocation.env
: if notNone
, a dictionary of variable names-values to update the environment with.timeout
: run subprocess with timeout.encoding
: stdout and stderr encoding (default: autodetect).
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall call.decorate(0)=fuzzinator.call.SubprocessPropertyDecorator [sut.foo.call] command=./bin/foo - cwd=/home/alice/foo [sut.foo.call.decorate(0)] property=version command=git rev-parse --short HEAD cwd=${sut.foo.call:cwd} env={"GIT_FLUSH": "1"}
- class fuzzinator.call.TestRunnerSubprocessCall(*, command, cwd=None, env=None, end_texts=None, init_wait=None, timeout_per_test=None, encoding=None, **kwargs)¶
Note
Not available on platforms without fcntl support (e.g., Windows).
- class fuzzinator.call.UniqueIdDecorator(*, properties, **kwargs)¶
Decorator for SUT calls to extend issues with
'id'
property.Mandatory parameter of the decorator:
properties
: array of issue property names, which are concatenated (separated by a space) to form the new'id'
property.
Example configuration snippet:
[sut.foo] call=fuzzinator.call.StdinSubprocessCall call.decorate(0)=fuzzinator.call.RegexFilter call.decorate(1)=fuzzinator.call.UniqueIdDecorator [sut.foo.call] command=/home/alice/foo/bin/foo - [sut.foo.call.decorate(0)] stderr=[": (?P<file>[^:]+):(?P<line>[0-9]+): (?P<func>[^:]+): (?P<msg>Assertion `.*' failed)"] [sut.foo.call.decorate(1)] properties=["msg", "file", "func"]