Modules¶
Interface¶
This document contains all the information needed to configure modules for your f4pga project as well as some info about the API used to write modules.
Configuration interface:¶
Modules are configured through an internal API by f4pga.
The basic requirement for a module script is to expose a class with Module
interface.
f4pga reads its configuration from two different sources: platform’s flow definition, which is a file that usually comes bundled with f4pga and project’s flow configuration, which is a set of configuration options provided by the user through a JSON file or CLI interface.
Those sources contain snippets of module configurations.
A module configuration is a structure with the following fields:
takes
- a dictionary that contains keys which are names of the dependencies used by the module. The values are paths to those dependencies. They can be either singular strings or lists of strings.produces
- a dictionary that contains keys which are names of the dependencies produced by the module. The values are requested filenames for the files generated by the module. They can be either singular strings or lists of strings.values
- a dictionary that contains other values used to configure the module. The keys are value’s names and the values can have any type.
Platform-level configuration¶
In case of platform’s flow definition, a values
dictionary can be defined
globally and the values defined there will be passed to every module’s config.
Those values can be overridden per-module through module_options
dictionary.
Parameters used during module’s construction can also be defined in module_options
as params
(those are not a part of module configuration, instead they are used
during the actual construction of a module instance, before it declares any of its
input/outputs etc.. This is typically used to achieve some parametrization over module’s
I/O).
Defining dictionaries for takes
and produces
is currently disallowed within
platform’s flow definition.
For examples of platform’s flow definition described here, please have a look at
f4pga/platforms/
directory. It contains platform flow definitions that come bundled
with f4pga.
Project-level configuration¶
This section describes project’s flow configuration.
Similarly to platform’s flow definition, values
dict can be provided.
The values provided there will overwrite the values from
platform’s flow definition in case of a collision.
Unlike platform’s flow definition, project’s flow configuration may contain
dependencies
dict. This dictionary would be used to map symbolic dependency
names to actual paths. Most dependencies can have their paths resolved implicitly
without the need to provide explicit paths, which is a mechanism that is described
in a later section of this document. However some dependencies must be provided
explicitly, eg. paths to project’s Verilog source files. It should be noted that
depending on the flow definition and the dependency in question, the path does not
necessarily have to point to an already existing file. If the dependency is a
product of a module within the flow, the path assigned to it will be used
by the module to build that dependency. This is also used to in case of on-demand
dependencies, which won’t be produced unless the user explicitly provides a path
for them.
project’s flow configuration cannot specify params
for modules and does not
use module_options
dictionary. Neither it can instantiate any extra stages.
Any entry with a couple exceptions* is treated as a platform name.
Enabling support for a given platform within a project’s flow configuration file
requires having an entry for that platform.
Each of those entries may contain dependencies
, values
fields which will
overload the dependecies
and values
defined in a global scope of
project’s flow configuration. Any other field under those platform entries
is treated as a stage-specific-configuration. The key is a name of a stage within
a flow for the specified platform and the values are dicts which may contain
dependencies
and values
fields that overload dependencies
and values
respectively, locally for the stage. Additionally a default_target
field can be
provided to specify a default target to built when the user does not specify it through
a CLI interface.
The aforementioned *exceptions are:
dependencies
- dependencies shared by all platforms.values
- values shared by all platformsdefault_platform
- default platform to chose in case it doesn’t get specified by the user
Those apply only to flow configuration file.
Internal environmental variables¶
It’s very useful to be able to refer to some data within platform’s flow definition and project’s flow configuration to either avoid redundant definitions or to store and access results of certain operations. f4pga allows doing that by using a special syntax for accessing internal environmental variables.
The syntax is ${variable_name}
. Any string value within
platform’s flow definition and project’s flow configuration that contains
such patterns will have them replaced with the values of the variables referenced
if those values are strings. Eg.:
With the following values defined:
{
"a_value": "1234",
"another_value": "a_value: ${a_value}"
}
another_value
will resolve to:
"a_value: 1234"
If the value is a list however, the result would be a list with all entries being the original string with the reference to a variable replaced by following items of the original list. Eg.:
With the following values defined
{
"list_of_values": ["a", "b", "c"],
"some_string": "item: ${list_of_values}"
}
some_string
will resolve to
["item: a", "item: b", "item: c"]
Be careful when using this kind of resolution, as it’s computational and memory complexity grows exponentially in regards to the number of list variables being referenced, which is a rather obvious fact, but it’s still worth mentioning.
The variables that can be referenced within a definition/configuration fall into 3 categories:
value references - anything declared as a
value
can be accessed by it’s namedependency references - any dependency path can be referenced using the name of the dependency prefaced with a ‘:’ prefix. Eg.:
${:eblif}
will resolve to the path ofeblif
dependency. Make sure that the dependency can be actually resolved when you are using this kind of reference. For example you can’t use the a reference toeblif
dependency in a module which does not rely on it. An exception is the producer module which can in fact reference it’s own outputs but these references cannot be used during the mapping stage (more on that later).built-in references - there are a couple of built-in variables which are very handy:
shareDir
- path to f4pga’s share directory.binDir
- path to f4pga’s bin directory.prjxray_db
- Project X-Ray database path.python3
- path to Python 3 interpreter.noisyWarnings
- (this one should probably get removed)
Module
class¶
Each module is represented as a class derived from Module
class.
The class should implement the following methods:
execute(self, ctx: ModuleContext)
- executes the module in exec modemap_io(self, ctx: ModuleContext) -> 'dict[str, ]'
- executes the module in mapping mode__init__(self, params: 'dict[str, ]')
- initializer. Theparams
is a dict with optional parameter for the module.
Each module script should expose the class by defining it’s name/type alias as
ModuleClass
. f4pga tries to access a ModuleClass
attribute within a package
when instantiating a module.
Module’s execution modes¶
A module has essentially two execution modes:
mapping mode
exec mode
mapping mode¶
In mapping mode the module is provided with an incomplete configuration which includes:
takes
namespace: this maps names of input dependencies to the paths of these dependenciesvalues
namespace: this maps names of variables to the values of those variables.
The module has to provide a dictionary that will provide every output dependency that’s not on-demand a default path. This is basically a promise that when executed in exec mode, the module will produce files for this paths. Typically such paths would be derived from a path of one of it’s input dependencies. This mechanism allows the user to avoid specifying an explicit path for each intermediate target.
It should be noted that variables referring to the output dependencies can’t be accessed at this stage for the obvious reason as their values are yet to be evaluated.
exec mode¶
In exec mode the module does the actual work.
The configuration passed into this mode is full and it includes:
takes
namespace: this maps names of input dependencies to the paths of these dependenciesvalues
namespace: this maps names of variables to the values of those variables.produces
namespace: this maps names of output dependencies to explicit paths. This should not be used directly really, but it’s useful forModuleContext.is_output_explicit
method.outputs
namespace: this maps names of output dependencies to their paths.
When the module finishes executing in exec mode, all of the dependencies
described in outputs
should be present.
Module initialization/instantiation¶
In the __init__
method of module’s class, the following fields should be
set:
takes
- a list of symbolic dependency names for dependencies used by the moduleproduces
- a list of symbolic dependencies names for dependencies produced by the module.values
- a list of names given to the variables used withing the moduleprod_meta
- A dictionary which maps product names to descriptions of these products. Those entries are optional and can be skipped.
Qualifiers/decorators¶
By default the presence of all the dependencies and values is mandatory
(In case of produces
that means that the module always has to produce the listed
dependencies). This can be changed by “decorating” a name in one of the following
ways:
‘
?
’ suffixIn
takes
- the dependency is not necessary for the module to executeIn
produces
- the dependency may be produced, but it is not guaranteed.In
values
the value is not required for the module to execute. Referring to it throughModuleContext.values.value_name
won’t raise an exception if the value is not present, insteadNone
will be returned.
‘
!
’ suffixIn
produces
- the dependency is going to be produced only if the user provides an explicit path for it.
Currently it’s impossible to combine both ‘!
’ and ‘?
’ together. This limitation
does not have any reason behind it other than the way the qualifier system
is implemented at the moment. It might be removed in the future.