igor

Module

igor

Module summary

Igor: the Module Merger and Renamer.

Description

Igor: the Module Merger and Renamer.

The program Igor merges the source code of one or more Erlang modules into a single module, which can then replace the original set of modules. Igor is also able to rename a set of (possibly interdependent) modules, without joining them into a single module.

The main user interface consists of the functions merge/3 and rename/3. See also the function parse_transform/2.

A note of warning: Igor cannot do anything about the case when the name of a remote function is passed to the built-in functions apply and spawn unless the module and function names are explicitly stated in the call, as in e.g. apply(lists, reverse, [Xs]). In all other cases, Igor leaves such calls unchanged, and warns the user that manual editing might be necessary.

Also note that Erlang records will be renamed as necessary to avoid non-equivalent definitions using the same record name. This does not work if the source code accesses the name field of such record tuples by element/2 or similar methods. Always use the record syntax to handle record tuples, if possible.

Disclaimer: the author of this program takes no responsibility for the correctness of the produced output, or for any effects of its execution. In particular, the author may not be held responsible should Igor include the code of a deceased madman in the result.

For further information on Igors in general, see e.g. "Young Frankenstein", Mel Brooks, 1974, and "The Fifth Elephant", Terry Pratchett, 1999.

Data types

filename() = file:filename()
stubDescriptor() = {ModuleName, Functions, [Attribute]}

  • ModuleName = atom()

  • Functions = [{FunctionName, {ModuleName, FunctionName}}]

  • FunctionName = {atom(), integer()}

  • Attribute = {atom(), term()}

A stub module descriptor contains the module name, a list of exported functions, and a list of module attributes. Each function is described by its name (which includes its arity), and the corresponding module and function that it calls. (The arities should always match.) The attributes are simply described by key-value pairs.

syntaxTree() = erl_syntax:syntaxTree()

An abstract syntax tree. See the erl_syntax module for details.

Exports

create_stubs(Stubs::[stubDescriptor()], Options::[term()]) -> [string()]

Creates stub module source files corresponding to the given stub descriptors. The returned value is the list of names of the created files. See merge_sources/3 for more information about stub descriptors.

Options:

{backup_suffix, string()}
{backups, boolean()}
{printer, Function}
{stub_dir, filename()}
{suffix, string()}
{verbose, boolean()}

See merge/3 for details on these options.

See also: merge/3, merge_sources/3.

merge(Name::atom(), Files::[filename()]) -> [filename()]

Equivalent to merge(Name, Files, []).

merge(Name::atom(), Files::[filename()], Options::[term()]) -> [filename()]

Merges source code files to a single file. Name specifies the name of the resulting module - not the name of the output file. Files is a list of file names and/or module names of source modules to be read and merged (see merge_files/4 for details). All the input modules must be distinctly named.

The resulting source code is written to a file named "<em>Name</em>.erl" in the current directory, unless otherwise specified by the options dir and outfile described below.

Examples:

  • given a module m in file "m.erl" which uses the standard library module lists, calling igor:merge(m, [m, lists]) will create a new file "m.erl which contains the code from m and exports the same functions, and which includes the referenced code from the lists module. The original file will be renamed to "m.erl.bak".

  • given modules m1 and m2, in corresponding files, calling igor:merge(m, [m1, m2]) will create a file "m.erl" which contains the code from m1 and m2 and exports the functions of m1.

Stub module files are created for those modules that are to be exported by the target module (see options export, stubs and stub_dir).

The function returns the list of file names of all created modules, including any automatically created stub modules. The file name of the target module is always first in the list.

Note: If you get a "syntax error" message when trying to merge files (and you know those files to be correct), then try the preprocess option. It typically means that your code contains too strange macros to be handled without actually performing the preprocessor expansions.

Options:

{backup_suffix, string()}

Specifies the file name suffix to be used when a backup file is created; the default value is ".bak".

{backups, boolean()}

If the value is true, existing files will be renamed before new files are opened for writing. The new names are formed by appending the string given by the backup_suffix option to the original name. The default value is true.

{dir, filename()}

Specifies the name of the directory in which the output file is to be written. An empty string is interpreted as the current directory. By default, the current directory is used.

{outfile, filename()}

Specifies the name of the file (without suffix) to which the resulting source code is to be written. By default, this is the same as the Name argument.

{preprocess, boolean()}

If the value is true, preprocessing will be done when reading the source code. See merge_files/4 for details.

{printer, Function}

  • Function = (syntaxTree()) -> string()

Specifies a function for prettyprinting Erlang syntax trees. This is used for outputting the resulting module definition, as well as for creating stub files. The function is assumed to return formatted text for the given syntax tree, and should raise an exception if an error occurs. The default formatting function calls erl_prettypr:format/2.

{stub_dir, filename()}

Specifies the name of the directory to which any generated stub module files are written. The default value is "stubs".

{stubs, boolean()}

If the value is true, stub module files will be automatically generated for all exported modules that do not have the same name as the target module. The default value is true.

{suffix, string()}

Specifies the suffix to be used for the output file names; the default value is ".erl".

See merge_files/4 for further options.

See also: merge/2, merge_files/4.

merge_files(Name::atom(), Files::[filename()], Options::[term()]) -> {syntaxTree(), [stubDescriptor()]}

Equivalent to merge_files(Name, [], Files, Options).

merge_files(Name::atom(), Sources::[Forms], Files::[filename()], Options::[term()]) -> {syntaxTree(), [stubDescriptor()]}

Types:

Forms = syntaxTree() | [syntaxTree()]

Merges source code files and syntax trees to a single syntax tree. This is a file-reading front end to merge_sources/3. Name specifies the name of the resulting module - not the name of the output file. Sources is a list of syntax trees and/or lists of "source code form" syntax trees, each entry representing a module definition. Files is a list of file names and/or module names of source modules to be read and included. All the input modules must be distinctly named.

If a name in Files is not the name of an existing file, Igor assumes it represents a module name, and tries to locate and read the corresponding source file. The parsed files are appended to Sources and passed on to merge_sources/3, i.e., entries in Sources are listed before entries read from files.

If no exports are listed by an export option (see merge_sources/3 for details), then if Name is also the name of one of the input modules, that module will be exported; otherwise, the first listed module will be exported. Cf. the examples under merge/3.

The result is a pair {Tree, Stubs}, where Tree represents the source code that is the result of merging all the code in Sources and Files, and Stubs is a list of stub module descriptors (see merge_sources/3 for details).

Options:

{comments, boolean()}

If the value is true, source code comments in the original files will be preserved in the output. The default value is true.

{find_src_rules, [{string(), string()}]}

Specifies a list of rules for associating object files with source files, to be passed to the function filename:find_src/2. This can be used to change the way Igor looks for source files. If this option is not specified, the default system rules are used. The first occurrence of this option completely overrides any later in the option list.

{includes, [filename()]}

Specifies a list of directory names for the Erlang preprocessor, if used, to search for include files (cf. the preprocess option). The default value is the empty list. The directory of the source file and the current directory are automatically appended to the list.

{macros, [{atom(), term()}]}

Specifies a list of "pre-defined" macro definitions for the Erlang preprocessor, if used (cf. the preprocess option). The default value is the empty list.

{preprocess, boolean()}

If the value is false, Igor will read source files without passing them through the Erlang preprocessor (epp), in order to avoid expansion of preprocessor directives such as -include(...)., -define(...). and -ifdef(...), and macro calls such as ?LINE and ?MY_MACRO(x, y). The default value is false, i.e., preprocessing is not done. (See the module epp_dodger for details.)

Notes: If a file contains too exotic definitions or uses of macros, it will not be possible to read it without preprocessing. Furthermore, Igor does not currently try to sort out multiple inclusions of the same file, or redefinitions of the same macro name. Therefore, when preprocessing is turned off, it may become necessary to edit the resulting source code, removing such re-inclusions and redefinitions.

See merge_sources/3 for further options.

See also: epp_dodger, filename:find_src/2, merge/3, merge_files/3, merge_sources/3.

merge_sources(Name::atom(), Sources::[Forms], Options::[term()]) -> {syntaxTree(), [stubDescriptor()]}

Types:

Forms = syntaxTree() | [syntaxTree()]

Merges syntax trees to a single syntax tree. This is the main code merging "engine". Name specifies the name of the resulting module. Sources is a list of syntax trees of type form_list and/or lists of "source code form" syntax trees, each entry representing a module definition. All the input modules must be distinctly named.

Unless otherwise specified by the options, all modules are assumed to be at least "static", and all except the target module are assumed to be "safe". See the static and safe options for details.

If Name is also the name of one of the input modules, the code from that module will occur at the top of the resulting code, and no extra "header" comments will be added. In other words, the look of that module will be preserved.

The result is a pair {Tree, Stubs}, where Tree represents the source code that is the result of merging all the code in Sources, and Stubs is a list of stub module descriptors (see below).

Stubs contains one entry for each exported input module (cf. the export option), each entry describing a stub module that redirects calls of functions in the original module to the corresponding (possibly renamed) functions in the new module. The stub descriptors can be used to automatically generate stub modules; see create_stubs/2.

Options:

{export, [atom()]}

Specifies a list of names of input modules whose interfaces should be exported by the output module. A stub descriptor is generated for each specified module, unless its name is Name. If no modules are specified, then if Name is also the name of an input module, that module will be exported; otherwise the first listed module in Sources will be exported. The default value is the empty list.

{export_all, boolean()}

If the value is true, this is equivalent to listing all of the input modules in the export option. The default value is false.

{file_attributes, Preserve}

  • Preserve = yes | comment | no

If the value is yes, all file attributes -file(...) in the input sources will be preserved in the resulting code. If the value is comment, they will be turned into comments, but remain in their original positions in the code relative to the other source code forms. If the value is no, all file attributes will be removed from the code, unless they have attached comments, in which case they will be handled as in the comment case. The default value is no.

{no_banner, boolean()}

If the value is true, no banner comment will be added at the top of the resulting module, even if the target module does not have the same name as any of the input modules. Instead, Igor will try to preserve the look of the module whose code is at the top of the output. The default value is false.

{no_headers, boolean()}

If the value is true, no header comments will be added to the resulting module at the beginning of each section of code that originates from a particular input module. The default value is false, which means that section headers are normally added whenever more than two or more modules are merged.

{no_imports, boolean()}

If the value is true, all -import(...) declarations in the original code will be expanded in the result; otherwise, as much as possible of the original import declarations will be preserved. The default value is false.

{notes, Notes}

  • Notes = always | yes | no

If the value is yes, comments will be inserted where important changes have been made in the code. If the value is always, all changes to the code will be commented. If the value is no, changes will be made without comments. The default value is yes.

{redirect, [{atom(), atom()}]}

Specifies a list of pairs of module names, representing a mapping from old names to new. The set of old names may not include any of the names of the input modules. All calls to the listed old modules will be rewritten to refer to the corresponding new modules. The redirected calls will not be further processed, even if the new destination is in one of the input modules. This option mainly exists to support module renaming; cf. rename/3. The default value is the empty list.

{safe, [atom()]}

Specifies a list of names of input modules such that calls to these "safe" modules may be turned into direct local calls, that do not test for code replacement. Typically, this can be done for e.g. standard library modules. If a module is "safe", it is per definition also "static" (cf. below). The list may be empty. By default, all involved modules except the target module are considered "safe".

{static, [atom()]}

Specifies a list of names of input modules which will be assumed never to be replaced (reloaded) unless the target module is also first replaced. The list may be empty. The target module itself (which may also be one of the input modules) is always regarded as "static", regardless of the value of this option. By default, all involved modules are assumed to be static.

{tidy, boolean()}

If the value is true, the resulting code will be processed using the erl_tidy module, which removes unused functions and does general code cleanup. (See erl_tidy:module/2 for additional options.) The default value is true.

{verbose, boolean()}

If the value is true, progress messages will be output while the program is running; the default value is false.

Note: The distinction between "static" and "safe" modules is necessary in order not to break the semantics of dynamic code replacement. A "static" source module will not be replaced unless the target module also is. Now imagine a state machine implemented by placing the code for each state in a separate module, and suppose that we want to merge this into a single target module, marking all source modules as static. At each point in the original code where a call is made from one of the modules to another (i.e., the state transitions), code replacement is expected to be detected. Then, if we in the merged code do not check at these points if the target module (the result of the merge) has been replaced, we can not be sure in general that we will be able to do code replacement of the merged state machine - it could run forever without detecting the code change. Therefore, all such calls must remain remote-calls (detecting code changes), but may call the target module directly.

If we are sure that this kind of situation cannot ensue, we may specify the involved modules as "safe", and all calls between them will become local. Note that if the target module itself is specified as safe, "remote" calls to itself will be turned into local calls. This would destroy the code replacement properties of e.g. a typical server loop.

See also: create_stubs/2, rename/3, erl_tidy:module/2.

parse_transform(Forms::[syntaxTree()], Options::[term()]) -> [syntaxTree()]

Allows Igor to work as a component of the Erlang compiler. Including the term {parse_transform, igor} in the compile options when compiling an Erlang module (cf. compile:file/2), will call upon Igor to process the source code, allowing automatic inclusion of other source files. No files are created or overwritten when this function is used.

Igor will look for terms {igor, List} in the compile options, where List is a list of Igor-specific options, as follows:

{files, [filename()]}

The value specifies a list of source files to be merged with the file being compiled; cf. merge_files/4.

See merge_files/4 for further options. Note, however, that some options are preset by this function and cannot be overridden by the user; in particular, all cosmetic features are turned off, for efficiency. Preprocessing is turned on.

See also: compile:file/2, merge_files/4.

rename(Files::[filename()], Renamings) -> [string()]

Equivalent to rename(Files, Renamings, []).

rename(Files::[filename()], Renamings, Options::[term()]) -> [string()]

Types:

Renamings = [{atom(), atom()}]

Renames a set of possibly interdependent source code modules. Files is a list of file names of source modules to be processed. Renamings is a list of pairs of module names, representing a mapping from old names to new. The returned value is the list of output file names.

Each file in the list will be read and processed separately. For every file, each reference to some module M, such that there is an entry {<em>M</em>, <em>M1</em>} in Renamings, will be changed to the corresponding M1. Furthermore, if a file F defines module M, and there is an entry {<em>M</em>, <em>M1</em>} in Renamings, a new file named <em>M1</em>.erl will be created in the same directory as F, containing the source code for module M, renamed to M1. If M does not have an entry in Renamings, the module is not renamed, only updated, and the resulting source code is written to <em>M</em>.erl (typically, this overwrites the original file). The suffix option (see below) can be used to change the default ".erl" suffix for the generated files.

Stub modules will automatically be created (see the stubs and stub_dir options below) for each module that is renamed. These can be used to redirect any calls still using the old module names. The stub files are created in the same directory as the source file (typically overwriting the original file).

Options:

{backup_suffix, string()}
{backups, boolean()}
{printer, Function}
{stubs, boolean()}
{suffix, string()}

See merge/3 for details on these options.

{comments, boolean()}
{preprocess, boolean()}

See merge_files/4 for details on these options.

{no_banner, boolean()}

For the rename function, this option is true by default. See merge_sources/3 for details.

{tidy, boolean()}

For the rename function, this option is false by default. See merge_sources/3 for details.

{no_headers, boolean()}
{stub_dir, filename()}

These options are preset by the rename function and cannot be overridden by the user.

See merge_sources/3 for further options.

See also: merge/3, merge_files/4, merge_sources/3.

Richard Carlsson [email protected]

© 2010–2017 Ericsson AB
Licensed under the Apache License, Version 2.0.