diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 187 |
1 files changed, 180 insertions, 7 deletions
@@ -56,14 +56,26 @@ cmake --build build erebos-tester --verbose ``` -To run tests from a given test file, pass it as command-line argument: +To run all tests from project configuration (see below), run the tester without any argument: ``` -erebos-tester path/to/script.test +erebos-tester +``` + +To run only some named tests, list the names on command line: +``` +erebos-tester FirstTest SecondTest +``` + +To run tests from a given test file, pass it as command-line argument (the path +must contain a slash, so use e.g. `./script.et` for script in the current +directory): +``` +erebos-tester path/to/script.et ``` To select single test from a file, use `:` separator: ``` -erebos-tester path/to/script.test:TestName +erebos-tester path/to/script.et:TestName ``` Configuration @@ -76,6 +88,7 @@ This is a YAML file with following fields: * `tool`: path to the test tool, which may be overridden by the `--tool` command-line option. * `tests`: glob pattern that expands to all the test script files that should be used. +* `timeout`: initial timeout for test steps like `expect`, given as `int` or `float`; defaults to `1` if not specified. Script language --------------- @@ -166,6 +179,7 @@ let re2 = /$str$re1/ # match '.' followed by any character #### boolean Result of comparison operators `==` and `/=`. +Values are `True` and `False`. #### network @@ -178,8 +192,11 @@ and used by `spawn` or network configuration commands. Members: +`ifname` +: Name of the primary network interface of the node. + `ip` -: String representation of node's IP address. +: String representation of the node primary IP address. `network` : The network which the node belogs to. @@ -193,6 +210,15 @@ Members: `node` : Node on which the process is running. +#### asset + +Represents an asset (file or directory), which can be used during test execution. + +Members: + +`path` +: Path to the asset valid during the test execution. + #### list Lists are written using bracket notation: @@ -204,7 +230,7 @@ List elements can be of any type, but all elements of a particular list must hav Used in the `for` command. -### Build-in commands +### Built-in commands ``` subnet <name> [of <network>] @@ -219,11 +245,12 @@ node <name> [on <network>] Create a node on network `<network>` (or context network if omitted) and assign the new node to the variable `<name>`. ``` -spawn as <name> [on (<node> | <network>)] +spawn as <name> [on (<node> | <network>)] [args <arguments>] ``` Spawn a new test process on `<node>` or `<network>` (or one from context) and assign the new process to variable `<name>`. When spawning on network, create a new node for this process. +Extra `<arguments>` to the tool can be given as a list of strings using the `args` keyword. The process is terminated when the variable `<name>` goes out of scope (at the end of the block in which it was created) by closing its stdin. When the process fails to terminate successfully within a timeout, the test fails. @@ -248,10 +275,11 @@ In that case the expect command has to have the `capture` clause with matching n Results of the captures are then assigned to the newly created variables as strings. ``` -flush [from <proc>] +flush [from <proc>] [matching <regex>] ``` Flush memory of `<proc>` output, so no following `expect` command will match anything produced up to this point. +If the `matching` clause is used, discard only output lines matching `<regex>`. ``` guard <expr> @@ -309,12 +337,157 @@ with <expr>: Execute `<test block>` with `<expr>` as context. ``` +multiply_timeout by <multiplier> +``` + +Modify the timeout used for commands like `expect` by multiplying it with `<multiplier>`. +The effect lasts until the end of the block. + +``` wait ``` Wait for user input before continuing. Useful mostly for debugging or test development. +### Functions + +When calling a function, parameters are usually passed using argument keywords +(in the case of built-in commands, those keywords are typically prepositions +like `on`, `from`, etc.), and apart from those, there can be at most one +parameter passed without a keyword. This is done in order to avoid the need to +remember parameter order and to make the behavior of each call as clear as +possible, even without looking up the documentation. + +To make the syntax unambiguous, the keywordless parameter can be passed as +a literal (number, string, etc.), or using parentheses. So this is ok: + +``` +expect /something/ from p +``` + +but if the regular expression is stored in a variable, the parameter needs to +be enclosed in parentheses: +``` +expect (re) from p +``` +or in a literal: +``` +expect /$re/ from p +``` + +### Defining functions + +Custom functions can be defined on the top level using `def` keyword, and with +the parameters either followed by `=` sign to return a value: +``` +def quadruple of x = 4 * x +``` + +or followed by `:` to define test block: +``` +def say_hello to p: + send "hello" to p + expect /hi/ from p +``` + +Those then can be invoked elsewhere: +``` +test: + spawn as p + say_hello to p +``` + +When defining a function, the unnamed parameter, if any, must be enclosed in +parentheses: +``` +def twice (x) = 2 * x +``` + +### Modules, exports and imports + +Each test script file constitutes a module. As such, it can export definitions +for other modules to use, and import definitions from other modules. The name +of each module must match the filename with the file extension removed, and is +given using the `module` declaration. This declaration, if present, must be +given at the beginning of the file, before other declarations. + +For example a file `test/foo.et` can start with: +``` +module foo +``` +This name is also implicitly assigned when the `module` declaration is omitted. + +In case of a more complex hierarchy, individual parts are separated with `.` +and must match names of parent directories. E.g. a file `test/bar/baz.et` +can start with: + +``` +module bar.baz +``` + +Such declared hierarchy is then used to determine the root of the project in +order to find imported modules. + +To export a definition from module, use `export` keyword before `def`: +``` +export def say_hello to p: + send "hello" to p + expect /hi/ from p +``` +or list the exported name in a standalone export declaration: +``` +export say_hello + +... + +def say_hello to p: + send "hello" to p + expect /hi/ from p +``` + +To import module, use `import <name>` statement, which makes all the exported +definitions from the module `<name>` available in the local scope. +``` +module bar.baz + +import foo +``` + +### Assets + +To provide the used test tool with access to auxiliary files needed for the +test execution, asset objects can be defined. The definition is done on the +toplevel using the `asset` keyword, giving the asset object name and a path to +the asset on the filesystem, relative to the directory containing the test +script: + +``` +asset my_asset: + path: ../path/to/file +``` + +Such defined asset object can then be used in expressions within tests or function definitions: + +``` +test: + spawn p + send to p "use-asset ${my_asset.path}" +``` + +The `my_asset.path` expression expands to a string containing path to the asset +that can be used by the spawned process `p`. The process should not try to +modify the file. + +Assets can be exported for use in other modules using the `export` keyword, +just like other definitions: + +``` +export asset my_asset: + path: ../path/to/file +``` + + Optional dependencies --------------------- |