Run Variables

Introduction

A Test Run consists of one or more Script Executions. By default a test script is executed only once within the test run, however the --total command-line flag on the lua_test program (or the equivalent attribute in the JSON API) can request that a test script be run multiple times for the purpose of load-testing or checking for potential conflict in shared resources.

Within the run, the specified test script is executed multiple times, typically limited by the number of script instances running at once (--backlog) or by the number of scripts initiated per second (--cps).

Each test script execution operates generally independently in terms of Lua local variables. The interesting aspects arise when considering non-local variables, in that:

• Lua Global system variables are always initialized prior to executing each instance, e.g. INSTANCE_INDEX, SCRIPT_NAME.
• Lua Global site-specific variables are configured to be initialized in the <globals> section of the LogicApp configuration.
• If Lua context re-use is enabled, Lua context globals set by a script execution will “leak” into the recycled execution.

This is not really a desirable feature. Lua context re-use is done for the purpose of efficiency, and this leaking of globals is an unwelcome side-effect.

Note also that it is not possible to predict when a Lua context will or will not be re-used for a second serial execution of a script within a test run, and also that scripts run in parallel within a run will not share contexts.

In summary, scripts should always avoid setting or modifying Lua global variables, to avoid problems in the case of Lua context re-use. Note that modules returned by the Lua require function are global variables for this purpose as described in the Common LUA Globals section.

The RestTestLuaService Run Variables API

The “correct” way for test scripts to share and report information within a multi-script test run is by using “Run Variables”. In the original JSON scripts, this feature was called “globals” and was accesed by the $g variable. These variables are now named “run variables” and are accessed by Lua methods.

The API for Run Variables is:

.run_set [Synchronous]

This method sets a named variable which can be read by other script executions within the same run.

The run_set method returns the input value parameter.

Example:

local n2svcd = require "n2.n2svcd"
local rts = require "n2.n2svcd.rest_test_service"

local rest = ...

rts.run_set ('X', 300)

.run_get [Synchronous]

This method gets a named variable from the common run-variables area.

The run_get method returns the value parameter which may be string, number, table, UNDEF, or nil (not present).

local n2svcd = require "n2.n2svcd"
local rts = require "n2.n2svcd.rest_test_service"

local rest = ...

local v = rts.run_get ('X')
rts.run_set ('X', v .. '/ABC')

.run_delta [Synchronous]

This method performs an atomic adjustment on a numeric stored value across all script instances within the run.

The value is initialized to zero if not currently set.

The run_delta method returns the number value after the delta has been applied.

Example:

local n2svcd = require "n2.n2svcd"
local rts = require "n2.n2svcd.rest_test_service"

local rest = ...

rts.run_delta ('COUNTER', 1)

.run_key_set [Synchronous]

This method sets a key value within a named HASH variable, the value may be read by other script executions within the same run.

If the named HASH variable does not exist, it will be initialised as an empty HASH.

If the named variable is not a HASH, an error will occur.

The run_key_set method returns the input value parameter.

Example:

local n2svcd = require "n2.n2svcd"
local rts = require "n2.n2svcd.rest_test_service"

local rest = ...

-- INSTANCE_IDX is a Lua global for this script.
rts.run_key_set ('COUNTERS', INSTANCE_IDX, 300)

.run_key_get [Synchronous]

This method gets a key value within a named HASH variable from the common run-variables area.

If the named HASH variable does not exist, it will be initialised as an empty HASH.

If the named variable is not a HASH, an error will occur.

The run_key_get method returns the value parameter which may be string, number, table, UNDEF, or nil (not present).

local n2svcd = require "n2.n2svcd"
local rts = require "n2.n2svcd.rest_test_service"

local rest = ...

local v = rts.run_key_get ('SEEN_CODES', '505')
if (v) then
    error ('Another Script Has Seen CODE 505!')
end

.run_key_delta [Synchronous]

This method performs an atomic adjustment on a numeric value stored within a key for a HASH common across all script instances within the run.

If the named HASH variable does not exist, it will be initialised as an empty HASH.

If the named variable is not a HASH, an error will occur.

The value within the keyed attribute is initialized to zero if not currently set.

The run_key_delta method returns the number value after the delta has been applied.

Example:

local n2svcd = require "n2.n2svcd"
local rts = require "n2.n2svcd.rest_test_service"

local rest = ...

local second = os.date ("%S")

local newval = rts.run_key_delta ('COUNTERS_BY_SECOND', second, 1)

The RestTestLuaService Run Variables Return

The current (or final) value of all run variables is returned in the {vars} attribute in the result structure sent back in response to QueryRun or CancelRun REST requests.