Navigation :

EDR File Lua Agent

Introduction

The EdrFileLuaAgent is an synchronous helper for Lua scripts running within the LogicApp. The EDR File Agent may be used by any Lua script, regardless of which Service initiated that script.

The EdrFileLuaAgent monitors EDR files directly on disk, looking for new and modified files. It reads the EDR text lines into an in-memory cache and provides an API for querying those EDRs based on key field values.

The EdrFileLuaAgent is uses specifically for regression testing of Event Data Records generated by applications.

NOTE: Althought the EdrFileLuaAgent module lives within the EdrApp directory, it does not communicate directly with the EdrApp at run-time.

EDR File Lua Agent methods are accessed via the “n2.n2svcd.edr_file_agent” module:

local edr_file_agent = require "n2.n2svcd.edr_file_agent"

Configuring EdrFileLuaAgent

The EdrFileLuaAgent is configured within a LogicApp.

<?xml version="1.0" encoding="utf-8"?>
<n2svcd>
  ...
  <applications>
    ...
    <application name="Logic" module="LogicApp">
      <include>
        <lib>../apps/logic/lib</lib>
      </include>
      <config>
        <services>
          ...
        </services>
        <agents>
          <agent module="EdrApp::EdrFileLuaAgent" libs="../apps/edr/lib">
              <config max_edrs_per_stream="2000" edr_margin_secs="3">
                  <streams>
                      <stream key="n2logic" directory="/tmp/edr"/>
                  </streams>
              </config>
          </agent>
        </agents>
      </config>
    </application>
    ...
  </application>
  ...
</n2svcd>

Under normal installation, the following agent attributes apply:

Parameter Name	Type	XML Type	Description
`module`	`EdrApp::EdrFileLuaAgent`	Attribute	[Required] The module name containing the Lua Agent code.
`libs`	`../apps/db/lib`	Attribute	Location of the `module` for EdrFileLuaAgent.
`config`	Object	Element	Container for extended configuration for this Application instance.
`.max_edrs_per_stream`	Integer	Attribute	The maximum number of EDRs to cache for each stream. Excess EDRs are discarded. (Default = `1000`)
`.edr_check_ms`	Integer	Attribute	How often (in milliseconds) to check the EDR directory for new/modified EDR files and read new records. (Default = `200`)
`.edr_margin_secs`	Integer	Attribute	A grace margin when filtering for EDRs with a time-stamp that is more recent than "start of call". (Default = `2`)
`.edr_retire_secs`	Integer	Attribute	After the mtime for EDR file is this many seconds old, it is considered "retired" and the mtime will no longer be checked. (Default = `300`)
`.streams`	Array	Element	Array of `stream` elements defining EDR file streams.

EDR File Streams

The EdrApp writes EDRs in “streams” of files which are written to a common directory and which have a common filename prefix.

The EDR File Agent reads one or more of these streams. It does not need to process all streams written by the EdrApp, only those which are of interest to test scripts.

Each stream Object in the config.streams Array is configured as follows.

Parameter Name	Type	XML Type	Description
`key`	String	Attribute	[Required] The stream key, matching the key from the EDR App Configuration.
`directory`	String	Attribute	[Required] The full directory path where the EDR stream is being written, as configured in the EDR App Configuration or secondary directory after some post-processing.

The EdrFileLuaAgent API

All methods may raise a Lua Error in the case of exception, including:

Invalid input parameters supplied by Lua script.
Internal processing error.

Multiple Stream Keys

All methods may accept a single stream key to search, or a comma-separated list of stream keys.

In any case, then the first (or only) stream key in the list is the primary stream key, which will always be used initially to find the primary EDR, along with any other EDRs in that primary stream which are associated with the same instance GID.

Then if additional secondary stream keys are given, then those secondary streams will be searched to find any EDRs which have an upstream GID equal to the instance GID of the primary EDR(s).

Method Results

All methods return a single result with the following structure:

Parameter	Type	Description
`result`	Table	Container for the various returned parameters.
`.instance_idx`	String	The instance IDX for the instance whose EDR we matched. e.g. `6c064234`.
`.upstream_idxs`	Table	The instance IDXs for the instances upstream of the instance whose EDR we matched. The first entry is the instance IDX associated with the originating instance.
`.instance_gid`	String	The instance GID for the instance whose EDR we matched. e.g. `[ scp01~Logic~1626138304~6c064234 ]`.
`.upstream_gids`	Table	The instance GIDs for the instances that are upstream of the instance whose EDR we matched. The first entry is the instance GID associated with the originating instance.
`.secondary_gids`	Array of String	An array of secondary (downstream) GIDs, one for each additional (secondary) stream key in the original request. A `UNDEF` value will be placed in the array if the secondary stream did not match any secondary EDRs. e.g. `[ kirby~Logic~1699132449~42113d92 ]`.
`.secondary_idxs`	Array of String	Array of only the Instance IDX parts of the GIDs in the `secondary_gids` element described above. e.g. `[ 42113d92 ]`.
`.count`	Integer	The number of EDRs returned.
`.list`	List of Table	A structure which holds all returned EDRs. At the top level, this is a Lua table where the keys are the EDR event Types. Within each event type table entry, the value is a list of all EDRs of that type. Within that list each EDR is a table keyed by the EDR field names, plus header fields.

NOTE: The per-EDR tables in .list also include the following special field names which are parsed from the EDR header for each EDR.

__TIMESTAMP - The UTC generation time in YYYY-MM-DD HH:MM:SS.ss format.
__EVENT_TYPE - The event type e.g. INITIALDP.

.find_fields [Synchronous]

The find_fields method examines its cache of EDRs and begins by attempting to identify the first EDR:

From the EDR stream identified by stream_key.
Where the match_event event type matches the EDR.
Where all match_field key/values exact-match the EDR.
Where the “at” time of the EDR is later than the script execution start time or start_epoch (minus margin).

Once that first EDR is identified, it returns all EDRs:

From the EDR stream identified by stream_key.
Directly associated with the same instance as the first EDR.

An error is raised in the following cases:

If no matching EDR was found.
If EDRs were matched from more than one instance.

The find_fields method takes the following parameters.

Parameter	Type	Description
`stream_key`	String	[Required] The key of the stream to search, as configured in the `EdrFileLuaAgent` and `EdrApp`. This may be a comma-separated list of stream keys, as described above.
`match_event`	String	[Required] The required event type for the primary correlating EDR.
`match_fields`	Table	[Required] A table containing one or more key/value pairs to exact-match in an EDR.
`start_epoch`	Positive Integer	An epoch value to use as the start time to search from. Note that this is a value without sub-second precision. Consider waiting in scripts in order to avoid multiple instance matches. (Default: script execution start time)

Example (EDR Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

local arg_EDR_FILE = edr_file_agent.find_fields ('n2scp', 'INITIALDP', { CALLED = '4401234567' })
n2svcd.match_kpath_size (arg_EDR_FILE, 'list.INITIALDP', 1, true)
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].CALLING', '09773322')
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].TRIGGER', 'ORIG')

.confirm_none [Synchronous]

The confirm_none method acts in the opposite manner to the find_fields method by attempting to confirm any of:

That no EDRs exist in the EDR stream identified by stream_key, or
That no EDRs exist with an EDR event type that matches the match_event, or
That no EDRs exist where all match_field key/values exactly match the EDR.

In all cases, the “at” time of the EDR is treated identically to the find_fields method.

An error is raised if one or more matching EDRs were found.

All parameters for the confirm_none method are the same as the `find_fields' method.

Example (EDR Negative Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

local arg_EDR_FILE = edr_file_agent.confirm_none ('n2scp', 'INITIALDP', { CALLED = '4401234567' })
n2svcd.match_kpath_exists (arg_EDR_FILE, 'list.INITIALDP', false)

.find_gid [Synchronous]

The find_gid method examines its cache of EDRs and returns all EDRs:

From the EDR stream identified by stream_key.
Directly associated with the instance identfied by instance_gid.

The find_gid method takes the following parameters.

Parameter	Type	Description
`stream_key`	String	[Required] The key of the stream to search, as configured in the `EdrFileLuaAgent` and `EdrApp`. This may be a comma-separated list of stream keys, as described above.
`instance_gid`	String	[Required] The required instance GID that identifies the instance whose EDRs should be returned.

Example (EDR Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

arg_EDR_FILE = edr_file_agent.find_gid ('n2scp', 'scp01~LHO SCP~1626138304~6c064234')
n2svcd.match_kpath_size (arg_EDR_FILE, 'list.INITIAL_DP', 1, true)
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].CALLING', '09773322')
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].TRIGGER', 'ORIG')

.find_upstream_gid [Synchronous]

The find_upstream_gid method examines its cache of EDRs and begins by attempting to identify the first EDR:

From the EDR stream identified by stream_key.
Where the EDR was generated by an instance associated with the application identified by app_name.
Where the EDR was generated by an instance that was directly or indirectly caused by another instance identified by instance_gid.
Where the “at” time of the EDR is later than the script execution start time (minus margin).

Once that first EDR is identified, it return all EDRs:

From the EDR stream identified by stream_key.
Directly associated with the same instance as the first EDR.

Note that in general this find_upstream_gid method will not be required, since in most cases it is possible to achieve the same result more simply by specifying a comma-separated list of stream keys to the find_fields or find_gid methods.

An error is raised in the following cases:

If no matching EDR was found.
If EDRs were matched from more than one instance.

The find_upstream_gid method takes the following parameters.

Parameter	Type	Description
`stream_key`	String	[Required] The key of the stream to search, as configured in the `EdrFileLuaAgent` and `EdrApp`. This may be a comma-separated list of stream keys, as described above.
`instance_gid`	String	[Required] The instance GID that identifies a required preceding instance for the target EDRs.
`app_name`	String	[Required] The name of the application that is required to be associated with the instance for the primary correlating EDR.

Example (EDR Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

arg_EDR_FILE = edr_file_agent.find_upstream_gid ('n2scp', 'scp01~LHO SCP~1626138304~6c064234', 'Logic')
n2svcd.match_kpath_size (arg_EDR_FILE, 'list.LOGIC', 1, true)
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.LOGIC.[0].CUSTOM', 'Custom 1')