Skip to content

SCENARIO_GUIDE.rst

Latest commit

 

History

History
258 lines (190 loc) · 10.1 KB

SCENARIO_GUIDE.rst

File metadata and controls

258 lines (190 loc) · 10.1 KB
Raw

Writing scenarios

A scenario has three parts - a configuration in the header, description of the environment and sequential steps. The environment is described as a set of mock answers valid within a time range using RANGE keyword. A mocked DNS message is represented by an ENTRY block and may contain patterns for matching, DNS header and records in appropriate sections, it looks similar to a dig(1) output which makes generating test cases easier.

Each STEP keyword then describes a single step in the test scenario. Lines started with semicolon or hash are treated as comments.

Configuration

Configuration part is a list of "key : value" pairs, one pair per line. Configuration have no explicit start, it's assumed it starts immediately at scenario file begin.

  • query-minimization : on

    value "on" means query minimization algorithm will be used; any other value means query minimization algorithm will not be used.

  • stub-addr : ipv4-addr

    address, which will be listened by Python fake dns server immediately after startup. When configuration files will be generated by Jinja2 engine, ROOT_ADDR template variable will be replaced by this address.

  • thrust-anchor : string value

    Trust anchor, can be used for DNSSEC-related scenarios

  • val-override-timestamp : string value

    POSIX timestamp; the system time will be reported to binary under the test

  • val-override-date : string value

    the system time to be reported to binary under the test; string value has format YYYYMMDDHHMMSS, so 20120420235959 means Fri Apr 20 23:59:59 2012

  • features : jinja2_var1=v1;jinja2_var2=v2;...;jinja2_varN=vN

    semicolon-separated list of key=value pairs, all keys and values are strings; contains user-defined jinja2 template variables which can be used for improving flexibility of configuration generation process; when variable value is not needed, it can be omitted as well as equal sign; features configuration key can be repeated, so

    **features** : jinja2_var1=v1;jinja2_var2=v2;...;jinja2_varN=vN

    equal to

    **features** : jinja2_var1=v1
**features** : jinja2_var2=v2
...
**features** : jinja2_varN=vN

  • feature-list : jinja_var=value

    single key=value pair; key and value are strings; similar to features, it contains user-defined jinja2 template variable, but difference is that jinja_var behaves as list of values; each feature-list row appends v to the list named jinja_var; you can use loops in your jinja template to iterate through it

    feature-list: policy=policy:add(policy.suffix(policy.DENY, {todname('example.com')}))
feature-list: policy=policy:add(policy.all(policy.FORWARD('2001:DB8::1')))

Example

; config options
        query-minimization: on
        stub-addr: 193.0.14.129     # K.ROOT-SERVERS.NET.
        trust-anchor: ". 3600 IN DS 17272 13 4 B87AD8C76DC2244E7AA57285057BF533F2E248CC8D7E1A071D8A3837A711A5EA705C4707E6E8911DA653BE1AE019927B"
        val-override-date: "1442323400"
        features: dns64
        features: dns64_prefix = fe80::21b:aabb:0:0
CONFIG_END

Scenario

Scenario part starts with SCENARIO_BEGIN and ends with SCENARIO_END statements. SCENARIO_BEGIN can be followed by scenario description.

Example

SCENARIO_BEGIN Test basic query minimization www.example.com.
...
SCENARIO_END

RANGE datablock

RANGE datablock starts with RANGE_BEGIN and ends with RANGE_END statements. This datablock contains data, used Python fake dns server to make answers to binary's under test queries.

Format:

RANGE_BEGIN n1 n2
    ADDRESS addr
...
RANGE_END
  • n1 and n2 respectively minimal or maximal step ids (see below) to which this range can be applied.
  • addr - IP address for which RANGE datablock is prepared; this statement can be omitted.

Datablock will be used for fetching reply to query only for these steps, whose identificators greater then or equal n1 and lesser then or equal n2. Also one of the next condition must be met :

  • addr is not set
  • addr is set, and query is directed to this addr
  • address, to which query is directed, can not be found within the range addresses list for whole scenario

STEP datablock

STEP datablock starts with STEP statement and continues until to next STEP, RANGE or END_SCENARIO statement. This datablock defines action will be taken by testing environment - send next query to binary under test, send reply to binary under test, set faked system time or check the last answer.

Format

STEP id type [additional data]
  • id - step identificator, positive integer value; all steps must have different id's. This value used within RANGE datablock, see above.
  • type - step type; can be QUERY | REPLY | CHECK_ANSWER | TIME_PASSES ELAPSE <TIMESTAMP>
    • QUERY - at this step new query must be sent
    • REPLY - send answer to last query; steps of this type fired when eligible RANGE datablock can not be found
    • CHECK_ANSWER - last received answer must be checked; this step can have additional fields RETRY = integer value PAUSE = float value NEXT = integer value. This additional values are intended to ensure error recovery possibility. When answer checking failed, is possible to take step with predefined step id. For example, STEP CHECK_ANSWER RETRY = 3 PAUSE = 0.5 NEXT = 10 means that when current step fails, then step with id = 10 must be taken after pause. Pause duration is 0.5 seconds. Maximal number of retries is 3. When maximal number of retries is reached, scenario fails.
    • TIME_PASSES ELAPSE - new time must be set for binary under test; TIMESTAMP - POSIX timemestamp, it will be added to current system time.

ENTRY

ENTRY is an basic informational block, it has a DNS-message based structure. It contains all necessary data to perform action for which it was intended. Block starts with ENTRY_BEGIN and ends with ENTRY_END statements.

Format

ENTRY_BEGIN
MATCH <field list>
ADJUST <field list>
REPLY <flags>
SECTION <type>
   ...
RAW
   ...
ENTRY_END
  • MATCH <field list> - space-separated list of ENTRY block elements to be compared with elements of incoming query (answer); when all elements matches, this entry block will be used, otherwise next entry will be analyzed. <field list> can contain values :
    • opcode - check if the incominq query is a standard query (OPCODE is 0)
    • qtype - check if QTYPE fields of both question sections are equal
    • qname - check if domain name (QNAME) fields of question sections are equal
    • subdomain - check if domain from question section of incoming query (answer) is a subdomain of domain from question section of this ENTRY block.
    • flags - check if sets of dns flags (QR AA TC RD RA) are equal
    • rcode - check if response codes are equal
    • question,
    • answer,
    • authority,
    • additional - check if lists of RR sets for question,answer,authority and additional section respectively is equal
    • all - check if sets of dns flags are equal, rcodes and all sections presented in entry are equal to ones in incoming query (answer); incoming query (answer) can contain some extra sections which will not be compared
  • ADJUST <field list> - when ENTRY block is used as a pattern to prepare answer to incoming query, it must be preprocessed; values in <field list> defines actions will be taken:
    • copy_id - query id and domain name (question section QNAME field) only will be copied from incoming message
    • copy_query - whole question section will be copied from incoming message
  • REPLY <flags> - space-separated list of flags will be set in reply values can be used:
    • QR, AA, TC, RD, RA - i.e. standard dns flags
    • NOERROR, FORMERR, SERVFAIL, NXDOMAIN, NOTIMP, REFUSED, YXDOMAIN, YXRRSET, NXRRSET, NOTAUTH, NOTZONE, BADVERS - standard rcodes
    • DO - enable 'DNSSEC desired' flag
  • SECTION <type> - defines section of dns message, so <type> can be equal to QUESTION, ANSWER, AUTHORITY or ADDITIONAL each section contains rr sets like standard dns message

Example

SECTION QUESTION
www.example.com.      IN A
SECTION ANSWER
www.example.com.      IN A    10.20.30.40
SECTION AUTHORITY
example.com.  IN NS   ns.example.com.
SECTION ADDITIONAL
ns.example.com.       IN A    1.2.3.4
  • RAW - section used to sending raw dns messages. Contains a single-line data interpreted as hexadecimal string. This string will be sent to binary under test without any changes. Raw messages used to sending badly formed queries to binary under test. Queries assumed not be answered, so no waiting for answer is performed.Main goal of this behavior is to check if binary under test is able to serve valid queries after getting of series badly formed packets. So using RAW section in conjunction of other sections is meaningless. All sections other than RAW will be ignored. Also, ENTRY datablock can contain only one RAW section.

Example

RAW
    b5c9ca3d50104320f4120000000000000000

Writing load tests

The test harness support a simple benchmarking steps for writing load tests. Note that this is to test how the subject behaves under load, not for comparative benchmarking.

STEP REPLAY [query_count]
<qname> [qclass] <qtype>
...

This replays the list of queries described below replay step repetitively to the subject until query_count is reached. Following example replays 1000 queries (500 times each).

::
STEP REPLAY 1000 example.com A www.example.com AAAA

Tip

Define VERBOSE environment variable to see benchmarking results (queries sent, received and response rate).

SCRIPT EXAMPLE