This folder contians the start of some network vector/replay tests.

record_proxy.py is a recording proxy that can record a NDS1 session between a live server and client out to a json file.

sock_test.py can read the recording generated by record_proxy and play back the server responses.

replay.py is some common logic between the two scripts.



To record a session against the LLO DTS you might run the proxy as:

record_proxy.py -r llodts0.ligo-la.caltech.edu:31200

you could then start a client such as 'lho-prod.py'

This will accept connections on localhost:8088 and proxy them to the llodts0 machine.  The output would be captured in proxy_script-1.json.


To play back a session the following command could be run:

sock_test.py -i proxy_script-1.json

you could then start a client such as 'lho-prod.py' to see if the command sequence between the server and the client is the same.

This will playback the script generated by the proxy recorder and generate an error if a difference is found.

By default sock_test.py listens on localhost:0.  So you will need to get the port to connect to from its output.  If you would like to specify a specific interface and/or port use the -l option.  -l address:port.

Running the sock_test.py script in an automated test.  By default sock_test.py starts up and waits for connections.  It can be told to start up, and launch an external command/test.  Then it will run the test and exit.  Simply add the command name to the end of the arguments.  Here are some examples

./sock_test.py -l localhost:31204 -i test-json/llodts0-py.json python2.7 ./test-scripts/llodts0-query.py
./sock_test.py -i test-json/llodts0-nds_query.json bash ./test-scripts/llodts0-cmd.sh

These commands start up a replay server and then run a test program against the server.  On success the exit code will be 0.  On failure it will not be 0.

When running a test the following environment variables are set for the test client.

NDS_TEST_HOST - set to the hostname to bind to (typically localhost)
NDS_TEST_PORT - set to the port on NDS_TEST_HOST to connect to
NDS2_CHANNEL_DB_DIR - The database dir for the client.  Currently this points to an empty directory.

Because this is a replay server, not a full nds1/2 server it is currently problematic to try and test the state of systems with an existing channel cache database.  This will hopefully be addressed in the near future.


The proxy output is in a simple text format, JSON for now.

The top level object is a dictionary of scripts, keyed by name.  By default record_proxy.py and sock_test.py write and read from the 'recorded_script' entry.

There are various types of script entries.

1. compound script
  - This contains a list of sub scripts that are run
  - keys
   - type -> "compound"
   - scripts -> An array of script objects
2. basic scripts
  - This is a simple command/response pairing
  - keys
   - type ->  "script"
   - command -> The command input
   - response -> Textual response (if it can be cleanly encoded)
   - response_blob -> A reference to binary file containing the response.  This is used for large responses or binary data.
   - responses -> An array of textual responses that should be concatenanted together to get the full response.  This is used to keep the response string from being overly long.
  - notes, only one of response, response_blob, or responses must be present.
3. named scripts
  - These refer to other top level scripts and allow reuse
  - keys
    - type -> "named"
    - name -> Key into the top level script list.  If a cycle is generated, the sock_test.py script will probably crash after a while.
4. external scripts
  - This script loads in an external json script file.  Currently it loads all the scripts into the global script namespace.  This may change in the future.
  - keys
    - type -> "external"
    - file -> Path to the file to load.  Currently this will take any path.  It may be change to read from a specific directory in the future.
    - prefix/namespace -> Unsupported.  May be the key name that designates a namespace that the command is to live in when/if namespacing is added.
5. command scripts
  - These scripts execute an actions.
  - keys
    - type -> "command"
    - action -> The command name to execute
    - args -> An array of arguments for the command
  - implemented commands
    - sleep - takes one argument, the number (as a float) of seconds to sleep.  Partial seconds are allowed.
    - import - takes one argument, the name of a module to load.  The module must define 2 commands, validate_action(action, args) and execute_action(action, args, inp, outp).  validate_action is given a command name and an array of arguments and verifies that they are acceptabile.  An exception should be raised if they are not.  execute_action is given the command name and the arguments as an array as well as two file handles (input and output).  The names of the actions have the initial module name stripped off.
  - module commands
    - modules may implement commands.  The command names are prefixed by the module name.
  - nds1 module
   - register_chan
    - arguments are the channel information name, rate, type, tp_num, group, units, gain, slope, offset
   - gen_sc3_12_1 - generate the output of status channels 3 { named channel };
    - arguments are the name of the channel to output.  The channel must have been registered with register_chan prior to this.

Data blobs are stored in compressed form in the blobs directory.  The file names must be the sha1 hashes of their contents.  They may be compressed by gzip with a .gz extension.  We probably do not want to stick this in the svn.  The blobs can be uploaded to a server and pulled down on demand if the environment variable 'REPLAY_BLOB_REMOTE_CACHE' is set.

For now there are some blobs stored at http://www.ligo-wa.caltech.edu/~jonathan.hanks/blobs
