Archiver test#

Configurable test for checking data when upgrading an archiver.

Features#

  • Check the data from two archivers match for a list of PVs and time period.

  • Load test configuration from a json formatted file.

  • Load data from previous test runs to test against an old version of an archiver.

Installation#

Clone the repo, then hopefully:

pip install .

Otherwise see Development to install a development version.

Usage#

Please see the Command-line Reference for usage of the command line interface.

An example python usage:

from archiver_test import test_set

time_now = datetime.now(tz=UTC) - timedelta(seconds=30)
archiver_test = test_set.ArchiverTestSet(
    "Test against old archiver.", "DTL-010:EMR-GT-003:Pos", time_now - timedelta(seconds=30), time_now, "archiver-01.tn.esss.lu.se", "archiver-linac-01.tn.esss.lu.se", None
)
await archiver_test.check()

Notes on calculations#

Mean#

The mean is retrieved using the Archiver post processor with mean_10(pv_name). The difference between each 10 seconds interval mean is calculated from new archiver to old, and then the difference itself is averaged.

Valid differences#

Two archiver clusters archiving the same PV with the same parameters can archive different data for a variety of reasons. With only minor differences in parameters the data can be very different.

  • Archiver loses events due to PV updating faster than sampling period. See Archiver Policy Recommendations Page for description of different sampling methods. Should display possible valid reasons why the archiver could be dropping values.

  • The following sampling parameters are different:

    • Per PV

      • Sampling Method

      • Sampling Period

    • Per Archiver Appliance: variables set in epicsarchiverap ansible repo)

      • Capacity Adjustment Constant (epicsarchiverap_setting_sampleBufferCapacityAdjustment)

      • Minimum Sampling Period (epicsarchiverap_setting_minimumSamplingPeriod)

      • Seconds to buffer - or how long until write the buffer. (epicsarchiverap_setting_secondsToBuffer) The sampling parameters can lead to the calculation of different sample capacities, which can give some indication as to how many samples are expected to be different.

  • Host machine performance differences:

    • The CPU and Memory usage can affect how fast a buffer can be written to disk, causing a delay on refresh of the buffer.

  • Access to the IOC:

    • Possibly the Gateway being used by the archiver could be under heavy load. (TODO test impact)

  • PV not updated in time period being tested. In this case, the archiver will return the last value before the time period. Depending on the behaviour of the PV and when the archiver started the connection to the PV this can cause discrepancies between the values.

    • On first connection, some PVs will return a timestamp at the time of connection, which will mean the events can be different.

    • A PV could return not a number, this will mean the timestamp behaviour could be strange.

  • PVs are being archived by different protocols, and an inherent difference in the protocol. For instance the alarm status.

Development#

The package is built and packaged with Hatch.

pip install hatch

Run tests:

hatch run test

Run formatting and check:

hatch run fmt

Run local docs:

hatch run docs-live

License#

Distributed under the terms of the MIT license, Archiver Test is free and open source software.