IEC 61850-10 Conformance Testing: What It Verifies

What the standard says and what it is for

IEC 61850-10 describes how to test that the communication of a device complies with the IEC 61850 series and how to measure the performance parameters that are declared. Its practical purpose, according to the text itself, is that the integrator can incorporate IEDs easily, operate them correctly and support the intended applications. This guide is based on edition 1 (2005). The standard leaves out of its scope who runs the tests and how results are certified, and the testing of devices that act as a client; the defined cases are those of a server.

To understand where this test sits in the life of a device, the standard recalls the sequence: internal development tests, type test, routine tests in production and, once in the project, FAT and SAT. A conformance test is the type test of the communication and, because communication forms a system, also the test of the IED within that system.

What conformance does not do

• It does not guarantee all functional and performance requirements; it reduces the risk of costly problems at integration.
• It does not replace the FAT and SAT of each project, which start from the client’s requirements. The FAT/SAT is done by the integrator and normally witnessed by the client.
• It does not cover device functions: IEC 61850 does not standardize the functions of the devices that communicate, so their failure modes are out of scope. Verification of functional applications, such as the use of GOOSE in a logic scheme, is not part of the conformance test either.
• It does not test all seven layers. Since the series does not define new communication stacks, conformance of the lower layers can be shown with documentation that a conforming stack is used; the standard test covers only the application according to ACSI.
• It does not certify. The text makes clear that the role of test laboratories and the certification of results are outside part 10.

The three manufacturer documents

The test is adapted to each device according to the capabilities the manufacturer declares in three documents.

The PICS serves three purposes: selecting the set of tests, making sure the tests that correspond to what is claimed are run, and serving as the basis for reviewing static conformance. Conformance requirements are divided into static (what the implementation must meet) and dynamic (those that arise from the protocol used).

In addition, the device must be delivered with its ICD file. The laboratory generates from it the SCD of the test system, as explained in SCL files.

What is tested on a server

A device is tested against a single test device. The device is treated as a black box: its inputs and outputs and its communication interface are used. The tests include, as applicable, positive and negative cases for:

1. documentation and version control (part 4);
2. configuration file against the standardized schema (part 6) and against the object model (parts 7-3 and 7-4);
3. stack implementation against the applicable mapping (parts 8-1, 9-1 and 9-2);
4. ACSI services implemented against their definition (part 7-2);
5. manufacturer-specific extensions, according to the general rules of the series.

Configuration and data model

The tests check that the ICD meets the XML schema, that it matches the data, types and services the device actually exposes, and that, when parameters are changed in the SCD and the device is configured with its tool, the online configuration matches the file. In the data model, the presence of the mandatory objects of each node (M) and of the conditional ones (C) is verified, along with the type of each data item and that the values are in the specified range. Extensions are reviewed according to the namespace rules and must be documented in the MICS. The length of names, the organization of the functional constraints and the naming of control blocks and logs are also verified; the subject is covered in data names in MMS.

Service models

The cases are grouped by part 7-2 model:

A case is mandatory when the model and the service are declared as supported in the PICS, and the interpretation of pass or fail depends on the declared capabilities.

Positive and negative: examples

A positive test verifies normal conditions, normally with an affirmative response. A negative test verifies abnormal conditions, normally with a negative response. Examples taken from the text:

There is also a combined case: keep reports, logs, GOOSE and time synchronization active while the whole range of services is requested, continuously for ten minutes, to check that the device keeps responding.

The standard asks that the tests concentrate on what is hard to test in a FAT or SAT, such as behavior under delayed, lost, duplicated or out-of-order packets, mismatched names or types, exceeded limits, paths of the control state machine and simultaneous commands from several clients. On control, see MMS: reports, controls and files; on reporting and sampled values, see the guides on GOOSE and sampled values. The acronyms and terms of the standard (LN, LD, DO, CDC, SCL, ACSI) are grouped in IEC 61850 Glossary.

Acceptance criteria

Each case ends in one of three verdicts, taken from the ISO/IEC 9646 series:

• Pass: the observed result shows conformance with the requirement and there were no invalid events.
• Fail: the result shows at least one non-conformance or contains an invalid event.
• Inconclusive: neither of the two can be stated. It must always be resolved to know whether the origin is the standard, the implementation or the test procedure.

In general, a case passes if the device behaves as the series and the PIXIT say. When something is not specified, the device must keep responding to syntactically correct messages and ignore incorrect ones. The results must be reproducible in the same laboratory and in others.

The test report

The report includes: the list of procedure documents and standards cited with number, date and clause; the test equipment and the software used; data on the manufacturer, the applicant and the laboratory; the device name and all its hardware and firmware variants; the issue date; the name of the tester; a unique reference number; the list of cases run; and comments and problems found. For each case the report documents the description with objective, procedure and expected result, the reference to part and clause, a unique identifier, the result (pass, fail, inconclusive, not applicable, or empty if not tested) and the comparison with what was expected. Any change made to the device during the test is described in full.

The laboratory must have a quality system compliant with ISO 9001, a quality plan, an inspection and test plan with hold and witness points, and must allow audits by the applicant.

How to use it in a project

• Procurement: require the PICS, MICS, PIXIT and ICD of the device and review the supported services before choosing.
• Engineering: use the PIXIT to learn the limits (simultaneous clients, times, ranges).
• FAT and SAT: reuse the negative cases of part 10 as an integration checklist. See standards for FAT and SAT and the IEC 61850 test tools.

We can help with your tests

At Tiza Group we prepare and run the FAT and SAT of your IEC 61850 system: review of manufacturer documents, service tests, negative cases and a traceable report. We do it in our FAT and SAT testing service and with our IEC 61850 engineering. Field work is done in Colombia, or elsewhere on request, and document and configuration reviews can be done remotely, worldwide.

Related guides

• IEC 61850 test tools
• IEC 61850-10 performance testing: latency and time
• Standards for commissioning, FAT and SAT
• Test mode and simulation in IEC 61850
• IEC 61850 SCL files
• IEC 61850 MMS: reports, SBO controls and files
• IEC 61850 Glossary: Acronyms and Terms
• IEC 61850-4 System Life Cycle and Quality Assurance
• ICCP: Conformance Blocks 1 to 5 and What Each Does

References

• IEC 61850-10, edition 1 (2005): clauses 1, 3, 5, 6 and annex A.
• IEC 61850-7-2, annex A: PICS statement.
• ISO/IEC 9646: conformance testing methodology and framework.

Frequently asked questions