User Guide

Our Test Automation uses pytest for extensive plugin support meeting industry level standard and configurable YAML based system for platform setup.

It supports multiple execution targets:

  • FVP (Fast Models)

  • FPGA (Remote hardware via SSH)

Features

  • Telnet-based multi-terminal session handling for Arm FVP integration (extendable to FPGA and SoC environments).

  • Pytest-based test execution with modular plugin extensions.

  • Log-based result tracking and verification, including ANSI/log filtering utilities.

  • Reusable command execution utilities with prompt detection and automatic login handling.

  • Platform-based configuration system using properties and YAML templates.

Getting Started

This section provides a detailed, step-by-step guide for setting up an automated test environment for Arm Automotive Software Reference Stack.

Each run creates a parent folder inside the central logs directory. The folder name follows this format: logs/<target>_<platform>_<timestamp>/

Inside that folder, normalized log files are generated depending on the target type.

For FVP platforms, the following files are created:

  • Boot log - e.g. <platform>_<timestamp>.log

  • Telnet console logs - e.g. telnet_<console name>_<port number>.log (one per console/port)

  • Command outputs - e.g. cmd_output.txt for captured command results

For FPGA platforms, the following files are created:

  • Boot log - e.g. <platform>_boot_<timestamp>.log Captures stdout and stderr of the remote boot command.

  • Remote run logs directory - i.e. remote_run_logs which contains the downloaded contents of the remote RUN_DIR. This includes UART log files and any additional artifacts generated during FPGA boot.

This consistent layout provides a clear way to inspect results, share logs, and debug issues.

Installation

Ensure Python 3.10 or higher is installed.

  1. Create Python environment Create and activate a virtual environment, then install dependencies in editable mode:

$ python3 -m venv venv
$ source venv/bin/activate

After activating the environment, install the Test Automation framework using:

$ cd test_automation
$ pip install -e .

Add New Test Cases

Test files should be placed under the tests/ directory and follow the naming convention test_*.py.

Test Structure

A typical test file should contain:

  1. A pytest test function or test class

  2. Usage of built-in fixtures provided by the framework

  3. Validation using framework helpers

Example:

def test_basic_example():
    assert True

Note

Refer to FVP and FPGA sections for target specific fixtures and helpers.

Refer to the target specific documentation for detailed setup and execution steps:

Note

  • The --platform argument must match the platform defined in the config file.

  • Default logging level is INFO.

  • To enable DEBUG logs, add --debug-logs.

  • Extra pytest arguments can be used: -rs --setup-show -vv.