Google Git
Sign in
apache/nuttx-ntfc

NuttX Test Framework for Community.

Mirrored from https://github.com/apache/nuttx-ntfc.git
Clone this repo:

Branches

  1. 909820b Set up default protection ruleset for default and release branches by The Apache Software Foundation · 4 weeks ago main
  2. 9bba168 fix multi-session results placed in separate timestamp directories by raiden00pl · 8 weeks ago
  3. 09c4d7f Merge pull request #1 from szafonimateusz-mi/main by Matteo Golin · 2 months ago
  4. 3e00ab1 update link to NTFC default test cases by raiden00pl · 2 months ago
  5. d0111f0 initial asf configuration by raiden00pl · 2 months ago

NTFC

NuttX Test Framework for Community.

What is NTFC?

NTFC is a Python-based testing framework designed to automate the testing of NuttX RTOS on various platforms. It provides a structured way to develop, discover, and execute test cases against NuttX configurations.

Purpose

NTFC enables developers to:

  • Validate NuttX functionality across different configurations and hardware targets

  • Automate test execution on NuttX devices for real hardware and host-based target

  • Detect system failures including OS crashes and busy loop conditions

  • Dynamically discover applicable tests based on the NuttX image configuration

  • Standardize testing workflows using pytest, a widely-adopted Python testing framework

How It Works

NTFC acts as a bridge between pytest and NuttX devices/simulators:

  1. Loads a NuttX YAML configuration specifying the target device and build parameters

  2. Builds or prepares the NuttX image (optionally flashing to hardware)

  3. Discovers which test cases are applicable based on the NuttX configuration and ELF symbols

  4. Executes tests by sending NSH commands to the running NuttX instance

  5. Validates command output and monitors for system failures

This tool requires at least Python 3.10. Not tested with earlier versions of Python.

Features

  • NuttX simulator, QEMU and devices with serial port are supported

  • Run the command from NSH and compare with the expected output

  • OS crash detection and busy loop detection

  • Detection of supported test cases for a given NuttX image based on the configuration and image ELF file

Installation

NTFC can be installed from source using pip. PyPI package release is planned for future versions.

git clone <PATH_TO_NTFC_REPO>
cd ntfc
pip install -e .

NTFC requires Python 3.10 or later (not tested with older Python).

Optional dependencies

  • pytest-html - for HTML test reports
  • pytest-json - for JSON test reports

Documentation

Documentation is available in the Documentation/ directory. You can build the HTML documentation using Sphinx:

pip install -r Documentation/requirements.txt
cd Documentation
make html

The generated HTML will be available in Documentation/_build/html/index.html.

See Documentation/quickstart for a quick start guide and Documentation/usage for detailed usage information.

Preparing the Image for Testing

  • CONFIG_DEBUG_SYMBOLS is required for automatic command detection
  • CONFIG_DEBUG_FEATURES=y and CONFIG_DEBUG_ASSERTIONS=y are recommended for better error detection

Test Cases

Currently available test case repositories:

For guidance on writing your own test cases, see Documentation/writing-test-cases.

Contributing

To get started with developing NTFC, see CONTRIBUTING.md.

The roadmpa for the project can be found in ROADMAP.