Nova Preflight Checker

Running

After installing Nova Init on your platform, run the Nova Preflight Checker by executing this command from any folder:

$ nova_preflight_checker
=========================================== test session starts ============================================
platform linux -- Python 3.10.12, pytest-8.3.3, pluggy-1.5.0 -- /opt/nvidia/nova/python/venv/bin/python3
cachedir: .pytest_cache
rootdir: /opt/nvidia/nova
configfile: pytest.ini
plugins: timeout-2.3.1
timeout: 30.0s
timeout method: signal
timeout func_only: False
collected 92 items

compute/pfc/tests.py::test_service[nova-tegra-stats@1000] PASSED                                     [  1%]
compute/pfc/tests.py::test_service[nova-over-current-monitor@1] PASSED                               [  2%]
compute/pfc/tests.py::test_service[nova-power-monitor@1] PASSED                                      [  3%]
compute/pfc/tests.py::test_service[nova-nvpower] PASSED                                              [  4%]
compute/pfc/tests.py::test_service[nova-jetson-clocks] PASSED                                        [  5%]
time/pfc/tests.py::test_rtc0_device PASSED                                                           [ 21%]
metadata/pfc/tests.py::test_service[nova-metadata-generator] PASSED                                  [ 22%]
...
owl/pfc/tests.py::test_total_owl_devices PASSED                                                      [ 89%]
owl/pfc/tests.py::test_owl_capture[front_fisheye_camera] PASSED                                      [ 90%]
owl/pfc/tests.py::test_owl_capture[back_fisheye_camera] PASSED                                       [ 91%]
owl/pfc/tests.py::test_owl_capture[left_fisheye_camera] PASSED                                       [ 92%]
owl/pfc/tests.py::test_owl_capture[right_fisheye_camera] PASSED                                      [ 93%]
segway/pfc/tests.py::test_service[nova-can-setup] PASSED                                             [ 94%]
ssd/pfc/tests.py::test_ssd_write_speed PASSED                                                        [ 95%]
ssd/pfc/tests.py::test_filesystem_ssd PASSED                                                         [ 96%]
calibration/pfc/tests.py::test_calibration_host PASSED                                               [ 97%]
led/pfc/tests.py::test_service[nova-carter-led] PASSED                                               [ 98%]
isaac_cloud/pfc/tests.py::test_service[nova-isaac-ota] PASSED                                        [100%]

=========================================== 92 passed in 47.33s ============================================

Debugging

The PFC is organized modularly. Folders under /opt/nvidia/nova/ with an INSTALL.yaml are modules that setup a different part of the Nova system. Each module can have its own PFC tests to validate for the system. Here is an example of a PFC run that has failed some tests:

=========================================== test session starts ============================================
platform linux -- Python 3.10.12, pytest-8.3.3, pluggy-1.5.0 -- /opt/nvidia/nova/python/venv/bin/python3
cachedir: .pytest_cache
rootdir: /opt/nvidia/nova
configfile: pytest.ini
plugins: timeout-2.3.1
timeout: 30.0s
timeout method: signal
timeout func_only: False
collected 72 items

compute/pfc/tests.py::test_service[nova-tegra-stats@1000] PASSED                                     [  1%]
compute/pfc/tests.py::test_service[nova-over-current-monitor@1] PASSED                               [  2%]
compute/pfc/tests.py::test_service[nova-power-monitor@1] PASSED                                      [  4%]
...
ptp/pfc/tests.py::test_service[nova-ptp4l@eno1] FAILED                                               [ 50%]
...
hawk/pfc/tests.py::test_hawk_capture[front_stereo_camera] FAILED                                     [ 77%]
...
========================================= short test summary info ==========================================
FAILED hawk/pfc/tests.py::test_hawk_capture[front_stereo_camera] - AssertionError: Could not capture image for front_stereo_camera. Device ids:  3(left) 4(right)
...
FAILED ptp/pfc/tests.py::test_service[nova-ptp4l@eno1] - AssertionError: Service not in expected state (RUNNING). Found DEAD.
...
================================= 10 failed, 62 passed in 64.44s (0:01:04) =================================

The logs can be analyzed first by looking at where the tests are being called from. The first part before the :: is the path to the test on the system. It typically follows the structure <module>/pfc/<test>. This is relative to /opt/nvidia/nova/. For example, the failing test hawk/pfc/tests.py::test_hawk_capture[front_stereo_camera] is part of the hawk module and its test can be found at /opt/nvidia/nova/hawk/pfc/tests.py. It is running an image capture test on the front_stereo_camera. The other failing check is a standard service test ptp/pfc/tests.py::test_service[nova-ptp4l@eno1]. This is part of the ptp module and it’s testing that the service nova-ptp4l@eno1 is in its expected state. The Hawk can be debugged by checking that it’s connected and the GMSL board is powered. The service can be debugged by determining why it failed with systemctl status nova-ptp4l@eno1.

Add Custom Tests

To add custom tests to the PFC create a standard pytest file or directory and run:

$ nova_preflight_checker <custom_test_file_or_dir> <custom_test_file_or_dir> ...

Any number of custom tests can be forwarded to the PFC. The PFC will execute them alongside the tests for the Nova system.