checking the simulation results

This page guides you on how to check the results of simulations performed in Scenario Runner.

Checking Real-time Simulation Status

You can check the real-time simulation status through the 'Simulation Status' window.

1. Overview

It displays the name of the currently running scenario file and the simulation time..

2. Status Monitor

It shows the names and status information of the Storyboard elements defined in the scenario file..

 List of Storyboard Element States

• init : State before the Storyboard starts

• standby : Waiting for activation state, transitions to running state upon satisfying specific conditions.

• running : Element is in an active state, performing the defined action. After completing the action, it either regresses to standby state or transitions to the complete state.

• complete : State after the element's defined action is completed and terminated.

• end : State after the storyboard is concluded.

3. Evaluation Index

It displays the safety evaluation metrics for the ego vehicle's driving.

Safety Evaluation Metrics

1. Safe Distance (m) : Distance to the front vehicle.

2. Time To Collision (sec) : Estimated time to collision with the front vehicle.

3. Way-off Distance (m) : Distance the vehicle deviates from the center of the lane.

4. Lateral Acceleration (m/s²) : Vehicle's lateral acceleration.

5. Longitudinal Acceleration (m/s²) : Vehicle's longitudinal acceleration.

6. Speed Excess (km/h) : Excess speed over the maximum regulated speed on the current road.

7. Speed Deficit (km/h) : Speed deficit below the minimum regulated speed on the current road.

Checking Simulation Results

You can check the simulation results through the 'Simulation Result' window.

Opens image in full screenOpen

1. Overview

• Total : Total number of scenarios included in the batch simulation.

• ✔️ : Number of scenarios that succeeded based on the conditions defined in the scenario's Evaluation item

• ❌ : Number of scenarios that failed based on the conditions defined in the scenario's Evaluation item.

• 

  : Number of scenarios terminated by the Storyboard StopTrigger or by the user

• Clear : Data reset button

2. Scenario List

You can check the success or failure of each scenario and the total simulation time for each scenario.

3. Simulation Result Summary

The simulation result consists of the following two items

• Safety evaluation metrics for the Ego vehicle (values for the most critical situations for each metric)

  • For example, the minimum distance to the front vehicle, maximum lateral deviation from the lane center, maximum lateral acceleration, etc.

• Status of Storyboard elements and error logs over time

4. Progress Bar and Log Folder

• Progress Bar: Indicates the number of scenarios completed in relation to the total number of scenarios.

• Open Log Folder : Opens the folder where logs are stored in File Explorer.

  • Log Storage Path

    • logs_scenario_runner/simulation_{batch simulation timestamp}/{batch simulation timestamp}{map name}{scenario name}eventlog.csv

    • logs_scenario_runner/simulation_{batch simulation timestamp}/{batch simulation timestamp}{map name}{scenario name}_statelog.csv

As simulation progresses, simulation logs accumulate in this folder, so be cautious of increasing file sizes.