Log file does not get generated

[bbiro2]

Hi everyone,

I built and am using Carla 0.9.11 on Windows 10. What I would like to achieve is to run and record my scenario (sourced from an OpenSCENARIO file) by using the ScenarioRunner module and to have the log file generated.

Since it did not work, I tried using a default scenario (FollowLeadingVehicle_1) to check whether the issue is coming from my OpenSCENARIO file, but it does not work with the default scenario neither.

The problem is that the log file only gets generated when I use Carla without ScenarioRunner (by running the default start_recording.py). Once I start using ScenarioRunner as well, and try running any scenario with the proper command (written here), it seems like it does not want to log anything, only the json criteria file gets generated.

Because of this, I believe that the client.start_recorder() function works well, because in case of any map ran in Carla with the recording on, it generates the log file. However when the same function gets called during the ScenarioRunner recording functionality, it does not generate it.

Did it happen to anyone or does anyone have some advices what should I try to make it work?

Thank you in advance for any advices!

[glopezdiest]

Hey @bbiro2, SR argument to record the simulation makes it such that the given path is always relative to your scenario runner folder, which is definitely a different behavior from the default one at the recorder (this line). Did you take this into account when sending the path to SR?

[bbiro2]

Hi @glopezdiest,
thank you for the advice, and yes, I did take this into account. I have set my path properly, I believe that is the reason why I also got my json criteria file generated into the right folder, but unfortunately this is not the case regarding the log file.

[Kin-Zhang]

I also met this problem when I use CARLA from here https://carla-releases.s3.eu-west-3.amazonaws.com/Linux/CARLA_0.9.10.1.tar.gz on ubuntu 18.04, also record from leaderboard client.start_recorder

[Kin-Zhang]

I may find a result for my case, since I use the Client on a python docker container, it may need root to create a log file maybe....

[Kin-Zhang]

OK, now I knew that it's only happen when I use docker container to record log file, I tried directly in my system with log, it have log files

[rebelmachina]

Hey @Kin-Zhang glad it worked out for you. I'd have a few questions:

1. do you mean, you ran both carla and the scenario_runner locally and not on docker ?
2. btw did you try also running scenario_runner and carla on docker? did that solve the issue of the missing .log file ?

[Kin-Zhang]

1. yes
2. no I didn't try this way, let me know if you tried.

[neilsambhu]

I am using CARLA 0.9.13 on Pop!_OS 21.10 (based on Ubuntu 21.10). Repository: https://github.com/neilsambhu/carla-simulator3

I don't have any *.log file as output to hard disk. I am also using the FollowLeadingVehicle_1 scenario. I know the "ERROR: failed to destroy actor..." message is more like a warning (carla-simulator/scenario_runner#852).

(carla-simulator3) nsambhu@SAMBHU19:/data/data1/GitHub/carla-simulator3/scenario_runner$ python scenario_runner.py --scenario FollowLeadingVehicle_1 --record recording_files --reloadWorld
Preparing scenario: FollowLeadingVehicle_1
Neil got here 1
current directory /data/data1/GitHub/carla-simulator3/scenario_runner
recorder_name .//recording_files/FollowLeadingVehicle_1.log
Neil left here 1
ScenarioManager: Running scenario FollowVehicle
Not all scenario tests were successful
Please run with --output for further information
Neil got here 2
recorder_name .//recording_files/FollowLeadingVehicle_1.log
Neil left here 2
Destroying ego vehicle 374
ERROR: failed to destroy actor 374 : unable to destroy actor: not found 
No more scenarios .... Exiting

Similarly, my *.json output is filled with blank or null values:

(carla-simulator3) nsambhu@SAMBHU19:/data/data1/GitHub/carla-simulator3/scenario_runner$ python scenario_runner.py --scenario 
FollowLeadingVehicle_1 --record recording_files --reloadWorld
Preparing scenario: FollowLeadingVehicle_1
ScenarioManager: Running scenario FollowVehicle
Not all scenario tests were successful
Please run with --output for further information
Neil got here 1
file_name .//recording_files/FollowLeadingVehicle_1.json
Neil left here 1
Neil got here 2
criteria_dict {'CollisionTest': {'children': [], 'feedback_message': '', 'blackbox_level': <BlackBoxLevel.NOT_A_BLACKBOX: 4>, '_terminate_on_failure': False, 'test_status': 'SUCCESS', 'expected_value_success': 0, 'expected_value_acceptable': None, 'actual_value': 0, 'optional': False, 'list_traffic_events': [], '_collision_sensor': None, 'other_actor': None, 'other_actor_type': None, 'registered_collisions': [], 'last_id': None, 'collision_time': None, 'terminate_on_failure': False}}
Neil left here 2
Destroying ego vehicle 197
ERROR: failed to destroy actor 197 : unable to destroy actor: not found 
No more scenarios .... Exiting

Through the debugging process (after 4/11/2022 6:01:11 PM on https://github.com/neilsambhu/carla-simulator3 ), (1) CarlaUE4.sh will show Town01 and (2) manual_control.py will show Town10HD_Opt. I have --reloadWorld parameter when calling scenario_runner.py, and I have not changed any of the default ports.

Additionally, CarlaUE4.sh outputs "ERROR: Invalid session: no stream available with id [integer]". I tried to resolve this issue by adding parameters when calling CarlaUE4.sh

I assume (1) the "ERROR: Invalid session" is connected to (2) the Town01 not loading in manual_control.py: manual_control.py to prevent the "Not all scenario tests were successful" message (carla-simulator/scenario_runner#566 (comment) ), which is connected to (3) lack of *.log and *.json files In my "recording_files" directory. Chronologically, I found these issues in the order 3, 2, 1.

[neilsambhu]

I reinstalled the OS of my machine, downgrading from Pop!_OS 21.10 to Ubuntu 20.04: https://github.com/neilsambhu/carla-simulator4. This time I installed Carla from Debian instead of from souce. I still get an error:

(carla-test) nsambhu@SAMBHU19:~/data1/GitHub/carla-simulator4/scenario_runner-0.9.13$ python scenario_runner.py --scenario FollowLeadingVehicle_1 --record recording_files --reloadWorld
scenario_runner.py:94: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if LooseVersion(dist.version) < LooseVersion('0.9.12'):
Preparing scenario: FollowLeadingVehicle_1
Traceback (most recent call last):
  File "scenario_runner.py", line 408, in _load_and_run_scenario
    self.client.start_recorder(recorder_name, True)
RuntimeError: time-out of 10000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000
time-out of 10000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000
terminate called after throwing an instance of 'carla::client::TimeoutException'
  what():  time-out of 10000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000
Aborted (core dumped)

[cleonardi1999]

I spent hours to solve this problem and again, the bug is so stupid. You should rename the description in FileHeader In OpenScenario File, change "CARLA:YourScenario" to "CARLAYourScenario".

":" is the reason why the log and json files cant be saved.

[st3lzer]

Hello, thanks for the quick reply! I am working with route files (xml), which somehow reference to scenario files I think.
An example is this: https://github.com/autonomousvision/carla_garage/blob/leaderboard_2/data/50x38_Town12/Accident/1054_0.xml

[cleonardi1999]

i cant help much because i only work with OpenScenario, is there any error message when you run the --record flag??

[st3lzer]

That's the thing: There is no error message. When I run the client locally and the server in a docker, I just dont get a recording log. When I use the same scripts and run both locally, I get a recording log.

[orjahren]

I have a similar issue. Based on the code on the scenario runner, it seems that the log file will be made by Carla itself (self.client.start_recorder(recorder_name, True)) and not runner wheras the Json is made by the runner - which could explain why it is non-trivial for this to just work.
Note how the file is opened, as far as i can understand this will be done by the actual Carla process, i.e. the container in our case.

Update: I was able to hack around this issue by mounting the relevant folder on my host, allowing the Carla container to write to it. Just works.

[tri218138]

I have a similar issue. Based on the code on the scenario runner, it seems that the log file will be made by Carla itself (self.client.start_recorder(recorder_name, True)) and not runner wheras the Json is made by the runner - which could explain why it is non-trivial for this to just work. Note how the file is opened, as far as i can understand this will be done by the actual Carla process, i.e. the container in our case.

Update: I was able to hack around this issue by mounting the relevant folder on my host, allowing the Carla container to write to it. Just works.

I dit it successfully, thanks. We target to find .log in CarlaUE4/Saved folder. In my case (macOS)

find "/Applications/CARLA.app/Contents/drive_c/" -name "*PedestrianCrossing*.log" -type f 2>/dev/null

[tri218138]

In case, it was generated at ...CarlaUE4/Saved... .If you check your carla/libcarla.pyi file in carla packages

    def start_recorder(self, filename: str, additional_data: bool = False):
        """Enables the recording feature, which will start saving every information possible needed by the server to replay the simulation.

        Args:
            `filename (str)`: Name of the file to write the recorded data. A simple name will save the recording in 'CarlaUE4/Saved/recording.log'. Otherwise, if some folder appears in the name, it will be considered an absolute path.\n
            `additional_data (bool, optional)`: Enables or disable recording non-essential data for reproducing the simulation (bounding box location, physics control parameters, etc). Defaults to False.
        """

All of you need is export SCENARIO_RUNNER_ROOT with your absolute path that will be concatenated with --record. For example:

SCENARIO_RUNNER_ROOT="/Users/<yourname>/path/to/project/root"
export $SCENARIO_RUNNER_ROOT

# your carla client python command may like
python scenario_runner ... --record output/recordings