Thread safety issue in the logger

[cagrikymk]

Hello,
Probably I got unlucky and hit a weird bug when I was running vllm with TP 8. I got the following error from the quark library:

FileExistsError: [Errno 17] File exists: 'quark_logs'

From quark/shares/utils/log.py:20

        if not os.path.exists(debug_file_dir):
            os.makedirs(debug_file_dir)

I think this part is not multi-threading safe (potential race condition).

I think multiple threads passed the if guard and tried to create the folder which led to the error.

[fxmarty-amd]

Thank you, will fix.

[thiagocrepaldi]

@fxmarty-amd do we have an internal task for this? can we close it? is this part of 0.11?

[kewang-amd]

@cagrikymk Thanks for the report and the suggested fix! The logger code has been refactored in 0.12, eliminating the directory creation logic that caused this race condition.

Once 0.12 is released, please give it a try and let us know if the issue persists.