Developers often blame inefficient loops, database queries or hardware for performance issues. However, while developing a scanner in FreePascal, I discovered a massive bottleneck unrelated to code or hardware: the antivirus. If you are writing applications that need to open thousands of files – even just to read a few bytes –, you might be hitting a wall you didn’t know was there.
The investigation
To highlight this issue, I built a simple scanner to browse a folder containing 10,000 files, each weighing about 70 KB. The scanner ran on Windows 11 on an SSD with an NVMe interface. I ran four different tests:
| Test name | Explanation |
|---|---|
NO_STREAM |
No stream is assigned to the file at all |
STREAM_NO_LOAD |
A TFileStream object is created, but no bytes are read |
LOAD_4KB |
A TFileStream object is created and 4k bytes are read |
LOAD_64KB |
A TFileStream object is created and 64k bytes are read |
Each test was run a first time and then immediately again. I relied on GetTickCount64 to measure the time it took to
complete a run. I made the necessary calls to timeBeginPeriod(1) and timeEndPeriod(1) to have the most accurate
timer resolution possible. Once the two runs were completed, I restarted my computer to ensure any cache was flushed.
Here are the results:
| Test name | First run (ms) | Second run (ms) |
|---|---|---|
NO_STREAM |
16 | 16 |
STREAM_NO_LOAD |
72,750 | 297 |
LOAD_4KB |
74,672 | 485 |
LOAD_64KB |
75,016 | 532 |
The jump from 16 ms to over 72 seconds in the first run of the STREAM_NO_LOAD test was surprising, as it only created
and freed a stream without reading data:
1Stream := nil;
2try
3 Stream := TFileStream.Create(FilePath, fmOpenRead or fmShareDenyNone);
4finally
5 Stream.Free;
6end;
While the second run of the same test was much faster, it was pointless. I needed the scan to be fast the very first time.
The culprit: the antivirus
After much research and hair pulling, I finally found the culprit: Windows Defender. Each time I created a TFileStream
object, the antivirus intercepted the call. When performed 10,000 times, this added significant overhead. Fortunately,
Windows Defender gave me the option to whitelist my application. I did so and ran a comparison test:
| Test name | First run (ms) | Second run (ms) |
|---|---|---|
NO_STREAM |
16 | 16 |
STREAM_NO_LOAD |
1,110 | 266 |
LOAD_4KB |
4,391 | 375 |
LOAD_64KB |
6,094 | 438 |
Whitelisting my application improved the STREAM_NO_LOAD first run performance by 66x (72.7 s down to 1.1 s). The first
run of the LOAD_4KB and LOAD_64KB tests also fared much better. Amazing!
Conclusion
If your application relies on high-volume file I/O:
-
Test whitelisting: try adding your application to your antivirus whitelist.
-
Educate your users: advise them that whitelisting your application can significantly improve performance for heavy scanning tasks.
-
Optimize your code: minimize opening and closing file handles, as you may pay the “antivirus tax” unless whitelisted.
Some people on the Internet claim that signing your application helps establish trust and can reduce the likelihood of the antivirus software aggressively intercepting file I/O calls. I haven’t tested that out, but if I do, I will update this article.