I recently ran into an issue on a server running version 11 of Symantec Endpoint Protection (SEP). I was able to track down the solution, and it greatly improved the performance of Symantec Endpoint Protection Manager (SEPM). Read on for the full details.
Essentially, SEPM was frequently locking up. It would work for a number of days, then randomly lock up. Workstations would lose their connection to SEPM, SEPM would stop downloading (let alone distributing) updates, and attempts to login to SEPM would fail. The easy fix was to restart the server (or just the SEPM service), but that was a temporary solution, at best.
Upon digging into the SEPM logs (specifically, the scm-server-0.log file, located in %programfiles%\symantec\symantec endpoint protection manager\tomcat\logs\), I saw a number of Java-related memory errors. Errors like this:
java.lang.OutOfMemoryError: GC overhead limit exceeded
This lead me to the on Symantec’s website. Here’s Symantec’s explanation of the problem:
The amount of dynamic memory allocated to the Java Virtual Machine / Java Runtime Environment is insufficient. Dynamic memory allocation (also known as heap-based memory allocation) is the allocation of memory storage for use in a computer program during the runtime of that program. It can be seen also as a way of distributing ownership of limited memory resources among many pieces of data and code. Dynamically allocated memory exists until it is released either explicitly by the programmer, or by the Garbage Collector (GC): – Garbage Collection (GC) is a type of automatic memory management. – Its purpose is to reclaim garbage that is memory occupied by objects that are no longer in use by a program. – It deallocates memory used by objects at runtime automaticallly by using various algorithms to assess the reachability and life cycle of an object. – The basic principles of garbage collection are: 1) Find data objects in a program that cannot be accessed in the future 2) Reclaim theresources used by those object.
Essentially, SEPM is written in Java, and the Java Virtual Machine used to run it is running out of memory. Nicely, it is possible to specify the amount of memory that SEPM allocates when running. There are two memory-related parameters that you can set. The first specifies how much memory SEPM should start with. The second specifies the maximum amount of memory SEPM is allowed to use. On my system, SEPM was set to start with only 64MB, and grow to a max of 256MB. Pretty low numbers considering the large amount of memory available to modern servers.
I set my system to start with 512MB, and grow to 512MB (the max amount of memory that SEPM 11 supports). I have to assume that SEPM 12 (the current version) supports more memory, as 512MB is pretty low. Upon making the change, the frequent SEPM lockups went away, resulting in much smoother operation.
While you can visit for the full details of the solution, here’s the gist of it:
While it’s a mute point, considering that SEPM 11 is now a bit older, the low memory settings that SEPM ships with seem a bit odd. A better solution would have been for SEPM to set a value appropriate to the amount of memory installed in the system, as part of the installation process.