Package org.apache.beam.fn.harness.status

Class MemoryMonitor

  • All Implemented Interfaces:
    java.lang.Runnable
    public class MemoryMonitor
extends java.lang.Object
implements java.lang.Runnable
    A runnable which monitors a server for GC thrashing.

    Note: Only one instance of this should be initialized per server and it should be done when the server starts running.

    This runnable works as follows:

    • It wakes up periodically and determines how much time was spend on garbage collection since the last time it woke up.
    • If the time spent in garbage collection in the last period of time exceeds a certain threshold, that period is marked as "being in GC thrashing"
    • It keeps track of the GC thrashing status of the last few periods.
    • Every time the runnable's thread wakes up, it computes the ratio (# monitored periods in GC thrashing) / (# monitored periods).
    • If this ratio exceeds a certain threshold, it is assumed that the server is in GC thrashing.
    • It can also shutdown the current jvm runtime when a threshold of consecutive gc thrashing count is met. A heap dump is made before shutdown.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static interface  MemoryMonitor.GCStatsProvider
      Abstract interface for providing GC stats (for testing).

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.lang.String describeMemory()
      Return a string describing the current memory state of the server.
      java.io.File dumpHeap()
      Dump the current heap profile to a file in the given directory and return its name.
      static MemoryMonitor fromOptions​(org.apache.beam.sdk.options.PipelineOptions options)  
      boolean isThrashing()  
      void run()
      Runs this thread.
      void stop()
      Request the memory monitor stops.
      @Nullable java.io.File tryToDumpHeap()
      Dumps the heap to a file and return the name of the file, or null if the heap should not or could not be dumped.
      void waitForResources​(java.lang.String context)
      Return only when the server is not in the GC thrashing state.
      void waitForThrashingState​(boolean desiredThrashingState)
      For testing only: Wait for thrashing status to be updated to given value.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • DEFAULT_SLEEP_TIME_MILLIS

        public static final long DEFAULT_SLEEP_TIME_MILLIS
        Amount of time (in ms) this thread must sleep between two consecutive iterations.
        See Also:
        Constant Field Values

      • LOCAL_HEAPDUMP_DIR_SYSTEM_PROPERTY_NAME

        public static final java.lang.String LOCAL_HEAPDUMP_DIR_SYSTEM_PROPERTY_NAME
        See Also:
        Constant Field Values

    • Method Detail

      • fromOptions

        public static MemoryMonitor fromOptions​(org.apache.beam.sdk.options.PipelineOptions options)

      • waitForThrashingState

        public void waitForThrashingState​(boolean desiredThrashingState)
        For testing only: Wait for thrashing status to be updated to given value.

      • stop

        public void stop()
        Request the memory monitor stops.

      • isThrashing

        public boolean isThrashing()

      • tryToDumpHeap

        public @Nullable java.io.File tryToDumpHeap()
        Dumps the heap to a file and return the name of the file, or null if the heap should not or could not be dumped.
        Returns:
        The name of the file the heap was dumped to, otherwise null.

      • run

        public void run()
        Runs this thread.
        Specified by:
        run in interface java.lang.Runnable

      • waitForResources

        public void waitForResources​(java.lang.String context)
        Return only when the server is not in the GC thrashing state.

      • dumpHeap

        public java.io.File dumpHeap()
                      throws javax.management.MalformedObjectNameException,
                             javax.management.InstanceNotFoundException,
                             javax.management.ReflectionException,
                             javax.management.MBeanException,
                             java.io.IOException
        Dump the current heap profile to a file in the given directory and return its name.

        NOTE: We deliberately don't salt the heap dump filename so as to minimize disk impact of repeated dumps. These files can be of comparable size to the local disk.

        Throws:
        javax.management.MalformedObjectNameException
        javax.management.InstanceNotFoundException
        javax.management.ReflectionException
        javax.management.MBeanException
        java.io.IOException

      • describeMemory

        public java.lang.String describeMemory()
        Return a string describing the current memory state of the server.