Class ScratchFile

  • All Implemented Interfaces:
    Closeable, AutoCloseable

    public class ScratchFile
    extends Object
    implements Closeable
    Implements a memory page handling mechanism as base for creating (multiple) RandomAccess buffers each having its set of pages (implemented by ScratchFileBuffer). A buffer is created calling createBuffer().

    Pages can be stored in main memory or in a temporary file. A mixed mode is supported storing a certain amount of pages in memory and only the additional ones in temporary file (defined by maximum main memory to be used).

    Pages can be marked as 'free' in order to re-use them. For in-memory pages this will release the used memory while for pages in temporary file this simply marks the area as free to re-use.

    If a temporary file was created (done with the first page to be stored in temporary file) it is deleted when close() is called.

    Using this class for RandomAccess buffers allows for a direct control on the maximum memory usage and allows processing large files for which we otherwise would get an OutOfMemoryError in case of using RandomAccessBuffer.

    This base class for providing pages is thread safe (the buffer implementations are not).

    • Constructor Detail

      • ScratchFile

        public ScratchFile​(File scratchFileDirectory)
                    throws IOException
        Initializes page handler. If a scratchFileDirectory is supplied, then the scratch file will be created in that directory.

        All pages will be stored in the scratch file.

        Parameters:
        scratchFileDirectory - The directory in which to create the scratch file or null to created it in the default temporary directory.
        Throws:
        IOException - If scratch file directory was given but don't exist.
      • ScratchFile

        public ScratchFile​(MemoryUsageSetting memUsageSetting)
                    throws IOException
        Initializes page handler. If a scratchFileDirectory is supplied, then the scratch file will be created in that directory.

        Depending on the size of allowed memory usage a number of pages (memorySize/PAGE_SIZE) will be stored in-memory and only additional pages will be written to/read from scratch file.

        Parameters:
        memUsageSetting - set how memory/temporary files are used for buffering streams etc.
        Throws:
        IOException - If scratch file directory was given but don't exist.
    • Method Detail

      • getMainMemoryOnlyInstance

        public static ScratchFile getMainMemoryOnlyInstance()
        Getter for an instance using only unrestricted main memory for buffering (same as new ScratchFile(MemoryUsageSetting.setupMainMemoryOnly())).
        Returns:
        instance configured to only use main memory with no size restriction
      • createBuffer

        public RandomAccess createBuffer()
                                  throws IOException
        Creates a new buffer using this page handler.
        Returns:
        A new buffer.
        Throws:
        IOException - If an error occurred.
      • createBuffer

        public RandomAccess createBuffer​(InputStream input)
                                  throws IOException
        Creates a new buffer using this page handler and initializes it with the data read from provided input stream (input stream is copied to buffer). The buffer data pointer is reset to point to first byte.
        Parameters:
        input - The input stream that is to be copied into the buffer.
        Returns:
        A new buffer containing data read from input stream.
        Throws:
        IOException - If an error occurred.
      • close

        public void close()
                   throws IOException
        Closes and deletes the temporary file. No further interaction with the scratch file or associated buffers can happen after this method is called. It also releases in-memory pages.
        Specified by:
        close in interface AutoCloseable
        Specified by:
        close in interface Closeable
        Throws:
        IOException - If there was a problem closing or deleting the temporary file.