[Bug 62081] New: java.lang.OutOfMemoryError: Map failed occurs when processing a bunch of msg files

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 62081] New: java.lang.OutOfMemoryError: Map failed occurs when processing a bunch of msg files

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=62081

            Bug ID: 62081
           Summary: java.lang.OutOfMemoryError: Map failed occurs when
                    processing a bunch of msg files
           Product: POI
           Version: 3.16-FINAL
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: critical
          Priority: P2
         Component: POI Overall
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ---

I am trying to read and extract contents of msg file using POI 3.16.  While
trying to process a bunch of msg files (not large, in the <500kb) and the
following error occurs.  
==========================================================================
org.apache.poi.EmptyFileException: The supplied file was empty (zero bytes
long)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:218)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.<init>(NPOIFSFileSystem.java:168)
        at
com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.java:236)
        at
com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:1352)
        at
com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520)
        at
com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405)
        at
com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314)
        at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
java.lang.RuntimeException: java.io.IOException: Map failed
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:173)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:142)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.getBlockAt(NPOIFSMiniStore.java:69)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.createBlockIfNeeded(NPOIFSMiniStore.java:97)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBlockIfNeeded(NPOIFSStream.java:226)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NPOIFSStream.java:246)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:143)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.<init>(NPOIFSDocument.java:84)
        at
org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.java:422)
        at
com.testcode.MSGWriter.MessageWriterHelper.writeNamedPropertyDocuments(MessageWriterHelper.java:938)
        at
com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.java:242)
        at
com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:1352)
        at
com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520)
        at
com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405)
        at
com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314)
        at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940)
        at
org.apache.poi.poifs.nio.FileBackedDataSource.read(FileBackedDataSource.java:99)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getBlockAt(NPOIFSFileSystem.java:484)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:169)
        ... 18 more
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937)
        ... 21 more
java.lang.RuntimeException: java.io.IOException: Map failed
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:173)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:142)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.getBlockAt(NPOIFSMiniStore.java:69)
        at
org.apache.poi.poifs.filesystem.NPOIFSMiniStore.createBlockIfNeeded(NPOIFSMiniStore.java:97)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.createBlockIfNeeded(NPOIFSStream.java:226)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBuffer.write(NPOIFSStream.java:246)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.store(NPOIFSDocument.java:143)
        at
org.apache.poi.poifs.filesystem.NPOIFSDocument.<init>(NPOIFSDocument.java:84)
        at
org.apache.poi.poifs.filesystem.DirectoryNode.createDocument(DirectoryNode.java:422)
        at
com.testcode.MSGWriter.MessageWriterHelper.writeNamedPropertyDocuments(MessageWriterHelper.java:938)
        at
com.testcode.MSGWriter.MapiMessage.addNamedPropertiesToMessage(MapiMessage.java:242)
        at
com.testcode.MSGReader.MessageReader.parseAttachment(MessageReader.java:1352)
        at
com.testcode.MSGReader.MessageReader.parseDirectory(MessageReader.java:520)
        at
com.testcode.MSGReader.MessageReader.parseMessage(MessageReader.java:405)
        at
com.testcode.MSGReader.MessageReader.readMessageFile(MessageReader.java:314)
        at txtGeneration.txtGeneration.Task.run(MTMsgTotxtConvertor.java:83)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940)
        at
org.apache.poi.poifs.nio.FileBackedDataSource.read(FileBackedDataSource.java:99)
        at
org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getBlockAt(NPOIFSFileSystem.java:484)
        at
org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:169)
        ... 18 more
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937)
        ... 21 more
Java HotSpot(TM) 64-Bit Server VM warning: INFO:
os::commit_memory(0x00007fd7c8a1a000, 262144, 0) failed; error='Cannot allocate
memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 262144 bytes for committing
reserved memory.
# An error report file with more information is saved as:
# /home/testuser/CVLibTest/hs_err_pid6684.log
[thread 140564401960704 also had an error]
Java HotSpot(TM) 64-Bit Server VM warning: INFO:
os::commit_memory(0x00007fd7c93f0000, 65536, 1) failed; error='Cannot allocate
memory' (errno=12)
[thread 140564395820800 also had an error]
==========================================================================
I see the following bug has been addressed in 3.15 already, but still seeing
the issue.  for now sysctl -w vm.max_map_count to a larger value seems to be
working, but can this be fixed?

Bug 60140 - OOM caused by Memory Leak in FileBackedDataSource

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 62081] java.lang.OutOfMemoryError: Map failed occurs when processing a bunch of msg files

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=62081

Dominik Stadler <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--- Comment #1 from Dominik Stadler <[hidden email]> ---
Can you share some sample files which reproduce this?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 62081] java.lang.OutOfMemoryError: Map failed occurs when processing a bunch of msg files

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=62081

Dominik Stadler <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |RESOLVED
         Resolution|---                         |LATER

--- Comment #2 from Dominik Stadler <[hidden email]> ---
We cannot reproduce the problem with the information that was provided. No
sample file was provided, so we cannot do much here unless we get more
information. Thus closing this as LATER for now, please reopen with more
information if this is still a problem for you.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]