[Bug 65042] New: Writing pircture to workbook cause memory leak due to picture data is to much in memory.

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

            Bug ID: 65042
           Summary: Writing pircture to workbook cause memory leak due to
                    picture data is to much in memory.
           Product: POI
           Version: 4.1.2-FINAL
          Hardware: Macintosh
                OS: Mac OS X 10.4
            Status: NEW
          Severity: blocker
          Priority: P2
         Component: SXSSF
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ---

Created attachment 37670
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=37670&action=edit
Memory Usage Dump

Sxssf using Xssf handle its picture. So picture will save in memory, not flush
with row. This means if we have a lot of pictures had to write in xlsx, this
will cause memory leak which was I am struggling with. I can't find a way to
flush picture data before workbook.write().

The test case is, 300kb picture, one row write 10 pictures. JVM -Xmx128m.
There will be an oom when write 50 rows. In that time, the picture data list
will have 60mb in memory.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

--- Comment #1 from Runzhi <[hidden email]> ---
Created attachment 37671
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=37671&action=edit
Memory I count in program

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

Dominik Stadler <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|blocker                     |major

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

Dominik Stadler <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Writing pircture to         |Adding pictures to workbook
                   |workbook cause memory leak  |causes memory leak
                   |due to picture data is to   |
                   |much in memory.             |

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

Dominik Stadler <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--- Comment #2 from Dominik Stadler <[hidden email]> ---
Can you provide a small piece of code which reproduces the problem for you?
Ideally as a self-contained unit-test so we can reproduce the problem and take
a closer look?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

--- Comment #3 from Runzhi <[hidden email]> ---
Sorry for the late response.
I have created a repository that can reproduce the problem.
Run unit test with max heap size 128m.

Repository URL is :
https://github.com/foresx/poi-memory-leak-demo

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

Dominik Stadler <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |NEW
           Severity|major                       |enhancement

--- Comment #4 from Dominik Stadler <[hidden email]> ---
Thanks for the detailed reproducing code, I took a look at your sample-project
now.

Pictures in .xlsx files are not stored per "row" or "sheet", but rather
globally in a separate structure along the other parts.

The current SXSSFWorkbook only flushes and removes rows based on the
"rowAccesswindowSize".

So flushing picture data for SXSSFWorkbook is currently not supported, we can
consider adding it as an enhancement, naturally it will happen sooner if you
can propose an implementation that offers this as additional option for
SXSSFWorkbook in some way, however it will require some coding as you likely
need to flush out pictures in a similar way as the rows and then a write-time
combine the information into the final document as well.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

https://bz.apache.org/bugzilla/show_bug.cgi?id=65042

--- Comment #5 from Runzhi <[hidden email]> ---
Thanks a lot. When I have free time, I think I will have a try.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]