[Bug 61478] New: POI OOXML-Schema lookup uses wrong classloader

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #18 from Andreas Beeker <[hidden email]> ---
Just a short feedback ... as I'm not sure, if I can finish it this weekend.
When I tested #60226 / #57857, I used a sample project [1]
Additional to Glucazeaus project, one needs the additional service mix patch in
#57857

If the Class.forName approach works in this and manifoldCF, it should be
hopefully sufficient to render the workaround obsolete.


[1] https://github.com/glucazeau/test-poi-sling

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #19 from Greg Woolsey <[hidden email]> ---
From the attached stacktrace, and the sources, it can be seen that the error
originates from

org.openxmlformats.schemas.wordprocessingml.x2006.main.DocumentDocument.Factory.parse(*)

which is found in

ooxml-schemas-1.3.jar

This is a generated XMLBeans class.  Further, it isn't doing any explicit class
loading, it is just importing a class.  So it looks to me like the issue is
down in XMLBeans somewhere, or in the user's ClassLoader hierarchy.  

I searched the entire POI codebase, and there are no calls to *.forName(String,
ClassLoader) or anything similar.  All calls are using the static
Class.forName(String classname) method, which should be fine.

I'm fairly certain the issue for the end user is the placement of
ooxml-schemas-1.3.jar in their class loader library structure.  It could be
defined at too high a level, and thus inaccessible to a lower classloader, in
which case that library should be pushed down rather than POI pulled up.  Note
that this is a standard dependency for POI.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #20 from Greg Woolsey <[hidden email]> ---
Argh, I should go to bed.  I see now the issue was calling from
DocumentDocument.Factory in ooxml-schemas-1.3.jar to POIXMLTypeLoader in
poi-ooxml.jar.

So it still could be the placement of ooxml-schemas-1.3.jar, but in the reverse
case - it could be that the client code can see that jar, but that jar can't
see POI.  It could be that ooxml-schemas.1.3.jar is loaded too early, and thus
can't see POI.  Perhaps some library at a higher-order classloader also loads
that jar somehow.

(In reply to Greg Woolsey from comment #19)

> From the attached stacktrace, and the sources, it can be seen that the error
> originates from
>
> org.openxmlformats.schemas.wordprocessingml.x2006.main.DocumentDocument.
> Factory.parse(*)
>
> which is found in
>
> ooxml-schemas-1.3.jar
>
> This is a generated XMLBeans class.  Further, it isn't doing any explicit
> class loading, it is just importing a class.  So it looks to me like the
> issue is down in XMLBeans somewhere, or in the user's ClassLoader hierarchy.
>
>
> I searched the entire POI codebase, and there are no calls to
> *.forName(String, ClassLoader) or anything similar.  All calls are using the
> static Class.forName(String classname) method, which should be fine.
>
> I'm fairly certain the issue for the end user is the placement of
> ooxml-schemas-1.3.jar in their class loader library structure.  It could be
> defined at too high a level, and thus inaccessible to a lower classloader,
> in which case that library should be pushed down rather than POI pulled up.
> Note that this is a standard dependency for POI.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #21 from Karl Wright <[hidden email]> ---
Hi Greg,

As I've explained before, there are exactly two classloader levels in
ManifoldCF - the root level, and the "connectors" level.  In previous (working)
Tika integrations, tika-core was at the root level, and tika-parsers (and all
its dependencies, including xmlbeans) was at the "connectors" level.

When we went to the most recent version of Tika we discovered that having
poi-ooxml-schemas at the higher level meant that the Tika connector could not
find anything in poi-ooxml-schemas, even though that also was at the higher
level, as was xmlbeans.  We needed to move ALL poi jars and their dependencies
to the lower level in order to get it to work again.  (I'm in the process of
voting on a point release for this as we speak so that our users will not be
impacted.)

We'd much prefer having these jars once again live at the higher level, since
putting them at the lower level means they must be included multiple times (in
multiple webapps etc.), and that bloats our binary considerably.

Hope this helps.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #22 from Karl Wright <[hidden email]> ---
If there are no invocations of Class.forName(String, boolean, ClassLoader),
then there's no way the POI libraries can be a problem.  But a glance at the
code shows that that's not entirely true; the POIXMLTypeLoader class does the
latter.

It may be the case that the only reason for that code was written was to work
around this problem when it was discovered by others (e.g. #60226).  But, in
that case, the problem might well be that some other dependency, e.g. xmlbeans,
is doing the wrong thing and you guys need to request a fix from them.

Here are some data points:

- Running all poi jars and their dependencies at the "connectors" level with
poi-3.15 *definitely* uses the wrong classloader at some point -- probably the
thread classloader
- The testbed I constructed and uploaded, on the other hand, *succeeds* - and
that setup mimics MCF's classloader setup, but without Tika in between the MCF
"connector" and the poi jars

Maybe the right approach is to analyze the difference in code paths between
these two ways of calling into poi and see what differences there are, if any?
The stack traces are helpful here, yes, but maybe also looking at the testbed I
uploaded could provide some insight into a way of getting into poi that does
not seem to have the problem?

The only thing I'm pretty sure of is that it probably isn't Tika that does
this, because it *does* manage to find the poi classes, just not those in
poi-ooxml-schemas.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #23 from Andreas Beeker <[hidden email]> ---
Created attachment 35283
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=35283&action=edit
Get Classloader from SchemaType instead of ContextClassLoader/preset CL

I've tested that patch against the OSGi example mentioned above (the latest
servicemix bundle for POI 3.16 is also incomplete as far as I can say).
I wasn't sure (and I'm still not), if the cached SchemaTypeLoader would result
in a failing resource loading. At least I've added some kind of OOM-check to
see if the ThreadLocals would result in memory leaks.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #24 from Andreas Beeker <[hidden email]> ---
I should add ... if the schema type loader is not cached, this will result in
OOM in the tests.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

Andreas Beeker <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--- Comment #25 from Andreas Beeker <[hidden email]> ---
Karl (or should I say Mr. Wright?), I'll wait for your response, before I
commit the patch and release(-candidate) POI 3.17 final - the last version to
support Java 6.

Please apply the patch to the POI trunk, build it and give us feedback if it
works for you.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #26 from Karl Wright <[hidden email]> ---
Will do, as time permits.  Hopefully you should have an answer by tomorrow
evening at the latest.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

--- Comment #27 from Karl Wright <[hidden email]> ---
I was able to confirm that this fix corrected the problem in the MCF
environment with Tika extraction.

Thanks to all for your help on this!

When do you expect this code to "go live" and be available for download from
Maven?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

Karl Wright <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEEDINFO                    |RESOLVED

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

Nick Burch <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---

--- Comment #28 from Nick Burch <[hidden email]> ---
Re-opening the bug, as I think Andreas still needs to commit the proposed fix
to svn!

Release wise - we're holding off doing the 3.17 release candidate for this fix,
as we want to play nicely with other ASF projects :)

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

Andreas Beeker <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #29 from Andreas Beeker <[hidden email]> ---
Applied via r1807418

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 61478] POI OOXML-Schema lookup uses wrong classloader

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=61478

Andreas Beeker <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Hardware|PC                          |All
            Version|unspecified                 |3.17-dev
          Component|POI Overall                 |OPC

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12