Project structure / gradle build

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Project structure / gradle build

kiwiwings
Hi Devs,

since a few days / every now and then I try to get into our gradle build, but I have some problems, because I somehow try to start big (with the current gradle build) instead of building up from small example projects.

As a few of you prefer gradle to the current ant build [2], I would be willing to convert it, but I struggle with understanding the warnings IntelliJ throws me at and also the way multi-module build / DSL usually look like.

So I'd wished to have two things:

* maybe someone of you has time to go through the current gradle build and explain me a few things.

* modifying the project structure to something more modular, e.g. to have a separate sub-tree for the main jar and the others modules.

I'm not sure if this makes sense in the current project phase, but in case you would argue, that the patches wouldn't match anymore ... with the JPMS reorganization we already have that situation.

So if there's anyone brave enough [1] drop me a note.

Btw. if you would decide that a maven only build (opposed to the lite version now in place)  is also an improvement, I'm happy and capable of doing that ... but the project structure should be change then too.

Andi


[1] https://practical-tech.com/1983/01/01/real-men-dont-use-pascal/

[2] http://apache-poi.1045710.n5.nabble.com/Switch-to-maven-tp5722213.html


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Project structure / gradle build

Marius Volkhart
Hi Andi,

I'm quite familiar with Gradle and am happy to be involved. I think I have
a decent understanding of the POI Gradle build as it stands and can
certainly try answering any questions!

I have no opinion about Maven vs Gradle. I know my way around Gradle
better, and am familiar with JPMS modularizing a project with it.

--
Cheers,
Marius Volkhart



On Tue, Mar 23, 2021 at 9:30 PM Andreas Beeker <[hidden email]> wrote:

> Hi Devs,
>
> since a few days / every now and then I try to get into our gradle build,
> but I have some problems, because I somehow try to start big (with the
> current gradle build) instead of building up from small example projects.
>
> As a few of you prefer gradle to the current ant build [2], I would be
> willing to convert it, but I struggle with understanding the warnings
> IntelliJ throws me at and also the way multi-module build / DSL usually
> look like.
>
> So I'd wished to have two things:
>
> * maybe someone of you has time to go through the current gradle build and
> explain me a few things.
>
> * modifying the project structure to something more modular, e.g. to have
> a separate sub-tree for the main jar and the others modules.
>
> I'm not sure if this makes sense in the current project phase, but in case
> you would argue, that the patches wouldn't match anymore ... with the JPMS
> reorganization we already have that situation.
>
> So if there's anyone brave enough [1] drop me a note.
>
> Btw. if you would decide that a maven only build (opposed to the lite
> version now in place)  is also an improvement, I'm happy and capable of
> doing that ... but the project structure should be change then too.
>
> Andi
>
>
> [1] https://practical-tech.com/1983/01/01/real-men-dont-use-pascal/
>
> [2] http://apache-poi.1045710.n5.nabble.com/Switch-to-maven-tp5722213.html
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Project structure / gradle build

Dominik Stadler
Hi,

I am the one guilty of adding the original build.gradle which is basically
still an ugly hack via custom source folders and other stuff.

The intention was to over time get rid of both the build.xml and
Sonar-Maven build to reduce maintenance overhead.

Depdendency-handling is so much more simple in Gradle compared to our
home-grown things in the build.xml. Furthermore Ant is showing it's age,
which to me manifests e.g. in the slow and bumpy support for JUnit 5.

Also the support for Gradle in IDEA is usually very good, not sure about
the current state on Eclipse, though.

I am in favour of Gradle personally as I am not too familiar with Maven and
struggled a lot with it at times as well. I agree that Gradle is also
acting in strange ways sometimes, sometimes more than Ant, but it does many
many things out of the box if you stick to the default setup, which in my
opinion is a big plus.

I would also be interesting in contributing to a branch on github where we
modify the structure to a clean Gradle-based build including moving source
folders into place for a proper multi-project build

This would show how it will look like in the end and we can then experiment
with adding some of the more advanced things that we currently do in the
build.xml.

Dominik.

On Tue, Mar 23, 2021 at 10:00 PM Marius Volkhart <[hidden email]>
wrote:

> Hi Andi,
>
> I'm quite familiar with Gradle and am happy to be involved. I think I have
> a decent understanding of the POI Gradle build as it stands and can
> certainly try answering any questions!
>
> I have no opinion about Maven vs Gradle. I know my way around Gradle
> better, and am familiar with JPMS modularizing a project with it.
>
> --
> Cheers,
> Marius Volkhart
>
>
>
> On Tue, Mar 23, 2021 at 9:30 PM Andreas Beeker <[hidden email]>
> wrote:
>
> > Hi Devs,
> >
> > since a few days / every now and then I try to get into our gradle build,
> > but I have some problems, because I somehow try to start big (with the
> > current gradle build) instead of building up from small example projects.
> >
> > As a few of you prefer gradle to the current ant build [2], I would be
> > willing to convert it, but I struggle with understanding the warnings
> > IntelliJ throws me at and also the way multi-module build / DSL usually
> > look like.
> >
> > So I'd wished to have two things:
> >
> > * maybe someone of you has time to go through the current gradle build
> and
> > explain me a few things.
> >
> > * modifying the project structure to something more modular, e.g. to have
> > a separate sub-tree for the main jar and the others modules.
> >
> > I'm not sure if this makes sense in the current project phase, but in
> case
> > you would argue, that the patches wouldn't match anymore ... with the
> JPMS
> > reorganization we already have that situation.
> >
> > So if there's anyone brave enough [1] drop me a note.
> >
> > Btw. if you would decide that a maven only build (opposed to the lite
> > version now in place)  is also an improvement, I'm happy and capable of
> > doing that ... but the project structure should be change then too.
> >
> > Andi
> >
> >
> > [1] https://practical-tech.com/1983/01/01/real-men-dont-use-pascal/
> >
> > [2]
> http://apache-poi.1045710.n5.nabble.com/Switch-to-maven-tp5722213.html
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Project structure / gradle build

Dominik Stadler
I have now spent half an hour to move around files in a branch to show how
the directory structure and a full multi-project build would like.

It's at https://github.com/centic9/poi/tree/gradle_build

Each sub-module has it's own build.gradle, with the main build.gradle
providing shared definitions.

compiling, jars and running tests works now. Things like JavaDoc and Source
packages should be easy to add from here.

The Ant-build is likely broken now, but could still made runnable by simply
adjusting all the source locations

Based on this we could try to move more and more stuff from the Ant build
into Gradle, e.g. rat-check, forbidden-apis, building release artifacts, ...

Dominik.

On Wed, Mar 24, 2021 at 5:36 PM Dominik Stadler <[hidden email]>
wrote:

> Hi,
>
> I am the one guilty of adding the original build.gradle which is basically
> still an ugly hack via custom source folders and other stuff.
>
> The intention was to over time get rid of both the build.xml and
> Sonar-Maven build to reduce maintenance overhead.
>
> Depdendency-handling is so much more simple in Gradle compared to our
> home-grown things in the build.xml. Furthermore Ant is showing it's age,
> which to me manifests e.g. in the slow and bumpy support for JUnit 5.
>
> Also the support for Gradle in IDEA is usually very good, not sure about
> the current state on Eclipse, though.
>
> I am in favour of Gradle personally as I am not too familiar with Maven
> and struggled a lot with it at times as well. I agree that Gradle is also
> acting in strange ways sometimes, sometimes more than Ant, but it does many
> many things out of the box if you stick to the default setup, which in my
> opinion is a big plus.
>
> I would also be interesting in contributing to a branch on github where we
> modify the structure to a clean Gradle-based build including moving source
> folders into place for a proper multi-project build
>
> This would show how it will look like in the end and we can then
> experiment with adding some of the more advanced things that we currently
> do in the build.xml.
>
> Dominik.
>
> On Tue, Mar 23, 2021 at 10:00 PM Marius Volkhart <[hidden email]>
> wrote:
>
>> Hi Andi,
>>
>> I'm quite familiar with Gradle and am happy to be involved. I think I have
>> a decent understanding of the POI Gradle build as it stands and can
>> certainly try answering any questions!
>>
>> I have no opinion about Maven vs Gradle. I know my way around Gradle
>> better, and am familiar with JPMS modularizing a project with it.
>>
>> --
>> Cheers,
>> Marius Volkhart
>>
>>
>>
>> On Tue, Mar 23, 2021 at 9:30 PM Andreas Beeker <[hidden email]>
>> wrote:
>>
>> > Hi Devs,
>> >
>> > since a few days / every now and then I try to get into our gradle
>> build,
>> > but I have some problems, because I somehow try to start big (with the
>> > current gradle build) instead of building up from small example
>> projects.
>> >
>> > As a few of you prefer gradle to the current ant build [2], I would be
>> > willing to convert it, but I struggle with understanding the warnings
>> > IntelliJ throws me at and also the way multi-module build / DSL usually
>> > look like.
>> >
>> > So I'd wished to have two things:
>> >
>> > * maybe someone of you has time to go through the current gradle build
>> and
>> > explain me a few things.
>> >
>> > * modifying the project structure to something more modular, e.g. to
>> have
>> > a separate sub-tree for the main jar and the others modules.
>> >
>> > I'm not sure if this makes sense in the current project phase, but in
>> case
>> > you would argue, that the patches wouldn't match anymore ... with the
>> JPMS
>> > reorganization we already have that situation.
>> >
>> > So if there's anyone brave enough [1] drop me a note.
>> >
>> > Btw. if you would decide that a maven only build (opposed to the lite
>> > version now in place)  is also an improvement, I'm happy and capable of
>> > doing that ... but the project structure should be change then too.
>> >
>> > Andi
>> >
>> >
>> > [1] https://practical-tech.com/1983/01/01/real-men-dont-use-pascal/
>> >
>> > [2]
>> http://apache-poi.1045710.n5.nabble.com/Switch-to-maven-tp5722213.html
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: [hidden email]
>> > For additional commands, e-mail: [hidden email]
>> >
>> >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Project structure / gradle build

kiwiwings
I will now successively adapt the svn source to your proposel - including the fixes to the ant build.
Even if the learning effect for me (of getting into gradle) would be less for the beginning - we would have at least a project structure which fits better to the gradle builds.

Andi

On 24.03.21 21:11, Dominik Stadler wrote:

> I have now spent half an hour to move around files in a branch to show how
> the directory structure and a full multi-project build would like.
>
> It's at https://github.com/centic9/poi/tree/gradle_build
>
> Each sub-module has it's own build.gradle, with the main build.gradle
> providing shared definitions.
>
> compiling, jars and running tests works now. Things like JavaDoc and Source
> packages should be easy to add from here.
>
> The Ant-build is likely broken now, but could still made runnable by simply
> adjusting all the source locations
>
> Based on this we could try to move more and more stuff from the Ant build
> into Gradle, e.g. rat-check, forbidden-apis, building release artifacts, ...
>
> Dominik.
>
> On Wed, Mar 24, 2021 at 5:36 PM Dominik Stadler <[hidden email]>
> wrote:
>
>> Hi,
>>
>> I am the one guilty of adding the original build.gradle which is basically
>> still an ugly hack via custom source folders and other stuff.
>>
>> The intention was to over time get rid of both the build.xml and
>> Sonar-Maven build to reduce maintenance overhead.
>>
>> Depdendency-handling is so much more simple in Gradle compared to our
>> home-grown things in the build.xml. Furthermore Ant is showing it's age,
>> which to me manifests e.g. in the slow and bumpy support for JUnit 5.
>>
>> Also the support for Gradle in IDEA is usually very good, not sure about
>> the current state on Eclipse, though.
>>
>> I am in favour of Gradle personally as I am not too familiar with Maven
>> and struggled a lot with it at times as well. I agree that Gradle is also
>> acting in strange ways sometimes, sometimes more than Ant, but it does many
>> many things out of the box if you stick to the default setup, which in my
>> opinion is a big plus.
>>
>> I would also be interesting in contributing to a branch on github where we
>> modify the structure to a clean Gradle-based build including moving source
>> folders into place for a proper multi-project build
>>
>> This would show how it will look like in the end and we can then
>> experiment with adding some of the more advanced things that we currently
>> do in the build.xml.
>>
>> Dominik.
>>
>> On Tue, Mar 23, 2021 at 10:00 PM Marius Volkhart <[hidden email]>
>> wrote:
>>
>>> Hi Andi,
>>>
>>> I'm quite familiar with Gradle and am happy to be involved. I think I have
>>> a decent understanding of the POI Gradle build as it stands and can
>>> certainly try answering any questions!
>>>
>>> I have no opinion about Maven vs Gradle. I know my way around Gradle
>>> better, and am familiar with JPMS modularizing a project with it.
>>>
>>> --
>>> Cheers,
>>> Marius Volkhart
>>>
>>>
>>>
>>> On Tue, Mar 23, 2021 at 9:30 PM Andreas Beeker <[hidden email]>
>>> wrote:
>>>
>>>> Hi Devs,
>>>>
>>>> since a few days / every now and then I try to get into our gradle
>>> build,
>>>> but I have some problems, because I somehow try to start big (with the
>>>> current gradle build) instead of building up from small example
>>> projects.
>>>> As a few of you prefer gradle to the current ant build [2], I would be
>>>> willing to convert it, but I struggle with understanding the warnings
>>>> IntelliJ throws me at and also the way multi-module build / DSL usually
>>>> look like.
>>>>
>>>> So I'd wished to have two things:
>>>>
>>>> * maybe someone of you has time to go through the current gradle build
>>> and
>>>> explain me a few things.
>>>>
>>>> * modifying the project structure to something more modular, e.g. to
>>> have
>>>> a separate sub-tree for the main jar and the others modules.
>>>>
>>>> I'm not sure if this makes sense in the current project phase, but in
>>> case
>>>> you would argue, that the patches wouldn't match anymore ... with the
>>> JPMS
>>>> reorganization we already have that situation.
>>>>
>>>> So if there's anyone brave enough [1] drop me a note.
>>>>
>>>> Btw. if you would decide that a maven only build (opposed to the lite
>>>> version now in place)  is also an improvement, I'm happy and capable of
>>>> doing that ... but the project structure should be change then too.
>>>>
>>>> Andi
>>>>
>>>>
>>>> [1] https://practical-tech.com/1983/01/01/real-men-dont-use-pascal/
>>>>
>>>> [2]
>>> http://apache-poi.1045710.n5.nabble.com/Switch-to-maven-tp5722213.html
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Project structure / gradle build

kiwiwings
In reply to this post by Dominik Stadler
Hi Devs,

I hope to commit my gradle setup by Wednesday evening.

I haven't yet migrated all the features, but the important ones:
* JPMS / mrJar co-existing to Java 8 build
* Fixed the batik-script issue in the tests - not generally
* ooxml-lite is now created after its dependencies have been executed, so all used schema files should be in

Things I don't know how to fix:
* TestSignatureInfo often causing a SIGSEV when calling the XML dom api in Java 14 (14.0.2+12-46 on Ubuntu, but works in Oracle 8)
* I've tried the fix described in https://github.com/java9-modularity/gradle-modules-plugin/issues/97 but to no avail, therefore I've opened (--add-exports) the junit modules.
* IntelliJ integration - it mostly works, but the syntax checker gives the impression, that not all working constructs are understood

Open todos:
* sonarqube calls
* refactor the gradle code duplications/mess into the root gradle - as I was doing constant code changing iterations, I haven't dared to move it up to the root gradle, just to find out, that the approach doesn't work with the next to-be migrated module
* release logic
* site generation
* fix the paths in the ant build (before tomorrows commit)
* You'll see some weird duplicated dependencies, because I thought the build order is mainly influenced by it AND I need the *-tests.jars to execute the tests in the JPMS context. Although I've added another set of dependsOn to the corresponding compileJava9 tasks, I don't understand how gradles concept of using the expanded classes directories should work with mrJars. (see https://blog.gradle.org/mrjars)
* fix the javadocs, so the HTML5 linting can be enabled
* check out JPMS plugins like https://github.com/java9-modularity/gradle-modules-plugin or Axels plugin (https://github.com/xzel23/JpmsGradlePlugin)

As I have shuffled again the directories, please commit your code so I can merge it - as I used "svn mv" this should be also ok, if you don't commit now, but it's easier for you, if you do it before ...

Although I've spent many many hours with gradle in the last 2 weeks and sometimes were baffled about its reactions, I have to admit, that certain features are slick, e.g. have a look at the ooxml-lite generation when it's committed.

Andi


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Project structure / gradle build

Dominik Stadler
Thanks Andi for all the work, I am interested in how it will look like
afterwards :)

Maybe you can push some of the smaller/isolated changes first in separate
commits so we get CI on each of those separately?

On the JDK crash, I saw that elsewhere also with JDK 11! If we can make it
reproducible, I can report a JDK bug via our contacts at Oracle.

Thanks... Dominik.

On Wed, Apr 7, 2021 at 1:53 AM Andreas Beeker <[hidden email]> wrote:

> Hi Devs,
>
> I hope to commit my gradle setup by Wednesday evening.
>
> I haven't yet migrated all the features, but the important ones:
> * JPMS / mrJar co-existing to Java 8 build
> * Fixed the batik-script issue in the tests - not generally
> * ooxml-lite is now created after its dependencies have been executed, so
> all used schema files should be in
>
> Things I don't know how to fix:
> * TestSignatureInfo often causing a SIGSEV when calling the XML dom api in
> Java 14 (14.0.2+12-46 on Ubuntu, but works in Oracle 8)
> * I've tried the fix described in
> https://github.com/java9-modularity/gradle-modules-plugin/issues/97 but
> to no avail, therefore I've opened (--add-exports) the junit modules.
> * IntelliJ integration - it mostly works, but the syntax checker gives the
> impression, that not all working constructs are understood
>
> Open todos:
> * sonarqube calls
> * refactor the gradle code duplications/mess into the root gradle - as I
> was doing constant code changing iterations, I haven't dared to move it up
> to the root gradle, just to find out, that the approach doesn't work with
> the next to-be migrated module
> * release logic
> * site generation
> * fix the paths in the ant build (before tomorrows commit)
> * You'll see some weird duplicated dependencies, because I thought the
> build order is mainly influenced by it AND I need the *-tests.jars to
> execute the tests in the JPMS context. Although I've added another set of
> dependsOn to the corresponding compileJava9 tasks, I don't understand how
> gradles concept of using the expanded classes directories should work with
> mrJars. (see https://blog.gradle.org/mrjars)
> * fix the javadocs, so the HTML5 linting can be enabled
> * check out JPMS plugins like
> https://github.com/java9-modularity/gradle-modules-plugin or Axels plugin
> (https://github.com/xzel23/JpmsGradlePlugin)
>
> As I have shuffled again the directories, please commit your code so I can
> merge it - as I used "svn mv" this should be also ok, if you don't commit
> now, but it's easier for you, if you do it before ...
>
> Although I've spent many many hours with gradle in the last 2 weeks and
> sometimes were baffled about its reactions, I have to admit, that certain
> features are slick, e.g. have a look at the ooxml-lite generation when it's
> committed.
>
> Andi
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Project structure / gradle build

kiwiwings
OK ... I've committed my part and had to resolve 2 tree conflicts.

the JDK crash is happening every second/third time I test poi-ooxml - so maybe it's a JIT error.
I have a hs_err_pid*.log if that helps.
For a real heap dump, I would need to check on how to use apport.

Andi.

On 07.04.21 17:01, Dominik Stadler wrote:

> Thanks Andi for all the work, I am interested in how it will look like
> afterwards :)
>
> Maybe you can push some of the smaller/isolated changes first in separate
> commits so we get CI on each of those separately?
>
> On the JDK crash, I saw that elsewhere also with JDK 11! If we can make it
> reproducible, I can report a JDK bug via our contacts at Oracle.
>
> Thanks... Dominik.
>
> On Wed, Apr 7, 2021 at 1:53 AM Andreas Beeker <[hidden email]> wrote:
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]