HSMF enhancements

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

HSMF enhancements

Hölzl, Dominik
Hello!

Can somebody please have a look https://github.com/apache/poi/pull/167?

Thank you and regards,
Dominik Hölzl

Reply | Threaded
Open this post in threaded view
|

Re: HSMF enhancements

kiwiwings
Hello Dominik,

I'm having a look at the patch.

Where is the CRC32 code from? ... and please point me to the documentation of the GUID logic.

Thank you and best wishes,
Andi

On 24.02.20 13:04, Hölzl, Dominik wrote:
> Hello!
>
> Can somebody please have a look https://github.com/apache/poi/pull/167?
>
> Thank you and regards,
> Dominik Hölzl
>
>



signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

AW: HSMF enhancements

Hölzl, Dominik
> -----Ursprüngliche Nachricht-----

> Von: Andreas Beeker <[hidden email]>

> Gesendet: Montag, 2. März 2020 00:24

> An: POI Developers List <[hidden email]>

> Cc: Hölzl, Dominik <[hidden email]>

> Betreff: Re: HSMF enhancements

>

> Hello Dominik,

>

> I'm having a look at the patch.

>

> Where is the CRC32 code from? ... and please point me to the

> documentation of the GUID logic.

>

> Thank you and best wishes,

> Andi

>

> On 24.02.20 13:04, Hölzl, Dominik wrote:

> > Hello!

> >

> > Can somebody please have a look

> https://github.com/apache/poi/pull/167?

> >

> > Thank you and regards,

> > Dominik Hölzl

> >

> >

>





Hello!



The GUID-logic is exactly documented here:



https://docs.microsoft.com/en-us/openspecs/exchange_server_protocols/ms-oxmsg/b046868c-9fbf-41ae-9ffb-8de2bd4eec82



(Chapter 2.2.3 Named Property Mapping Storage)



As there is no explicit documentation on the CRC-32 parameters there (2.2.3.2.2.1 Stream ID Equation - the only hint is "X25" which seems to be a dead end), I grabbed this from here:



https://github.com/yorkshiretwist/ReadAnOutlookMsg/blob/master/OutlookStorage.cs

(Seems to be originated from https://github.com/jukka/jtnef/blob/master/src/net/freeutils/tnef/CompressedRTFInputStream.java)



The only thing I needed from there were the CRC-32 parameters, which are poly=0x04C11DB7 (which is the "default" CRC-32 value), init=0x00000000, xor=0x00000000, revin=true, revout=true.

This leads to the check value of 0x2DFD2D88 which can e.g. be tested here: http://www.zorc.breitbandkatze.de/crc.html



Public CRC32 algorithm implementation is broadly available.

https://en.wikipedia.org/wiki/Cyclic_redundancy_check



Regards,

Dominik


Reply | Threaded
Open this post in threaded view
|

Re: AW: HSMF enhancements

kiwiwings
Hello Dominik,

thank you very much for your explanations!

I'll replace the crc code by the following - we have commons codec anyway as a dependency, j.u.z.CRC32 can be used likewise:

static private long calculateCRC32(byte[] buf, int off, int len) {
    PureJavaCrc32 crc = new PureJavaCrc32();
    // set initial crc value to 0
    crc.update( new byte[] {-1,-1,-1,-1}, 0, 4);
    crc.update(buf, off, len);
    return ~crc.getValue() & 0xFFFFFFFFL;
}

I haven't finished with the rest yet, but the next release will anyway take a while ...

Andi.


signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

AW: AW: HSMF enhancements

Hölzl, Dominik
Hello!

Thank you!
I didn't know that, your CRC32-code works perfectly. All Unit-Tests are OK with it.

Regards,
Dominik

Von: Andreas Beeker <[hidden email]>
Gesendet: Dienstag, 3. März 2020 23:32
An: [hidden email]
Betreff: Re: AW: HSMF enhancements

Hello Dominik,

thank you very much for your explanations!

I'll replace the crc code by the following - we have commons codec anyway as a dependency, j.u.z.CRC32 can be used likewise:


static private long calculateCRC32(byte[] buf, int off, int len) {

    PureJavaCrc32 crc = new PureJavaCrc32();

    // set initial crc value to 0

    crc.update( new byte[] {-1,-1,-1,-1}, 0, 4);

    crc.update(buf, off, len);

    return ~crc.getValue() & 0xFFFFFFFFL;

}

I haven't finished with the rest yet, but the next release will anyway take a while ...

Andi.

Reply | Threaded
Open this post in threaded view
|

Re: HSMF enhancements

kiwiwings
In reply to this post by Hölzl, Dominik
Hello Dominik,

thank you for your patch!

I've applied a modified version of it via #1874990 [1].

The reasons for my modifications are:
- remove the custom CRC32 code:
   I don't want to mix technical code with module specific code ... thankfully it could be calculated with off the shelf CRC32 classes anyway.
- use enums instead of String constants (... where we didn't have already a lot of legacy constants)
- try to break the cyclomatic complexity:
   wherever I see lots of nested ifs, I introduce private methods to limit the scope of variables
- In the ByteChunkDeferred class, the linked node can be read multiple times and we don't face exhausted InputStreams.

Best wishes,
Andi


[1] https://svn.apache.org/viewvc?view=revision&revision=1874990

On 24.02.20 13:04, Hölzl, Dominik wrote:
> Hello!
>
> Can somebody please have a look https://github.com/apache/poi/pull/167?
>
> Thank you and regards,
> Dominik Hölzl
>
>



signature.asc (499 bytes) Download Attachment