Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly handle bitshifting if system does not support unaligned a… #4366

Closed
wants to merge 1 commit into from

Conversation

normanmaurer
Copy link
Member

Motivation:

We had a bug in our implemention which double "reversed" bytes on systems which not support unaligned access.

Modifications:

- Correctly only reverse bytes if needed.
- Share code between unsafe implementations.

Result:

No more data-corruption on sytems without unaligned access.

@normanmaurer normanmaurer self-assigned this Oct 16, 2015
@normanmaurer normanmaurer added this to the 4.0.33.Final milestone Oct 16, 2015
@normanmaurer
Copy link
Member Author

@nmittler @Scottmitch PTAL

*/

package io.netty.buffer;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: kill line

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@Scottmitch
Copy link
Member

lgtm


static short getShort(long address) {
if (UNALIGNED) {
short v = PlatformDependent.getShort(address);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just update the PlatformDependent methods to do this for us?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nmittler nope we need to do this outside of it as we need the "raw" memory access for UnsafeDirectSwappedByteBuf (which had a bug because of this as well).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@normanmaurer ok SGTM. Thanks!

@nmittler
Copy link
Member

@normanmaurer LGTM ... just one comment.

@normanmaurer normanmaurer changed the title Correctly handle byte shifting if system does not support unaligned a… Correctly handle bitshifting if system does not support unaligned a… Oct 16, 2015
@normanmaurer
Copy link
Member Author

@nmittler @Scottmitch actually there is a bug in the impl.... Let me fix it first 👍 .

@normanmaurer
Copy link
Member Author

@Scottmitch @nmittler please check again. We need to always operate on BigEndian order here. Now it's correct.

@nmittler
Copy link
Member

@normanmaurer looks like you rebased ... what exactly did you change?

@Scottmitch
Copy link
Member

@nmittler +1 - I was about to ask the same thing. What does We need to always operate on BigEndian order here. mean?

@normanmaurer
Copy link
Member Author

All the methods here need to set values in big endian order and return values in big endian order as our buffers use big endian by default (just like ByteBuffer). The converting to little endian only happens in the SwappedByteBuf implementations.

Am 17.10.2015 um 00:35 schrieb Scott Mitchell notifications@github.com:

@nmittler +1 - I was about to ask the same thing. What does We need to always operate on BigEndian order here. mean?


Reply to this email directly or view it on GitHub.

}

static int getUnsignedMedium(long address) {
return (PlatformDependent.getByte(address) & 0xff) << 16 |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if UNALIGNED we could reduce the number of PlatformDependent calls (and shifting / masking / ORing) by using PlatformDependent.getShort(..).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we don't need to check here, since getShort already does the right thing if UNALIGNED.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nmittler - Are you talking about re-using UnsafeByteBufUtil.getShort(..)? I was thinking of staying consistent with the rest of the methods in this class and directly using PlatformDependent.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I meant calling UnsafeByteBufUtil.getShort ... either way works.

@Scottmitch
Copy link
Member

@normanmaurer - LGTM. Be good once we get some tests up an running on the different Endian and non-unaligned machines.

@nmittler
Copy link
Member

@normanmaurer lgtm

…ccess.

Motivation:

We had a bug in our implemention which double "reversed" bytes on systems which not support unaligned access.

Modifications:

- Correctly only reverse bytes if needed.
- Share code between unsafe implementations.

Result:

No more data-corruption on sytems without unaligned access.
}
}

static void setMedium(long address, int value) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nmittler please check

@normanmaurer
Copy link
Member Author

@nmittler please check updated PR... Please take special care when review getUnsignedMedium and setMedium. Especially the "NATIVE_ORDER" branch.

@normanmaurer
Copy link
Member Author

Cherry-picked into 4.0 (eadf1bf) , 4.1 (f30a51b) and master (2377dd9)

@normanmaurer normanmaurer deleted the unalligned_bug branch October 20, 2015 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants