Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Unix Domain Sockets when using native epoll transport #3344

Closed
wants to merge 1 commit into from

Conversation

normanmaurer
Copy link
Member

Motivation:

Using Unix Domain Sockets can be very useful when communication should take place on the same host and has less overhead then using loopback. We should support this with the native epoll transport.

Modifications:

  • Add support for Unix Domain Sockets.
  • Adjust testsuite to be able to reuse tests.

Result:

Unix Domain Sockets are now support when using native epoll transport.

@normanmaurer
Copy link
Member Author

@trustin please review

@normanmaurer normanmaurer added this to the 4.0.26.Final milestone Jan 21, 2015
if((ctrl_msg->cmsg_level == SOL_SOCKET) && (ctrl_msg->cmsg_type == SCM_RIGHTS)) {
socketFd = *((int *) CMSG_DATA(ctrl_msg));
// set as non blocking as we want to use it with epoll
if (fcntl(socketFd, F_SETFL, O_NONBLOCK) == -1) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a consensus on ioctl vs fcntl (i.e. always use fcntl for portability/standardization reasons)? Word on the street is ioctl can sometimes be less expensive for this case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope... can you tell me more about these "word on the street" ? Maybe links etc...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@normanmaurer - Sure. For example see nginx comment at the top.

The "word on the street" expression is because I didn't have anything concrete but was also interested if you had any feed back (been outstanding question of mine, and haven't dug into the kernel to investigate).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting... did not know this. I guess we should use ioctl then :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I also looked at that stackoverflow post. The comments in nginx were more convincing to me (listing the calls that fcntl does). I think for setting non-blocking ioctl is the way to go, but unfortunately I'm not sure if ioctl is always better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me open another issue for that. I would love to investigate a bit more here before doing any changes that may affect things in a bad way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #3357

@NiteshKant
Copy link
Member

@normanmaurer Nice! This is required in netflix, so its great that you got it going!

@normanmaurer
Copy link
Member Author

@NiteshKant BOOM ! ;)

break;
}
expectedWrittenBytes -= localWrittenBytes;
writtenBytes += localWrittenBytes;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid redundant computation would it make sense to compute this once at the end? For example:

final long initialExpectedWrittenBytes = expectedWrittenBytes;

for (;;) { /*do writes and decrement expectedWrittenBytes*/ }

in.removeBytes(initialExpectedWrittenBytes - expectedWrittenBytes);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess yeah we could do this. Not sure it makes any difference though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just less computation and memory access required. A reduction from O(n) load/add/store operations (where n is the number of iterations required for this loop) to O(1).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Scottmitch done... I guess we should do the same in nio from which this code derived :)

@normanmaurer normanmaurer self-assigned this Jan 22, 2015
return fd;
}

JNIEXPORT jint JNICALL Java_io_netty_channel_epoll_Native_bindDomainSocket(JNIEnv * env, jclass clazz, jint fd, jstring socketPath) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see:

  • JNIEnv * env,
  • JNIEnv *env, and
  • JNIEnv* env

Could you use JNIEnv* env everywhere?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for other pointer declarations, such as const char*

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@trustin
Copy link
Member

trustin commented Jan 26, 2015

First pass done. Please ping me once ready for another pass.

@normanmaurer
Copy link
Member Author

@trustin ping :)

Motivation:

Using Unix Domain Sockets can be very useful when communication should take place on the same host and has less overhead then using loopback. We should support this with the native epoll transport.

Modifications:

- Add support for Unix Domain Sockets.
- Adjust testsuite to be able to reuse tests.

Result:

Unix Domain Sockets are now support when using native epoll transport.
@normanmaurer
Copy link
Member Author

Cherry-picked into 4.0 (b898bdd), 4.1 (f771a97) and master (19ec0f3)

@zuoqinbo
Copy link

zuoqinbo commented Sep 4, 2019

i want to got a link for Unix domain Socket guide demo ,use netty!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants