Currently ports that are forwarded to containers are not accessible on the host via localhost:forwarded-port.

Do you have an example of commands to show what is not available? I tried the following:

sudo ./build-rkt-0.7.0+git/bin/rkt --local --insecure-skip-verify run --mds-register=false --volume volume-data,kind=host,source=/tmp/redis docker://redis

It exposes the port 6379:

        "ports": [
            {
                "name": "6379-tcp",
                "protocol": "tcp",
                "port": 6379,
                "count": 1,
                "socketActivated": false
            }
        ]

And then on the host: nc localhost 6379 and I could talk successfully to redis in the pod.

I also tried to setup a ssh forwarded port with ssh -L 6378:localhost:6379 localhost and to connect with nc localhost 6378 and it worked as well.

Oh sorry, I should not use the defaut network namespace for testing this. So the full command I tried is:

sudo ./build-rkt-0.7.0+git/bin/rkt --local --insecure-skip-verify run --mds-register=false --volume volume-data,kind=host,source=/tmp/redis --private-net=default --port=6379-tcp:8080 docker://redis

And then I have the behavior you described when trying with nc 192.168.0.13 8080 (it works) and nc localhost 8080 (it does not work).

Do you know if kube-proxy uses localhost to connect to a kubernetes endpoint?

@eyakubovich @steveej ping?

I'm okay with the current behavior as I would not expect it to behave differently. I brought up the question to see what the general assumption would be. We have to make the decision if we leave it as is (my vote) or if we enable NAT loopback.

It seems pretty unintuitive to me. Can you help explain your vote?

I was bitten by this yesterday. I think the principle of least surprise would dictate that the forwarded port is also reachable via localhost. I'd be fine with it being a runtime option (to --port or something). But in the case of CoreOS where everything must run as a container, it really would simplify connecting to services if I didn't first have to look up the local host's IP, or depend on $HOSTNAME or $( hostname ).

I was debugging this for a few hours yesterday, trying to reach a specific container from the host is really hard without doing this, I don't think there's any disadvantage to making it work on localhost.

@blalor @yorickvP thanks for your feedback on this, supporting @jonboulle's intuition. This topic has come up often enough lately to convince me that we should have NAT loopback.

@steveej Any chance we can get this for the next release?

@iaguis it looks like we're having trouble getting this in at all.

Rewriting localhost:forwarded-port -> container-IP:forwarded-port doesn't work by design with iptables.

As an example, the following rule

-A OUTPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 1024 -j DNAT --to-destination 172.16.28.2:1024

Will be matched, but it will leave the package destination unchanged, as can be seen in the TRACE output:

Jan 22 21:47:05 steveej-laptop kernel: TRACE: raw:OUTPUT:policy:2 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=55893 DF PROTO=TCP SPT=39992 DPT=1024 SEQ=1195156359 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A029579D20000000001030307) UID=1000 GID=100 
Jan 22 21:47:05 steveej-laptop kernel: TRACE: mangle:OUTPUT:policy:1 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=55893 DF PROTO=TCP SPT=39992 DPT=1024 SEQ=1195156359 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A029579D20000000001030307) UID=1000 GID=100 
Jan 22 21:47:05 steveej-laptop kernel: TRACE: nat:OUTPUT:rule:1 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=55893 DF PROTO=TCP SPT=39992 DPT=1024 SEQ=1195156359 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A029579D20000000001030307) UID=1000 GID=100

Any suggestions are welcome!

UPDATE: Fortunately I was wrong with my conclusion from last evening. I'll prepare a PR with a proposal.