[kvm] Entering into pod #1303

jellonek · 2015-08-19T14:40:57Z

First step on route to implement entering into POD mounted as lkvm VM.

RFC. Especially @krnowak

alban · 2015-08-20T09:31:49Z

stage1/usr_from_kvm/keys/lkvm

@@ -0,0 +1,12 @@
+-----BEGIN RSA PRIVATE KEY-----


Does it mean that anyone (even non-root) who has IP connectivity to the container port 22 will be able to ssh using this well-known "private" key?

Similar idea as in vagrant. sshd can be also enforced (but this would require rewriting config file in time of image preparation) to run only on particular interface/port (for example on primary interface with /31 netmask configured for... host<->vm connection purpose).

jellonek · 2015-08-24T12:01:05Z

Ok, I've changed behavior. Instead of using hardcoded key pair, I'm generating such one per host machine, once in run or prepare time, and then I'm copying public key into roots authorized_keys for ssh.

So keys are generated once per host, and are copied once per POD.

ppalucki · 2015-08-26T14:17:23Z

stage0/kvm.go

+	}
+
+	err = ensureKeyExistenceInPod(rootfsPath, dataDir)
+	if err != nil {


it is enough to just return err or even return ensureKeyExistenceInPod(rootfsPath, dataDir)

Ack. Good catch.

ppalucki · 2015-08-26T16:53:07Z

stage1/enter_kvm/enter_kvm.go

+	return nets[0].IP.String(), nil
+}
+
+func execToSSH() int {


I don't like "int as error code return here" - we should stick with error created by errors.Errorf() and then in main()

func main() { ... err := execToSSH() if err != nil { fmt.Fprintf(os.Stderr, "execToSSH: %v\n", err) os.Exit(1) } }

the resposbility how to handle/display error should be left to caller

So probably You should also comment this same form in stage1/init/init.go
I was only replicating principles from other parts of rkt.

Your application is completely independent, and there is not need to copy IMHO ~~wrong~~ pattern (also there is no custom to comment existing code within new PR 😏 )

update: I have looked more detailed in init.go, and this pattern was used, because different error codes are returned for every case (in your situation, you return 1 everywhere)

jellonek · 2015-08-28T11:09:32Z

typos in commit message...

jonboulle · 2015-09-01T19:28:06Z

stage0/kvm.go

+
+func ensureDirExists(path string) error {
+	err := os.MkdirAll(path, 0700)
+	if err != nil && os.IsExist(err) {


slightly superfluous:

If path is already a directory, MkdirAll does nothing and returns nil.

jonboulle · 2015-09-01T22:02:44Z

@vcaputo do you have any time to review this one?

krnowak · 2015-09-18T14:41:12Z

stage1/usr_from_kvm/files.mk

+
+STAGE1_INSTALL_DIRS += \
+	$(ACIROOTFSDIR)/etc:0755 \
+	$(ACIROOTFSDIR)/root/.ssh:0700 \


Nothing depends on that directory, so it will not be created. Anyway, you create it in the code.

jonboulle · 2015-09-18T19:19:04Z

Sorry for the lateness, I've started writing this comment a few times over the past week and it keeps getting eaten by my browser.

In OOB discussions about this, @philips raised a serious concern about permanently running an SSH server in a pod. Aside from technical concerns (attack vector, resource usage, whatever) there is a conceptual/spec issue. Essentially, we're violating the network contract of the pod: instead of giving the application complete control over the networking namespace/interface(s) we provide them, we've now arbitrarily controlled one of their ports.

There were a couple of proposed solutions to mitigate this concern that we thought might be worth exploring:

could we spin up an sshd instance on-demand when a user does rkt enter, rather than having it always run? For example, signalling something to the pod via 9P and having it create a oneshot sshd on a random port
Eugene had an idea about inverting the connection; basically, rkt client would signal to the kvm pod that it should initiate a connection back to the client (which would effectively then be acting as the server). @eyakubovich I think you had an example of this from MS land, can you remind me?

jellonek · 2015-09-18T19:29:23Z

Yes, there is a method to spin up ssh server only when it's needed - we can achieve this by socket activation. I planned to add this as next step in separate pull request (it also fixes tests prepared by @ppalucki) but if there is desire for this, in time of this weekend I'll code this.

Other thing is port on which ssh is listening (or systemd will wait for connection in inetd style after changing to use socket activation) - if such port will be randomized (some upper ports) - this also potentially increases "security by obscurity", and also gives back 22 port for user usage.

ppalucki · 2015-09-21T14:43:06Z

@jellonek FYI TestEnv (races between run&enter) passes 👍 , even without explicitly defined ordering between application & sshd service.

Please consider moving host keys generation kvmCheckSSHSetup from run to enter command to optimize application startup.

jellonek · 2015-09-21T15:55:33Z

As I mentioned before (or maybe in discussion with @krnowak) I will move this from run into enter (hopefully to stage1 part of entering) but after initial key generation this should have now minimal impact to application speed. Once keys are generated and stored into __datadir__/lkvm - there should be no slowdown of anything. Tests are a separate case, because (as i remember) datadir in time of tests is somewhere in /tmp.

jellonek · 2015-09-29T12:45:34Z

Rebased onto master.

krnowak · 2015-09-30T14:12:57Z

stage0/kvm.go

+		"-N", "", // no passphrase
+	).Output()
+	if err != nil {
+		return fmt.Errorf("error in keygen time. ret_val: %v, output: %v", err, string(out[:len(out)]))


Isn't string(out[:len(out)]) same as simply string(out)?

Try it ;)
No, it isn't. It would not trim ending multiple \0

krnowak · 2015-09-30T14:57:00Z

Ok, minor nits I haven't noticed before (and this named return value issue that was not addressed yet), but otherwise I'm ready to merge it.

Once this is merged, I'll open an issue about stuff that needs to be addressed WRT. entering on KVM. That would be:

Use different data directory (/var/lib/rkt-lkvm-or-something)
Run the shell-providing daemon (funny wording to avoid using specific names) on demand by:
- using socket activation with port randomization, or by
- running something else, like:
  - something that does not require opening a port in container (which would in turn violates the spec/contract whatever), or
  - something simpler (rsh, telnet, whatever), but
  - it is problematic, because:
    - there is no suitable replacement in CoreOS distro, or
    - it would require to write your own stuff, or
    - it would require to build additional tool from sources (like we do with lkvm) to put it in stage1
Move the key generation out from stage0 (into stage1/enter_kvm)

krnowak · 2015-10-01T11:38:28Z

stage1/usr_from_kvm/files/sshd_config

@@ -0,0 +1,36 @@
+Port 22


Shouldn't it be 122?

This doesn't matter, we are using inetd mode of ssdh running (-i parameter) for "old style" socket activation.
I'll try to remove this line (should be working ok), but at this moment I don't have time to test this, so this could be done after merge in separate small issue.

jellonek · 2015-10-01T13:35:30Z

Not ready to merge

[kvm] Entering into pod

jonboulle added this to the v0.9.0 milestone Aug 19, 2015

jonboulle added the component/stage1 label Aug 19, 2015

alban reviewed Aug 20, 2015
View reviewed changes

jellonek force-pushed the entering branch 2 times, most recently from c335041 to a16434f Compare August 24, 2015 11:27

jellonek force-pushed the entering branch 2 times, most recently from 48d1577 to cbe96c5 Compare August 26, 2015 14:01

ppalucki reviewed Aug 26, 2015
View reviewed changes

jellonek force-pushed the entering branch 2 times, most recently from 46e85d7 to 76a9b9d Compare August 26, 2015 14:42

ppalucki reviewed Aug 26, 2015
View reviewed changes

jellonek force-pushed the entering branch 2 times, most recently from 15597df to 84b0444 Compare August 28, 2015 11:03

jellonek changed the title ~~[WIP] kvm: entering into POD~~ [kvm] Entering into POD Aug 28, 2015

jellonek force-pushed the entering branch 2 times, most recently from 3019bb1 to 9792e45 Compare August 28, 2015 11:09

jonboulle reviewed Sep 1, 2015
View reviewed changes

krnowak reviewed Sep 18, 2015
View reviewed changes

jonboulle changed the title ~~[kvm] Entering into POD~~ [kvm] Entering into pod Sep 18, 2015

jellonek force-pushed the entering branch 3 times, most recently from 8fb2259 to 994a206 Compare September 21, 2015 12:51

jellonek force-pushed the entering branch from 994a206 to 2b1a2df Compare September 29, 2015 10:51

jellonek mentioned this pull request Sep 29, 2015

[kvm] partial fix networking #1501

Closed

jellonek mentioned this pull request Sep 29, 2015

[kvm] Macvlan network type #1364

Merged

krnowak reviewed Sep 30, 2015
View reviewed changes

jellonek force-pushed the entering branch 2 times, most recently from d7965b1 to c647a4f Compare October 1, 2015 11:26

krnowak reviewed Oct 1, 2015
View reviewed changes

jellonek force-pushed the entering branch from c647a4f to 7af8ee6 Compare October 1, 2015 13:16

kvm: Entering into pod

355b3dd

jellonek force-pushed the entering branch from 7af8ee6 to 355b3dd Compare October 1, 2015 13:37

krnowak added a commit that referenced this pull request Oct 1, 2015

Merge pull request #1303 from intelsdi-x/entering

62b27c6

[kvm] Entering into pod

krnowak merged commit 62b27c6 into rkt:master Oct 1, 2015

krnowak mentioned this pull request Oct 1, 2015

kvm: Followup issues on entering into pod #1525

Open

4 tasks

jellonek deleted the entering branch October 1, 2015 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[kvm] Entering into pod #1303

[kvm] Entering into pod #1303

jellonek commented Aug 19, 2015

alban Aug 20, 2015

jellonek Aug 20, 2015

jellonek commented Aug 24, 2015

ppalucki Aug 26, 2015

jellonek Aug 26, 2015

ppalucki Aug 26, 2015

jellonek Aug 27, 2015

ppalucki Aug 27, 2015

jellonek commented Aug 28, 2015

jonboulle Sep 1, 2015

jonboulle commented Sep 1, 2015

krnowak Sep 18, 2015

jonboulle commented Sep 18, 2015

jellonek commented Sep 18, 2015

ppalucki commented Sep 21, 2015

jellonek commented Sep 21, 2015

jellonek commented Sep 29, 2015

krnowak Sep 30, 2015

jellonek Oct 1, 2015

krnowak commented Sep 30, 2015

krnowak Oct 1, 2015

jellonek Oct 1, 2015

jellonek commented Oct 1, 2015

[kvm] Entering into pod #1303

[kvm] Entering into pod #1303

Conversation

jellonek commented Aug 19, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jellonek commented Aug 24, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jellonek commented Aug 28, 2015

Choose a reason for hiding this comment

jonboulle commented Sep 1, 2015

Choose a reason for hiding this comment

jonboulle commented Sep 18, 2015

jellonek commented Sep 18, 2015

ppalucki commented Sep 21, 2015

jellonek commented Sep 21, 2015

jellonek commented Sep 29, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

krnowak commented Sep 30, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jellonek commented Oct 1, 2015