Could you please tweak your commit message slightly?

stage1: upgrade lkvm to current master branch to fix 9pfs bug

Fixes #1917

thanks!

/cc @jellonek @ppalucki

I had this on my agenda also, LGTM.

semaphore error seems not connected with this changes, so IMO this is ready to be merged.

Hold a second to merge this PR, since there's another bug introduced with this fix.

After lkvm upgrades, "apt-get update" works fine, but there's another bug with some "apt-get install". Here's the detailed outputs:

root@clr:/# apt-get install vim
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libgpm2 libpython2.7 libpython2.7-minimal libpython2.7-stdlib vim-runtime
Suggested packages:
gpm ctags vim-doc vim-scripts
The following NEW packages will be installed:
libgpm2 libpython2.7 libpython2.7-minimal libpython2.7-stdlib vim
vim-runtime
0 upgraded, 6 newly installed, 0 to remove and 2 not upgraded.
Need to get 9076 kB of archives.
After this operation, 42.9 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main libgpm2 amd64 1.20.4-6.1 [16.5 kB]
Get:2 http://archive.ubuntu.com/ubuntu/ trusty-updates/main libpython2.7-minimal amd64 2.7.6-8ubuntu0.2 [308 kB]
Get:3 http://archive.ubuntu.com/ubuntu/ trusty-updates/main libpython2.7-stdlib amd64 2.7.6-8ubuntu0.2 [1869 kB]
Get:4 http://archive.ubuntu.com/ubuntu/ trusty-updates/main libpython2.7 amd64 2.7.6-8ubuntu0.2 [1039 kB]
Get:5 http://archive.ubuntu.com/ubuntu/ trusty/main vim-runtime all 2:7.4.052-1ubuntu3 [4888 kB]
Get:6 http://archive.ubuntu.com/ubuntu/ trusty/main vim amd64 2:7.4.052-1ubuntu3 [956 kB]
Fetched 9076 kB in 9s (976 kB/s)
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
Selecting previously unselected package libgpm2:amd64.
(Reading database ... 11542 files and directories currently installed.)
Preparing to unpack .../libgpm2_1.20.4-6.1_amd64.deb ...
Unpacking libgpm2:amd64 (1.20.4-6.1) ...
Selecting previously unselected package libpython2.7-minimal:amd64.
Preparing to unpack .../libpython2.7-minimal_2.7.6-8ubuntu0.2_amd64.deb ...
Unpacking libpython2.7-minimal:amd64 (2.7.6-8ubuntu0.2) ...
Selecting previously unselected package libpython2.7-stdlib:amd64.
Preparing to unpack .../libpython2.7-stdlib_2.7.6-8ubuntu0.2_amd64.deb ...
Unpacking libpython2.7-stdlib:amd64 (2.7.6-8ubuntu0.2) ...
Selecting previously unselected package libpython2.7:amd64.
Preparing to unpack .../libpython2.7_2.7.6-8ubuntu0.2_amd64.deb ...
Unpacking libpython2.7:amd64 (2.7.6-8ubuntu0.2) ...
Selecting previously unselected package vim-runtime.
Preparing to unpack .../vim-runtime_2%3a7.4.052-1ubuntu3_all.deb ...
Adding 'diversion of /usr/share/vim/vim74/doc/help.txt to /usr/share/vim/vim74/doc/help.txt.vim-tiny by vim-runtime'
Adding 'diversion of /usr/share/vim/vim74/doc/tags to /usr/share/vim/vim74/doc/tags.vim-tiny by vim-runtime'
Unpacking vim-runtime (2:7.4.052-1ubuntu3) ...
dpkg: error processing archive /var/cache/apt/archives/vim-runtime_2%3a7.4.052-1ubuntu3_all.deb (--unpack):
* unable to create /usr/share/vim/vim74/syntax/dnsmasq.vim.dpkg-new' (while processing./usr/share/vim/vim74/syntax/dnsmasq.vim'): Too many open files in system*
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Removing 'diversion of /usr/share/vim/vim74/doc/help.txt to /usr/share/vim/vim74/doc/help.txt.vim-tiny by vim-runtime'
Removing 'diversion of /usr/share/vim/vim74/doc/tags to /usr/share/vim/vim74/doc/tags.vim-tiny by vim-runtime'
Selecting previously unselected package vim.
Preparing to unpack .../vim_2%3a7.4.052-1ubuntu3_amd64.deb ...
Unpacking vim (2:7.4.052-1ubuntu3) ...
Errors were encountered while processing:
/var/cache/apt/archives/vim-runtime_2%3a7.4.052-1ubuntu3_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@clr:/#

I'm trying to find why "Too many open files in system" here and have no idea about this. And also I don't know how to replay this bug in a more simple way.

Thanks,
Chunqi

Hi All,

Bug #1917 has a pretty complexed solution. Simply, bug #1917 triggered by fstat and unlinked file in 9pfs. Both kernel and lkvm need patch to fix this bug.

On stage-1 kernel side, a patch has been committed by 9pfs author Eric, but has not been merged to kernel upstream. When a file is unlinked, stage-1 kernel need to search inode instead to find the real fid. Here's the patch:

http://git.kernel.org/cgit/linux/kernel/git/ericvh/v9fs.git/commit/?h=v9fs-devel&id=5eb393c464294b2183d526a25a344d7ae6ba8383

On lkvm side, 9pfs server need to use fstat to retrieve the attr of an unlinked file. Here's a patch from ericvh:

ericvh/linux-kvm@2fa2f7e

Only with patches above, bug #1917 could be completely fixed.

I think the "Too many open files in system" bug is another bug of lkvm and still have no idea of why it happens. I will have more investigation into it.

Thanks,
Chunqi

IMO this PR even if does not resolve an issue, moves it forward.

If You want to add these patches into this PR (IMO they should be added in separate commits which could be easy reverted when this modifications will go into upstream) please add lkvm patch (as single file) into stage1/usr_from_kvm/lkvm/patches/ and similarly for kernel into stage1/usr_from_kvm/kernel/patches/.

I have restarted the semaphore build twice and it failed with unrelated errors, but the last one is quite interesting. @jonboulle, you added some goroutine leak detection recently, right? There is some error in the semaphore build seemingly related to it.

I'll restart the semaphore build when it is clear what's wrong with the current error.

I don't really understand why that's showing up now - is this the only place it's occurring?

No idea, I saw it for the first time.

Heisenbug?

LGTM

lol at heisenbug :)
ok, so my work on lkvm/kernel upgrade will be shorter ;)