Error enabling TokuDB through ps_tokudb_admin

Bug #1520890 reported by Rafal
22
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Status tracked in 5.7
5.6
Fix Released
Medium
Tomislav Plavcic
5.7
Fix Released
Medium
Tomislav Plavcic

Bug Description

Ubuntu 15.10
Docker 1.9.1

On container building:
Processing triggers for libc-bin (2.19-0ubuntu6.6) ...
151129 08:15:18 mysqld_safe Adding '/usr/lib/x86_64-linux-gnu/libjemalloc.so.1' to LD_PRELOAD for mysqld
151129 08:15:18 mysqld_safe Logging to '/var/lib/mysql/a52c17016130.err'.
151129 08:15:18 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
mysqld is alive
Checking if Percona Server is running with jemalloc enabled...
grep: /proc/840/environ: Permission denied
ERROR: Percona Server is not running with jemalloc, please restart mysql service to enable it and then run this script...

Previous version of ps_tokudb_admin:
# Check if server is running with jemalloc - if not warn that restart is needed
if [ $ENABLE = 1 ]; then
  printf "Checking if Percona Server is running with jemalloc enabled...\n"
  JEMALLOC_STATUS=$(grep -c jemalloc /proc/${PID_NUM}/environ)
  if [ $JEMALLOC_STATUS = 0 ]; then
    printf "ERROR: Percona Server is not running with jemalloc, please restart mysql service to enable it and then run this script...\n\n";
    exit 1
  else
    printf "INFO: Percona Server is running with jemalloc enabled.\n\n";
  fi
fi

Current version of ps_tokudb_admin with bug:
# Check if server is running with jemalloc - if not warn that restart is needed
if [ $ENABLE = 1 ]; then
  printf "Checking if Percona Server is running with jemalloc enabled...\n"
  grep -qc jemalloc /proc/${PID_NUM}/environ || ldd $(which mysqld) | grep -qc jemalloc
  JEMALLOC_STATUS=$?
  if [ $JEMALLOC_STATUS = 1 ]; then
    printf "ERROR: Percona Server is not running with jemalloc, please restart mysql service to enable it and then run this script...\n\n";
    exit 1
  else
    printf "INFO: Percona Server is running with jemalloc enabled.\n\n";
  fi
fi

Tags: pkg tokudb
Revision history for this message
Duke Williams (spmbx123) wrote :
Revision history for this message
Sveta Smirnova (svetasmirnova) wrote :

Thank you for the report.

I see "permission denied" in the output:

grep: /proc/840/environ: Permission denied

Under which user you run ps_tokudb_admin? Does it have read privileges to /proc directory?

Changed in percona-server:
status: New → Incomplete
Revision history for this message
Rafal (andruwn) wrote :

In docker container under root

Revision history for this message
Rafal (andruwn) wrote :

> Does it have read privileges to /proc directory?
In docker containers access to /proc/* closed by default

Revision history for this message
Duke Williams (spmbx123) wrote :

Please return this version:

# Check if server is running with jemalloc - if not warn that restart is needed
if [ $ENABLE = 1 ]; then
  printf "Checking if Percona Server is running with jemalloc enabled...\n"
  JEMALLOC_STATUS=$(grep -c jemalloc /proc/${PID_NUM}/environ)
  if [ $JEMALLOC_STATUS = 0 ]; then
    printf "ERROR: Percona Server is not running with jemalloc, please restart mysql service to enable it and then run this script...\n\n";
    exit 1
  else
    printf "INFO: Percona Server is running with jemalloc enabled.\n\n";
  fi
fi

I think all the changes need to be tested in docker before publishing.
Thanks

Revision history for this message
Tomislav Plavcic (tplavcic) wrote :

ps_tokudb_admin cannot run in docker container for sure because it also tries to change the transparent huge pages setting which is from the host and the script cannot change the setting on the host.
The version from before had the same error but it was continuing to run because it just skipped the error because it was checking for something else.
Will check if there's something that can be done for this to run in docker container, but it is very easy to add INSTALL PLUGIN stuff into Dockerfile and prepare a my.cnf with needed options so that docker container build works.

Revision history for this message
Rafal (andruwn) wrote :

It doesn't try to change the transparent hugepages setting. I have setted transparent hugepages on host, and ps_tokudb_admin in container writes ok. Thus it only checks transparent hugepages is enabled. But it does not work with jemalloc.

Revision history for this message
Duke Williams (spmbx123) wrote :

Tomislav Plavcic (tplavcic), I enabled jemalloc and transparent hugepages on host, and on host ps_tokudb_admin works fine. But in container it writes error. So jemalloc check in ps_tokudb_admin works incorrect. Bug opened a month ago and no one still does not know how to solve this problem. I propose to delete this check and close this bug.

Revision history for this message
Tomislav Plavcic (tplavcic) wrote :

This check is needed at least when not used inside a container and previous version was just ignoring the error (which is not good).

Rafal (andruwn), Duke Williams (spmbx123)
Would a new option "--skip-jemalloc-check" work for you?

Revision history for this message
Uoren Glocker (j1227444) wrote :

I think it should check permissions and display warning message without exiting
> Would a new option "--skip-jemalloc-check" work for you?
What about transparent hugepages and other checks?

Revision history for this message
Duke Williams (spmbx123) wrote :

> Would a new option "--skip-jemalloc-check" work for you?
Good idea, but perhaps it will be better to check permissions and display warning if read access to /proc/${PID_NUM}/environ closed, because in such situation it is impossible to define is jemalloc enabled or not.

Revision history for this message
Tomislav Plavcic (tplavcic) wrote :

Decided to put a warning for jemalloc check and libHotBackup.so check (same thing) - transparent huge pages works as expected (you need to set it up on the host and script check works ok).
If the below PR's get merged it should be available in the next releases.
PR 5.6: https://github.com/percona/percona-server/pull/287
PR 5.7: https://github.com/percona/percona-server/pull/288

Changed in percona-server:
assignee: nobody → Tomislav Plavcic (tplavcic)
milestone: none → 5.6.28-76.1
status: Incomplete → Fix Committed
importance: Undecided → Medium
tags: added: pkg
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1674

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.