Libzdb

Version 3.2.3

A small, easy to use Open Source Database Connection Pool Library with the following features:

  • Thread safe Database Connection Pool
  • Connect to multiple database systems simultaneously
  • Zero runtime configuration, connect using a URL scheme
  • Supports MySQL, PostgreSQL, SQLite and Oracle
Download

Requirements: Runs on iOS, Linux, OS X, FreeBSD, Solaris, OpenBSD and other POSIX systems. A C99 compiler is required to build the library.

Compatible with and can be included in C++ or Objective-C projects

Compatible with and can be included in C++ or Objective-C projects.

Modern, Object Oriented API design. Fully documented.

The library is licensed under a Free Open Source Software License.

Used in M/Monit

- The M/Monit Team mmonit.com

Used in DBMail

dbmail.org
Can I use libzdb in my iOS or macOS app?

Yes, libzdb can be used from and included in any C, C++ or Objective-C project. The Xcode project file used to develop libzdb is available from the repository and can be included in your own Xcode project.

Is the library thread-safe?

Libzdb is thread-safe and designed to be used in a multi-threaded program.

Is the connection pool dynamic?

Yes, the pool can be setup to dynamically change the number of Connections in the pool depending on the load.

Can I connect to multiple database systems at the same time?

Of course, libzdb is perfect as a glue layer between different database systems or databases. For instance, you can read data from MySQL and write to PostgresSQL in just a few lines of code

Public code repository?

The project is hosted at Bitbucket. Click the r icon in the footer below to visit

Are there plans to support additional database systems?

Libzdb currently supports SQLite, MySQL, PostgreSQL and Oracle. At the moment there are no plans to support additional database systems.

Libzdb API documentation
ConnectionPool URL Connection PreparedStatement ResultSet

Clickable API documentation

Connection URL:

The URL given to a Connection Pool at creation time specify a database connection on the standard URL format. The format of the connection URL is defined as:

database://[user:password@][host][:port]/database[?name=value][&name=value]...

The property names user and password are always recognized and specify how to login to the database. Other properties depends on the database server in question. User name and password can alternatively be specified in the auth-part of the URL. If port number is omitted, the default port number for the database server is used. Reserved characters used in the Connection URL must be URL encoded.

MySQL:

Here is an example on how to connect to a MySQL database server:

mysql://localhost:3306/test?user=root&password=swordfish

In this case the username, root and password, swordfish are specified as properties to the URL. An alternative is to use the auth-part of the URL to specify authentication information:

mysql://root:swordfish@localhost:3306/test

See mysql options for all properties that can be set for a mysql connection URL.

SQLite:

For a SQLite database the connection URL should simply specify a database file, since a SQLite database is just a file in the filesystem. SQLite uses pragma commands for performance tuning and other special purpose database commands. Pragma syntax on the form, name=value can be added as properties to the URL and will be set when the Connection is created. In addition to pragmas, the following properties are supported:

  • heap_limit=value [KB] - Make SQLite auto-release unused memory if memory usage goes above the specified value.

An URL for connecting to a SQLite database might look like:

sqlite:///var/sqlite/test.db?synchronous=normal&heap_limit=8000&foreign_keys=on

PostgreSQL:

The URL for connecting to a PostgreSQL database server might look like:

postgresql://localhost:5432/test?user=root&password=swordfish

As with the MySQL URL, the username and password are specified as properties to the URL. Likewise, the auth-part of the URL can be used instead to specify the username and the password:

postgresql://root:swordfish@localhost/test?use-ssl=true

In this example we have also omitted the port number to the server, in which case the default port number, 5432, for PostgreSQL is used. In addition we have added an extra parameter to the URL, so connection to the server is done over a secure SSL connection.

See postgresql options for all properties that can be set for a postgresql connection URL.

Oracle:

The URL for connecting to an Oracle database server might look like:

oracle://localhost:1521/servicename?user=scott&password=tiger

Instead of a database name, Oracle uses a service name. The information in the url above is typically specified in a tnsnames.ora configuration file, pointed to by the environment variable TNS_ADMIN. In the example below, instead of host, port and service name, we use a tnsname as defined in tnsnames.ora. We also use the auth-part of the URL to specify the username and the password. Finally, we specify that we want to connect to Oracle with the SYSDBA role.

oracle://sys:secret@/tnsname?sysdba=true

See oracle options for all properties that can be set for an oracle connection URL.

Examples:

To obtain a connection pool for a MySQL database, the code below can be used. The exact same code can be used for PostgreSQL, SQLite and Oracle, the only change needed is to modify the Connection URL. Here we connect to the database test on localhost and start the pool with the default 5 initial connections.

ConnectionPool, Connection and ResultSet:

URL_T url = URL_new("mysql://localhost/test?user=root&password=swordfish");
ConnectionPool_T pool = ConnectionPool_new(url);
ConnectionPool_start(pool);

Connection_T con = ConnectionPool_getConnection(pool);
ResultSet_T result = Connection_executeQuery(con, 
                     "select id, name, image from employee where salary > %d", aNumber);
while (ResultSet_next(result)) 
{
     int id = ResultSet_getInt(result, 1);
     const char *name = ResultSet_getString(result, 2);
     int blobSize;
     const void *image = ResultSet_getBlob(result, 3, &blobSize);
     [..]
}
                

Here is another example where a generated result is selected and printed:

ResultSet_T r = Connection_executeQuery(con, "SELECT count(*) FROM users");
printf("Number of users: %s\n", ResultSet_next(r) ? ResultSet_getString(r, 1) : "no users");
                

Insert, Update, and Delete

Use Connection_execute() to execute SQL statements that do not return a result set. Such statements can be INSERT, UPDATE, or DELETE statements, or SQL DDL statements. In the examples below, assume that data is a structure holding information about an employee.

Insert:
Connection_execute(con, "INSERT INTO employee (name, hired) VALUES ('%s', %ld);", 
data.name, data.hired);
                
Update:
Connection_execute(con, "UPDATE employee SET name = '%s' WHERE id = %d", 
data.name, data.id);
                
Delete:
Connection_execute(con, "DELETE FROM employee WHERE id = %d", data.id);
                

The functions Connection_execute() and Connection_executeQuery() are limited to handling values that can be represented as C-strings. If you need to manipulate binary data, such as inserting a blob value into the database, use a Prepared statement instead (see below). Utilizing a Prepared statement is also highly advised when dealing with user-submitted data, as it effectively safeguards against any potential SQL injection vulnerabilities.

Prepared statement:

PreparedStatement_T p = Connection_prepareStatement(con, 
                        "INSERT INTO employee(name, picture) VALUES(?, ?)");
PreparedStatement_setString(p, 1, "Kamiya Kaoru");
PreparedStatement_setBlob(p, 2, jpeg, jpeg_size);
PreparedStatement_execute(p);
               

Here, we use a Prepared Statement to execute a query which returns a Result Set:

PreparedStatement_T p = Connection_prepareStatement(con, 
                        "SELECT id FROM employee WHERE name LIKE ?"); 
PreparedStatement_setString(p, 1, "%Kaoru%");
ResultSet_T r = PreparedStatement_executeQuery(p);
while (ResultSet_next(r))
       printf("employee.id = %d\n", ResultSet_getInt(r, 1));
               

C++ Examples:

To use libzdb in your C++ project (C++17 or later), import zdbpp.h and use the namespace zdb:

#include <zdbpp.h>
using namespace zdb;
               

Query Example

ConnectionPool pool("mysql://192.168.11.100:3306/test?user=root&password=dba");
pool.start();
Connection con = pool.getConnection();
// Use C++ variadic template feature to bind parameter 
ResultSet result = con.executeQuery(
     "select id, name, hired, image from employee where id < ? order by id", 100
);
// Optionally set row prefetch, default is 100
result.setFetchSize(10);
while (result.next()) {
     int id = result.getInt("id");
     const char *name = result.getString("name");
     time_t hired = result.getTimestamp("hired");
     auto [image, size] = result.getBlob("image");
     ...
}
               

Execute statement

Connection con = pool.getConnection();
// Any execute or executeQuery statement which takes parameters are 
// automatically translated into a prepared statement. Here we also 
// demonstrate how to set a SQL null value by using nullptr
con.execute("update employee set image = ? where id = ?", nullptr, 11);
                

Test for SQL null value

ResultSet result = con.executeQuery("select name, image from employee");
while (result.next()) {
    if (result.isnull("image")) {
        ...
    }
}
               

Insert Data via Prepared Statement

Connection con = pool.getConnection();
PreparedStatement prep = con.prepareStatement(
     "insert into employee (name, hired, image) values(?, ?, ?)"
);
con.beginTransaction();
for (const auto &employee : employees) {
        // Polymorphic bind
        prep.bind(1, employee.name);
        prep.bind(2, employee.hired);
        prep.bind(3, employee.image);
        prep.execute();
}
con.commit();
               

Exception Handling

try {
    con = pool.getConnection();
    con.executeQuery("invalid query");
} catch (sql_exception& e) {
    std::cout <<  e.what();
}
               

Libzdb is the perfect glue between your databases

You can use libzdb to instantiate as many ConnectionPool objects as needed and against as many different database systems as needed. This means libzdb is perfect for querying, copying and moving data between different database systems and databases.

Below is a somewhat contrived example where we copy data from a MySQL database to a PostgreSQL database. The main point, however, is to demonstrate how libzdb can serve as an efficient glue layer between your various databases.

Copying Apples to Oranges:

// Start a ConnectionPool to the Apples Warehouse Database 
ConnectionPool apple_store("mysql://root:fruit@192.168.1.101:3306/apples");
apple_store.start();

// Start a ConnectionPool to the Oranges Warehouse Database                     
ConnectionPool orange_store("postgresql://root:ninja@192.168.1.210/oranges");
orange_store.start();

// Get a Connection to the Orange store
Connection orange_connection = orange_store.getConnection();

// Select Apples we want to copy to Oranges
ResultSet apples = apple_store.getConnection().executeQuery(
     "select name, color, weight from apples"
);

// Create a Prepared statements for storing Oranges
PreparedStatement orange = orange_connection.prepareStatement(
    "insert into oranges (name, color, weight) values(?, ?, ?)"
);

// Copy all Apples to Oranges under a transaction
orange_connection.beginTransaction();
while (apples.next()) {
        orange.bind(1, apples.getString("name"));
        orange.bind(2, apples.getString("color"));
        orange.bind(3, apples.getDouble("weight"));
        orange.execute();
}
orange_connection.commit();
                    

More examples can be found by clicking the API documentation image above.

Version 3.2.3

Released on 18 October 2022
  • New: Provide better error reporting if a Connection cannot be obtained from the Connection Pool.
  • Fixed: In C++, added a guard against stopping the Connection Pool with live active Connections to prevent a dealloc sequence problem.
  • Fixed: Plus many more minor improvements and fixes

Version 3.2.2

Released on 1 April 2020
  • Fixed: Removed Thread.h from the public API.

Version 3.2.1

Released on 6 March 2020
  • New: Include Library version number in zdb.h
  • Fixed: Simplified test/zdbpp.cpp and added missing header
  • Fixed: Improved support for MySQL 8 and MariaDB

Version 3.2

Released on 3 Apr 2019
  • New: C++17 support via zdbpp.h which is distributed with libzdb for more idiomatic use of libzdb from C++. Thanks to dragon jiang (jianlinlong)
  • New: Support prefetch rows for MySQL and Oracle. Either programatically via Connection_setFetchSize() or via ResultSet_setFetchSize() or via a new global fetch-size URL option. Thanks to dragon jiang (jianlinlong)
  • New: MySQL 5.7 and later. Added session query timeout accessible via Connection_setQueryTimeout()
  • New: MySQL 8. Added a new URL option auth-plugin which specify the authentication plugin to use when connecting to a MySQL server.
  • New: Oracle: Added a new URL option sysdba for connecting with sysdba privileges.
  • Fixed: Revert previous fix (#8) and remove last SQL terminator character ';' in statements, except if preceded with END; to allow for ending a pl/sql block.
  • Fixed: Oracle: Set SQL null value in prepared statement
  • Fixed: Oracle: Handle date/time literal values

Version 3.1

Released on 31 Aug 2015
  • New: Support Literal IPv6 Addresses in the Database Connection URL. Ref. RFC2732
  • New: Honour timezone information if provided with date-time column values in Result Sets
  • Fixed: Issue #7 Removed onstop handler
  • Fixed: #8 Do not remove trailing SQL termination charachter ';' from statement

Version 3.0

Released on 06 Jan 2014
  • New: Methods for retrieving Date, Time, DateTime and TimeStamp column values from a ResultSet. PreparedStatement_setTimestamp for setting Unix timestamp.
  • New: ResultSet_isnull, can be used to explicit test if a column value in a Result Set is SQL null. A Result Set already returns the NULL pointer for string and blob types and 0 for primitive data types if column value is SQL null, but to differ between SQL null and the value NULL or 0, this method can be used.
  • New: PreparedStatement_getParameterCount, Returns the number of parameters in a prepared statement
  • New: It is now a checked runtime error for the url parameter given in ConnectionPool_new to be NULL.
  • New: No longer require flex installed as the generated file is part of the distribution.
  • Fixed: Oracle: memory corruption in OracleResultSet when a Blob value is retrieved as a String

Version 2.12

Released on 03 Sep 2013
  • New: PreparedStatement_rowsChanged added to PreparedStatement.h
  • Fixed: Oracle: OCIPing is used to check Oracle connections and to ensure that the Pool returns connected connections. Thanks to Pavlo Lavrenenko.

Version 2.11/3

Released on 05 Jun 2013
  • New: License Exception added to allow the library to be linked and distributed together with OpenSSL.
  • New: Throw SQLException if a database access error occurs when ResultSet_next() is called. Previously, access errors could be masked as end of result set. Thanks to JiaQiang Xu.
  • Fixed: (Volodymyr Tarasenko) Possible mem leak in Oracle's blob operation fixed.
  • Fixed: MySQL: A ResultSet bind memory error could occur if string or blob columns of different size caused successive buffer reallocation. Thanks to Ryan Addams for discovering the problem.
  • New: Added support for the new bytea hex encoding format introduced in PostgreSQL 9.0.
  • Fixed: MySQL: A Result Set with two or more columns larger than 256 bytes would cause libzdb to truncate the second column to 256 bytes in the first row. Thanks to Me from China for bug report and patch.
  • Fixed: Improved Build Configuration. Thanks to Johan Bergström for report.

Version 2.10/.6

Released on 29 Oct 2012
  • New: Libzdb is now compatible with and can be included in C++ or Objective-C(++) projects.
  • Fixed: Oracle: Fixed a connection parameter bug. Thanks to l00106600
  • Fixed: Oracle: Fixed a GCC C99 compile issue. Thanks to Stas Oginsky
  • New: MySQL: Improved error reporting
  • New: Automatically unescape the following URL components: credentials, path and parameter values
  • New: Connection Pool start now throws an SQLException instead of calling abort handler if starting the pool failed. Thanks to Christopher O'Hara
  • Fixed: MySQL: Using a stored procedure which returned a result set would freeze the connection until it was reaped. Thanks to Jesse White
  • Fixed: MySQL: Ensure that the library can be restarted after it was stopped without leaking resources. Only applicable for MySQL which could leak memory on runtime restart. Thanks to Axel Steiner

Version 2.9

Released on 15 Aug 2011
  • New: SQLite: Unescape path to allow for (white-)space in database file URL path. Thanks to Jonas Schnelli
  • Fixed: SQLite: Use sqlite3_open_v2 instead of sqlite3_enable_shared_cache which is deprecated in OS X Lion
  • Fixed: Oracle: Fixed a problem with ResultSet not returning all data.

Version 2.8/.1

Released on 15 Feb 2011
  • New: PostgreSQL: Allow sending application name to the server for logging. Thanks to Chris Mayo. See the PostgreSQL URL property, application-name.
  • Fixed: Oracle: Fixed a ResultSet memory leak
  • Fixed: Oracle: Fixed a transaction related memory leak

Questions or comments?

If you have questions or comments about the software or documentation please subscribe to the libzdb general mailing list and post your questions there.

Open Source

Libzdb is open source. It's hosted, developed, and maintained on Bitbucket.

Reporting a bug

If you believe you have found a bug, please use the issue tracker to report the problem. Remember to include the necessary information that will enable us to understand and reproduce this problem.