PostgreSQL, libpqxx and prepared statement example

As I promised, if people are finding interesting introductory article,  I will write more about PostgreSQL and libpqxx. But before I start with programming – rant.
I am still looking for work and finding none. Today went to Discovery Health, had very pleasant 45 minutes chat with their architect and BA and promise that we will talk again. Later guy from employment agency, who arranged meeting, calls and says that they do not want me since I do not have ANSI C in CV?! Like somebody from management decided to override decision of interviewers. If ANSI C in CV is precondition, why they wasted my time and invited me for interview? I wish them the same from their customers. BTW I am usually employed as senior developer and not as C developer or, Perl developer.
Back to programming, this time C++, had enough of ANSI C for today. Environment is Linux, I am using the same PostgreSQL and libpqxx as last time and to compile example we will use:

g++ hello_prep.cxx -o hello_prep -I/usr/local/include/ -lpqxx -lpq

This time I used test092.cxx, run dpkg with -L switch on libpqxx3-doc to see where is it. It tests passing binary parameter to prepared statement. Test macros are replaced with printing of tested values to standard output and setup of connection/transaction is included. Here is the code:

#include <iostream> #include <pqxx/pqxx> using namespace std; using namespace pqxx; int main(int argc, char** argv) { connection C("dbname=testpgdb user=testuser password=testpass hostaddr=127.0.0.1 port=5432"); if (C.is_open()) { cout << "We are connected to" << C.dbname() << endl; } else { cout << "We are not connected!" << endl; return 0; } work T(C); const char databuf[] = "Test\0data"; const string data(databuf, sizeof(databuf)); cout << "data.size() = " << data.size() << endl; cout << "databuf size = " << sizeof(databuf) << endl; const string Table = "pqxxbin", Field = "binfield", Stat = "nully"; T.exec("CREATE TEMP TABLE " + Table + " (" + Field + " BYTEA)"); if (!T.conn().supports(connection_base::cap_prepared_statements)) { cout << "Backend version does not support prepared statements. Skipping." << endl; return 0; } T.conn().prepare(Stat, "INSERT INTO " + Table + " VALUES ($1)") ("BYTEA", pqxx::prepare::treat_binary); T.prepared(Stat)(data).exec(); const result L( T.exec("SELECT length("+Field+") FROM " + Table) ); cout << "data.size() = " << data.size() << endl; cout << "L[0][0].as<size_t>() = " << L[0][0].as<size_t>() << endl; const result R( T.exec("SELECT " + Field + " FROM " + Table) ); const binarystring roundtrip(R[0][0]); cout << "roundtrip.str() = " << roundtrip.str() << endl; cout << "data = " << data << endl; cout << "roundtrip.size() = " << roundtrip.size() << endl; cout << "data.size() = " << data.size() << endl; T.exec("CREATE TEMP TABLE tuple (one INTEGER, two VARCHAR)"); pqxx::prepare::declaration d( T.conn().prepare("maketuple", "INSERT INTO tuple VALUES ($1, $2)") ); d("INTEGER", pqxx::prepare::treat_direct); d("VARCHAR", pqxx::prepare::treat_string); pqxx::prepare::invocation i( T.prepared("maketuple") ); const string f = "frobnalicious"; i(6); i(f); i.exec(); const result t( T.exec("SELECT * FROM tuple") ); cout << "t.size()= " << t.size() << endl; cout << "sz = " << 1u << endl; cout << "t[0][0].as<string>() = " << t[0][0].as<string>() << endl; cout << "l = " << 6 << endl; cout << "t[0][1].c_str() = " << t[0][1].c_str() << endl; cout << "f = " << f << endl; return 0; }

After connection is successfully obtained, transaction T is constructed. Temp table is created and after confirmation that prepared statements are supported, testing goes on. We have prepare::declaration and prepare::invocation, available in reference generated with doxygen. Adding parameters is in iterative fashion and feels natural, as they say in documentation like varargs from C. Library is well designed and easy to use, documentation, tutorial and tests are supplied. Lengths and contents should match and test succeeds.

SQLite C API another convenience routine example

Last time we presented sqlite3_exec example. Beside sqlite3_exec we can use wrapper around  sqlite3_exec and that is sqlite3_get_table. We will get, if call is successful, array of UTF-8 zero terminated strings and we have to free that array at the end. Here is interface for sqlite3_get_table and sqlite3_free_table:

int sqlite3_get_table( sqlite3 *db, /* An open database */ const char *zSql, /* SQL to be evaluated */ char ***pazResult, /* Results of the query */ int *pnRow, /* Number of result rows written here */ int *pnColumn, /* Number of result columns written here */ char **pzErrmsg /* Error msg written here */ ); void sqlite3_free_table(char **result);

It is quite self-explanatory. Number of rows doesn't count column names and we need to add one to it. Cleanup is required in the any case, if all is ОК we need to call sqlite3_free_table, if there is failure we need to do cleanup of error message, like with  sqlite3_exec. We will retrieve content of that jane table from sqlite3_exec example, then we inserted three rows into table. Here is the code:

#include <sqlite3.h> #include <stdio.h> int main(void) { sqlite3 *db; char *err_msg = NULL, *sql = NULL, **pazResult = NULL; int row, col,i,j; int rc = sqlite3_open_v2("test.db", &db, SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, NULL); if (rc != SQLITE_OK) { printf("Failure to open database: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } sql = "select * from jane;"; rc = sqlite3_get_table(db, sql, &pazResult, &row, &col, &err_msg); if (rc != SQLITE_OK ) { printf("Get table failure: %s\n", err_msg); sqlite3_free(err_msg); sqlite3_close(db); return 1; } for(i=0;i<row+1;i++){ for(j=0;j<col;j++) printf("%s\t",*(pazResult+i*col+j)); printf("\n"); } sqlite3_free_table(pazResult); sqlite3_close(db); return 0; }

Very simple and very user friendly API. If we build and execute binary, we should get error:

Get table failure: no such table: jane

Our table is in surogat.db, we repair example, recompile and we should see table printout:

id title full_name 1 Mr Smith 2 Mrs Doe 3 Mr Doe

That is legacy interface and usage is not recommended, though it is very user friendly and that is the reason I supplied example.
As usual, example is built on Linux using gcc and successfully tested. Didn't try on different OS or compiler.

Tests, pointers, arrays and GDB

While I was looking for work, actually I am still looking for work, they sent me to do some tests. Those are some “tech check” rubbish tests which are testing how much of man pages you know by hart. Not do you have logic of programmer and real working knowledge but how well have you memorized help files. So let me explain how you are going to deal with those test and real life problems in sensible way. While agile approach is very desirable in project management, memorizing help files is what industry expects from programmers. Everything further happens on Linux and we will do some debugging to find out answers.
About every book teaching C contains story how one can declare array and access array elements via pointer arithmetic. Something like this:

int arr1d[] = {1, 2, 3, 4, 5}, i; for(i=0;i<5;i++) printf("i = %d; %d = %d = %d\n", i, *arr1d+i, *(arr1d+i), arr1d[i]);

Expression *arr1d+i is not really pointer arithmetic since dereferencing will happen before addition, and everybody who worked in C longer than two weeks knows it, but it will also produce desired result. I also omitted array length and gcc managed to read it from initializer. Now we can declare some pointers and assign address of our array to them.

int *p1d = arr1d; for(i=0;i<5;i++) printf("i = %d; %d = %d\n", i, *(p1d+i), p1d[i]);

Array is just pointer to its first element, we got type and everything right. If we now take address of array we will have double pointer? Not really.

int **p1d12 = &arr1d;

Produces this warning:

warning: initialization from incompatible pointer type [enabled by default]

Since we do not know what is wrong, what type for pointer to array we are getting instead double pointer to integer, we will ask GDB. This is the code:

#include <stdio.h> int main(){ int arr1d[] = {1, 2, 3, 4, 5}, i; for(i=0;i<5;i++) printf("i = %d; %d = %d = %d\n", i, *arr1d+i, *(arr1d+i), arr1d[i]); int *p1d = arr1d; for(i=0;i<5;i++) printf("i = %d; %d = %d\n", i, *(p1d+i), p1d[i]); int **p1d12 = &arr1d; /* not really */ int *p1d11 = &arr1d[0]; return 0; }

and we will save it as untitled.c and build using

gcc -g -Wall -o untitled untitled.c
untitled.c: In function ‘main’:
untitled.c:12:16: warning: initialization from incompatible pointer type [enabled by default]
untitled.c:13:7: warning: unused variable ‘p1d11’ [-Wunused-variable]
untitled.c:12:8: warning: unused variable ‘p1d12’ [-Wunused-variable]

This is together with output. Switch -g means that we want debugg info and -Wall that we want all warnings. Now we start interactive session and ask GDB what we want to know:

gdb untitled GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04 Copyright (C) 2012 Free Software Foundation, Inc. … more advertising stuff … Reading symbols from ~/workshop/lrhw/untitled...done. (gdb) start Temporary breakpoint 1 at 0x4004fc: file untitled.c, line 4. Starting program: ~/workshop/lrhw/untitled Temporary breakpoint 1, main () at untitled.c:4 4 int arr1d[] = {1, 2, 3, 4, 5}, i; (gdb) info locals arr1d = {-6136, 32767, 4195877, 0, 1} i = 0 p1d = 0x0 p1d12 = 0x400410 p1d11 = 0x7fffffffe7f0 (gdb) next 5 for(i=0;i<5;i++) (gdb) info locals arr1d = {1, 2, 3, 4, 5} i = 0 p1d = 0x0 p1d12 = 0x400410 p1d11 = 0x7fffffffe7f0 (gdb) print arr1d $1 = {1, 2, 3, 4, 5} (gdb) print &arr1d $2 = (int (*)[5]) 0x7fffffffe6d0

It printed few lines of messages about license, where to report bugs and similar and loaded symbols for untitled. On prompt (gdb) we type in start and it starts and breaks on the first possible line. We try info locals and see that array is not initialized yet, so we execute next. Now array is initialized and we print it. Finally we ask it to print &arr1d and we learn what is the type of our “double pointer”.

int (*p1d12)[] = &arr1d;

This is what address of array returns and how “double pointer” should be declared, really ugly question on some idiotic test.
Things are becoming more interesting with multidimensional arrays. For example:

int arr2d[][4] = {{1, 2, 3, 4}, {5, 6, 7, 8}};

We can not omit everything even if we are supplying initializer, just the first square bracket may be empty. How about asking some questions? Start GDB session and ask all what you need to know:

(gdb) print &arr2d $1 = (int (*)[2][4]) 0x7fffffffe630 (gdb) print &arr2d[0] $2 = (int (*)[4]) 0x7fffffffe630 (gdb) print &arr2d[1] $3 = (int (*)[4]) 0x7fffffffe640 (gdb) print *arr2d $4 = {1, 2, 3, 4} (gdb) x 0x7fffffffe640 0x7fffffffe640: 0x00000005 (gdb) print **arr2d $5 = 1 (gdb) print **arr2d+1 $6 = 2 (gdb) print **(arr2d+1) $7 = 5 (gdb) print *(*arr2d+1) $8 = 2

Simple as that. There is one more question left, what will happen with double pointer initialized to address of array, why it not working? Again we are agile and write code:

int arr2d[][4] = {{1, 2, 3, 4}, {5, 6, 7, 8}}; int **p = arr2d; printf("*p = %d \n", *p);

That will execute and print *p = 1?! Start GDB session and check what is happening:

(gdb) p **p Cannot access memory at address 0x200000001 (gdb) p *p $1 = (int *) 0x200000001 (gdb) p sizeof(*p) $2 = 8 (gdb) p p $3 = (int **) 0x7fffffffe6e0 (gdb) x 0x7fffffffe6e0 0x7fffffffe6e0: 0x00000001 (gdb) x 0x7fffffffe6e4 0x7fffffffe6e4: 0x00000002 (gdb) x 0x7fffffffe6e8 0x7fffffffe6e8: 0x00000003 (gdb) p **arr2d $4 = 1 (gdb) p *arr2d $5 = {1, 2, 3, 4} (gdb) p sizeof(*arr2d) $6 = 16

Abbreviated print is p and x will print content of memory at some address. Array is not just pointer, there is size of it what counts. If we have used ld format in printf, we would see slightly bigger output than just one ;-)
That would be such lovely question for test, what would be output if we replace %d with %ld? Naturally it will be *p = 8589934593!
Ask yourself stupid questions for fun and for profit.

More SQLite C API

In previous instalment we used sqlite3_open and sqlite3_prepare to connect to db file and execute select statement. Let us take a better look at it again. Those functions are legacy interface, we could use new sqlite3_open_v2 and sqlite3_prepare_v2 which are allowing us more control over execution and they are also recommended by SQLite development team. Let us look at sqlite3_open. This is from SQLite documentation:

int sqlite3_open( const char *filename, /* Database filename (UTF-8) */ sqlite3 **ppDb /* OUT: SQLite db handle */ ); int sqlite3_open16( const void *filename, /* Database filename (UTF-16) */ sqlite3 **ppDb /* OUT: SQLite db handle */ ); int sqlite3_open_v2( const char *filename, /* Database filename (UTF-8) */ sqlite3 **ppDb, /* OUT: SQLite db handle */ int flags, /* Flags */ const char *zVfs /* Name of VFS module to use */ );

In all three cases, the first argument is name of database file to be opened and the second one is handle:

typedef struct sqlite3 sqlite3;

We will need that handle to manipulate db and close it at the end. From comment we see that string with name of db file could be UTF-8 or UTF-16 encoded. What are flags? Here are definitions:

#define SQLITE_OPEN_READONLY 0x00000001 /* Ok for sqlite3_open_v2() */ #define SQLITE_OPEN_READWRITE 0x00000002 /* Ok for sqlite3_open_v2() */ #define SQLITE_OPEN_CREATE 0x00000004 /* Ok for sqlite3_open_v2() */ #define SQLITE_OPEN_URI 0x00000040 /* Ok for sqlite3_open_v2() */ #define SQLITE_OPEN_NOMUTEX 0x00008000 /* Ok for sqlite3_open_v2() */ #define SQLITE_OPEN_FULLMUTEX 0x00010000 /* Ok for sqlite3_open_v2() */ #define SQLITE_OPEN_SHAREDCACHE 0x00020000 /* Ok for sqlite3_open_v2() */ #define SQLITE_OPEN_PRIVATECACHE 0x00040000 /* Ok for sqlite3_open_v2() */

I will wisely skip last four, for now. If we specify SQLITE_OPEN_READWRITE we should be able to read and write, unless OS have marked file as read only, file must exist or error is returned. In order to simulate behavior of sqlite3_open we should use  SQLITE_OPEN_READWRITE |  SQLITE_OPEN_CREATE. The fourth argument of sqlite3_open_v2 function is name of the sqlit3_vfs object that defines the operating system interface that the new database connection should use. If we pass NULL as the fourth argument we get default sqlit3_vfs object.
If  sqlite3_open_v2 doesn't return SQLITE_OK, we have error and we should exit. We still may have proper db handle or NULL handle, calling sqlite3_close with NULL pointer is not problem.
So, we have something like this as template:

#include <sqlite3.h> #include <stdio.h> int main(int argc, char **argv) { char *file = "some.db"; sqlite3 *db = NULL; int rc = 0; rc = sqlite3_open_v2( file, &db, SQLITE_OPEN_READWRITE, NULL ); if ( rc != SQLITE_OK) { printf("Failure to open database: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } /* do something with connection */ sqlite3_close(db); return 0; }

Call to sqlite3_errmsg should return string with description of message. We want to do resource management and to match  sqlite3_open with sqlite3_close. If we compile this and we do not have some.db file, after execution we should see descriptive error message.

./test
Failure to open database: unable to open database file

While passing to sqlite3_close NULL pointer is not problem, passing to it handle of already closed connection is problem.
Situation with sqlite3_prepare_v2 is similar to  sqlite3_open_v2, new API is preferable to legacy one and it should be used. Here both new and legacy functions are accepting the same number of arguments but behavior is different. Please check in documentation how they differ.

int sqlite3_prepare_v2( sqlite3 *db, /* Database handle */ const char *zSql, /* SQL statement, UTF-8 encoded */ int nByte, /* Maximum length of zSql in bytes. */ sqlite3_stmt **ppStmt, /* OUT: Statement handle */ const char **pzTail /* OUT: Pointer to unused portion of zSql */ );

There is also UTF-16 version with about the same arguments. It prepares statement, compiles it to bytecode  and if successful returns pointer to compiled  sqlite3_stmt.

sql = "select * from sqlite_master"; rc = sqlite3_prepare_v2(db, sql, strlen(sql), &stmt, 0); if(rc != SQLITE_OK) { printf("SQL error: %s\n", sqlite3_errmsg(db)); return 1; } rc = sqlite3_step(stmt); col = sqlite3_column_count(stmt); for(i=0; i < col; i++) { printf("%s\t",sqlite3_column_name(stmt, i)); } printf("\n------------------------------------------------------\n"); while(rc == SQLITE_ROW) { for(i=0; i < col; i++) { printf("%s\t", sqlite3_column_text(stmt, i)); } printf("\n"); rc = sqlite3_step(stmt); } sqlite3_finalize(stmt);

After statement is successfully prepared we iterate using sqlite3_step and at the end we finalize statement. In the case of sqlite3_prepare  returning error, we in general want to check is statement handle points to something and do cleanup if it does.
There is convenience wrapper around  prepare_v2-step-finalize, it is sqlite3_exec.

int sqlite3_exec( sqlite3*, /* An open database */ const char *sql, /* SQL to be evaluated */ int (*callback)(void*,int,char**,char**), /* Callback function */ void *, /* 1st argument to callback */ char **errmsg /* Error msg written here */ );

If we are doing insert, we do not really need callback, so we pass NULL pointer as the third and fourth argument.

#include <sqlite3.h> #include <stdio.h> int main(void) { sqlite3 *db; char *err_msg = NULL, *sql = NULL; int rc = sqlite3_open_v2("test.db", &db, SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, NULL); if (rc != SQLITE_OK) { printf("Failure to open database: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } sql = "INSERT INTO jane VALUES(1,'Mr','Smith');" "INSERT INTO jane VALUES(2,'Mrs','Doe');" "INSERT INTO jane VALUES(3,'Mr','Doe');"; rc = sqlite3_exec(db, sql, NULL, NULL, &err_msg); if (rc != SQLITE_OK ) { printf("Exec failure: %s\n", err_msg); sqlite3_free(err_msg); sqlite3_close(db); return 1; } sqlite3_close(db); return 0; }

If we don't have table jane in test.db exec will return error.

Exec failure: no such table: jane

or if we execute it more than once:

Exec failure: PRIMARY KEY must be unique

Error message is allocated using sqlite3_malloc and we have to free it using sqlite3_free.

SQLite C API introduction

I guess you are already using some kind of Linux if you are reading this. Install SQLite libraries and dev libraries, on my box they are called libsqlite3-0 and libsqlite3-dev. On your Linux packages may be called slightly differently. For example if your distro is Debian based you will check for packages executing in terminal:

apt-cache search sqlite

You do not need elevated privileges to execute that.
Also install sqlite3 command line interface for SQLite 3.
SQLite is embedded relational database engine. It was popular before but then Android made it part of OS and now it is like totally popular. There is no server and consequently no SQLite database administrator.
In order to build examples we need to specify linker switch, like this:

gcc testme.c -lsqlite3 -o test

Everything should be in path. Also we need to include appropriate header file in example:

#include <sqlite3.h> #include <stdio.h> int main(int argc, char **argv) { int rc, i, col; sqlite3 *db; sqlite3_stmt *stmt; char *sql; rc = sqlite3_open("surogat.db", &db); if(rc != SQLITE_OK) { printf("Can't open database: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); exit(1); } sql = "select * from sqlite_master"; rc = sqlite3_prepare(db, sql, strlen(sql), &stmt, 0); if(rc != SQLITE_OK) { printf("SQL error: %s\n", sqlite3_errmsg(db)); } rc = sqlite3_step(stmt); col = sqlite3_column_count(stmt); for(i=0; i < col; i++) { printf("%s\t",sqlite3_column_name(stmt, i)); } printf("\n------------------------------------------------------\n"); while(rc == SQLITE_ROW) { for(i=0; i < col; i++) { printf("%s\t", sqlite3_column_text(stmt, i)); } printf("\n"); rc = sqlite3_step(stmt); } sqlite3_finalize(stmt); sqlite3_close(db); return 0; }

We open database file, if it doesn't exist it will create it, if it can't we report error and exit. Next, we prepare statement, using legacy interface, and again check for error. Function sqlite3_step evaluates prepared statement, similarly we retrieve some metadata and print that to standard output.
Before we run example we will create table so that we can show some kind of output from our program. All that in the same folder where is binary, so that we don't have to fiddle with full path.

sqlite3 surogat.db SQLite version 3.7.9 2011-11-01 00:52:41 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> CREATE TABLE jane ( id INTEGER PRIMARY KEY NOT NULL, title TEXT, full_name TEXT NOT NULL ); sqlite> .exit

Now we can run our test program and see output like this.

type    name    tbl_name    rootpage    sql   
------------------------------------------------------
table    jane    jane    2    CREATE TABLE jane ( id INTEGER PRIMARY KEY NOT NULL, title TEXT, full_name TEXT NOT NULL )   

That was simple connect to database and select, there will be more in next instalment.

PostgreSQL C++ tutorial

Installation and configuration

This tutorial is done on LinuxMint 12 and it will work on Ubuntu 11.10. I tested it tonight on Linux Mint 13 Maya and can confirm that it still works here, that is Ubuntu Precise based Linux Mint.
Using terminal we find what is available:

apt-cache search postgresql

those are results we are interested in:

libpqxx-3.0 - C++ library to connect to PostgreSQL
libpqxx-3.0-dbg - C++ library to connect to PostgreSQL (debugging symbols)
libpqxx3-dev - C++ library to connect to PostgreSQL (development files)
libpqxx3-doc - C++ library to connect to PostgreSQL (documentation)
postgresql-9.1 - object-relational SQL database, version 9.1 server
postgresql-client - front-end programs for PostgreSQL (supported version)
postgresql-client-9.1 - front-end programs for PostgreSQL 9.1
postgresql-client-common - manager for multiple PostgreSQL client versions
postgresql-common - PostgreSQL database-cluster manager
postgresql-contrib - additional facilities for PostgreSQL (supported version)
postgresql-contrib-9.1 - additional facilities for PostgreSQL

It will return much more but we do not need them all. On Linux Mint 13 we have slightly fesher libs:

libpqxx-3.1 - C++ library to connect to PostgreSQL
libpqxx-3.1-dbg - C++ library to connect to PostgreSQL (debugging symbols)

Now in terminal we do

sudo apt-get install postgresql-9.1 postgresql-client postgresql-client-9.1 postgresql-client-common postgresql-common postgresql-contrib postgresql-contrib-9.1

or if one like gui, Software Manager or Synaptic will also do. Do not forget contrib packages, you will need them for pgAdmin III.
Again in terminal do:

sudo su postgres

afer entering password you are postgres. As postgres:

psql template1
psql (9.1.3)
Type "help" for help.
template1=# create role testuser login password 'testpass' superuser valid until 'infinity';
CREATE ROLE
template1=#\q

That escaped q quits psql and after one exit you are back to your login. If you like install now pgAdmin III or using psql create DB and table where you are going to practice.
To allow remote connections do:

sudo gedit /etc/postgresql/9.1/main/postgresql.conf

and modify listen_addresses, something like this:

listen_addresses = 'localhost, 192.168.0.42, 192.168.0.111'

Also in pg_hba.conf we need to enable remote users:

sudo gedit /etc/postgresql/9.1/main/pg_hba.conf

it should look something like this, all the way towards bottom of the file:

# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
host    template1       testuser        192.168.0.0/24           md5
host    testpgdb        testuser        192.168.0.0/24           md5

There is number of commented lines in file but I ignored them to save space. After saving changes restart PostgreSQL server:

sudo /etc/init.d/postgresql restart

Please create DB testpgdb with sufficient rights for testuser or rename DB in C++ example.
Now it is time to install libpqxx. From terminal execute:

sudo apt-get install libpqxx-3.0 libpqxx-3.0-dbg libpqxx3-dev libpqxx3-doc

and installation is done. As already mentioned on newer versions of Linux it may be newer version of libpxx, so please adjust accordingly.

C++ example

Code is slightly adjusted test 005 which comes with libpqxx3-doc, to see where is what use:

dpkg -L libpqxx3-doc

It connects to local instance of PostgreSQL, if you want remote – please edit connection string. If connection succeeds creates table, inserts data and at the end does one non-transactional select.

#include <iostream> #include <pqxx/pqxx> using namespace std; using namespace pqxx; /* * */ int main(int argc, char** argv) { connection C("dbname=testpgdb user=testuser password=testpass hostaddr=127.0.0.1 port=5432"); string tableName("tabletwo"); if (C.is_open()) { cout << "We are connected to" << C.dbname() << endl; } else { cout << "We are not connected!" << endl; return 0; } work Q(C); try { Q.exec("DROP TABLE " + tableName); Q.commit(); } catch (const sql_error &) { } work T(C); T.exec("CREATE TABLE "+tableName+" (id integer NOT NULL, name character varying(32) NOT NULL, salary integer DEFAULT 0);"); tablewriter W(T, tableName); string load[][3] = { {"1","John","0"}, {"2","Jane","1"}, {"3","Rosa","2"}, {"4","Danica","3"} }; for (int i=0;i< 4; ++i) W.insert(&load[i][0], &load[i][3]); W.complete(); T.exec("ALTER TABLE ONLY "+tableName+" ADD CONSTRAINT \"PK_IDT\" PRIMARY KEY (id);"); T.commit(); nontransaction N(C); result R(N.exec("select * from "+tableName)); if (!R.empty()) { for (result::const_iterator c = R.begin(); c != R.end(); ++c) { cout << '\t' << c[0].as(string()) << '\t' << c[1].as(string()) <<'\t' << c[2].as(string()) <<endl; } } return 0; }

In order to compile code you will need to tell to g++ where are libpqxx headers (they are not on the path) and also to linker what libs must be used for linking. Something like this:

g++ hello.cxx -o hello -I/usr/local/include/ -lpqxx -lpq

If your libpqxx or libpq are on unusual place you will use -L[path to where they are], there is lot of that on CentOS or Fedora ;-)
 After executing hello (./hello) you should be rewarded with the following output:

We are connected totestpgdb
NOTICE:  ALTER TABLE / ADD PRIMARY KEY will create implicit index "PK_IDT" for table "tabletwo"
        1       John    0
        2       Jane    1
        3       Rosa    2
        4       Danica  3

I can write more on subject if there is interest in it, will see from number of hits.

Few filters from ffmpeg filter library

If you are using WinFF with Display Cmd Line option, then from DVD preset you already know how scaling look like. For example

-vf scale=720:480

That would be width and height of output. With Additional Options activated WinFF will allow you to specify crop values. Unfortunately it will place them in wrong order if you are doing crop and scale. Placing crop in front of scale is preferable.

-vf crop=iw-50-10:ih-20-20:50:20,scale=720:480

Crop parameters are width, height and start point. It calculates it from input with, height and parameters which you have entered. BTW if you need to crop and scale inside complex filter that looks like this:

ffmpeg -i input.mpg -i detail.mpg -filter_complex\
 "[0:v] setpts=PTS-STARTPTS, crop=iw-6-6:ih-0-0:6:0,scale=800x640 [big];\
 [1:v] setpts=PTS-STARTPTS, crop=iw-50-10:ih-20-20:50:20,scale=240x192 [detail];\
 [big][detail] overlay=shortest=1:x=560" -c:v libx264 out.mkv

Filter expressions are separated with coma as in non complex case.
Beside size and crop, commonly used filter is unsharp. With unsharp filter output may be sharper or softer, blured. It accepts six parameters and those are default values:

-vf unsharp=5:5:1.0:5:5:0.0

Parameters are, in order of aperance, luma matrix size, horizontal and vertical, odd integers between 3 and 63. Next we have luma effect strength which is float. Then we have the same but for chroma. Positive values for effect strength will sharpen image, negative will blur it and zero will disable effect. For example scale and make it sharper:

-vf scale=960:540,unsharp=9:9:0.75:9:9:0.75

If we want to blur image we will make floats negative.
Another filter which we may want to use is curves. Those are the same ones which we know from GIMP, Curves Tool under Colors. While describing curves may be somewhat complicated, there is number of presets available and they are:

none
color_negative
cross_process
darker
increase_contrast
lighter
linear_contrast
medium_contrast
negative
strong_contrast
vintage

We can even chain them if required. So, if we want lighter output image:

-vf scale=960:540,curves=lighter

More ambitious users will take a look at man pages and maybe decide to create their own presets. According to documentation ffmpeg can import curves from photoshop but I can not confirm that since I am not using it.

Join two videos using ffmpeg and fade filter

Imagine that you retrieved your video from your camera, tablet, mobile phone, Internet and converted it to mpeg. After resizing and encoding of 43.8 Mbps 1920x1080 input video produced by Canon camera we can end up with some more modest resolution. Using Avidemux we can precisely find and cut segments from one keyframe to another. You can append those segments like this:

ffmpeg -i sa0.mpg -i sa1.mpg -filter_complex \
 "[0:0] [0:1] [1:0] [1:1] concat=n=2:v=1:a=1 [v] [a]" \
 -map '[v]' -map '[a]' -c:v mpeg2video -q:v 2 out.mpg

We want video quality 2, we want mpeg video codecs, and we want both video and audio to be in output. Audio will be default mp2 to match video, since we didn't specify any. ffmpeg will print to standard output how it “understands” our instructions:

Output #0, mpeg, to 'out.mpg':
  Metadata:
    encoder         : Lavf55.3.100
    Stream #0:0: Video: mpeg2video, yuv420p, 960x540 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 90k tbn, 29.97 tbc
    Stream #0:1: Audio: mp2, 44100 Hz, stereo, s16, 128 kb/s

Whatever we didn't specify ffmpeg will or pick up from input video or adjust to match output container. I assume here that both videos are from the same original video and for that reason I didn't resize or resample videos, if that is required one can do it as preprocessing. That will append our two videos but it will be rather abrupt and not very smooth event. Luckily ffmpeg got rich library of filters and we can fade out and in to make smooth transition between two segments. We can fade just video or we can fade both, video and audio.

ffmpeg -i sa0.mpg\
 -vf fade=out:8928:36 -af afade=t=out:st=297.5:d=1.5\
 -c:v mpeg2video -q:v 2 output1.mpg

This is the first segment and we want to fade out last 36 frames and also last 1,5 seconds of audio. Video fade parameters are expressed in frames, start at frame and fade duration in frames, audio afade is about the same but in seconds. If we do not want to fade audio track we will omit audio filter story. Now we want to fade in next segment:

ffmpeg -i sa1.mpg\
 -vf fade=in:0:36 -af afade=t=in:st=0:d=1.5\
 -vcodec mpeg2video -q:v 2 output2.mpg

Starting from zero we fade in 36 frames of video and 1,5 seconds of audio signal. After we join those two segments we will have smooth fade out and in transition:

ffmpeg -i output1.mpg -i output2.mpg -filter_complex \
 "[0:0] [0:1] [1:0] [1:1] concat=n=2:v=1:a=1 [v] [a]" \
 -map '[v]' -map '[a]' -c:v mpeg2video -q:v 2 out.mpg

The same as we did without fading.

Encoding with ffmpeg

Join two videos

Unless one have production ambitions, encoding on Linux is rather simple and user friendly. Install ffmpeg, WinFF and get presets for WinFF. Thanks to WinFF menu option Display Cmd Line one can see what arguments are passed to ffmpeg to achieve this or that. For example you want 30 seconds from one and 30 seconds from another place in video.

ffmpeg -i input.mp4 -filter_complex \
"[0:v]trim=500:530,setpts=PTS-STARTPTS[v0]; \
 [0:a]atrim=500:530,asetpts=PTS-STARTPTS[a0]; \
 [0:v]trim=900:930,setpts=PTS-STARTPTS[v1]; \
 [0:a]atrim=900:930,asetpts=PTS-STARTPTS[a1]; \
 [v0][a0][v1][a1]concat=n=2:v=1:a=1" \
 -c:v libx264 output.mkv

 Start at 500 seconds, use 30 seconds of video and audio and then from 900 seconds do the same. Concat output and encode using lib264. Filters trim and atrim will help you with ref frames, the first is video and the second one is audio version. Next we have setpts and asetpts, again video and audio version. We want PTS Presentation Time Stamp to start from zero. Finally we have concat filter where we specify two parts to be concatenated into one video and one audio. What are square brackets? Those at the start of line are saying take video or audio from input array at position zero. Since we have only one video all left tags are zero. Right tags are our friendly names to segments, we use them at the end to say how to concat segments to output.
If you see something like this, when you try concating two segments:

[AVFilterGraph @ 0x2e9be00] No such filter: 'trim'
Error configuring filters.

Then you download source, build and install ffmpeg. For more precise editing one may want to use Avidemux and for that video files must be converted to mpeg. WinFF have DVD settings which you are going to use. It will not specify audio conversion parameters, if you do not want default ones you just append this to what WinFF says:

-acodec mp2 -ar 44100 -ab 128k -ac 2

That is use mp2 codec, sampling 44.1KHz, bit rate 128Kbps and number of channels. With Avidemux you can do frame by frame, variable speed searching and so on.

Video in video

In existing video you want to insert some kind of detail in upper right corner.

ffmpeg -i input.mpg -i detail.mpg -filter_complex \
  "[0:v] setpts=PTS-STARTPTS, scale=800x640 [big];\
  [1:v] setpts=PTS-STARTPTS, scale=240x192 [detail];\
  [big][detail] overlay=shortest=1:x=560" -c:v libx264 out.mkv

Here we resize both videos, tag them with friendly names and for overlay specify where we want detail to start. Since we didn't specify y it will be zero. That shortest=1 means when end of shortest input is reached, stop encoding. Audio signal will be copied from the first video input.

Mosaic of several videos

This one is nicely explained at Create a mosaic out of several input videos at ffmpeg site. I will not bother you with explanation which you can read there but I am going to give you command to create 6x2 mosaic:

ffmpeg -i a.mpg -i b.mpg -i c.mpg -i d.mpg -i e.mpg -i f.mpg -filter_complex\
 "nullsrc=size=1920x960 [base];\
 [0:v] setpts=PTS-STARTPTS, scale=640x480 [upperleft];\
 [1:v] setpts=PTS-STARTPTS, scale=640x480 [uppermiddle];\
 [2:v] setpts=PTS-STARTPTS, scale=640x480 [upperright];\
 [3:v] setpts=PTS-STARTPTS, scale=640x480 [lowerleft];\
 [4:v] setpts=PTS-STARTPTS, scale=640x480 [lowermiddle];\
 [5:v] setpts=PTS-STARTPTS, scale=640x480 [lowerright];\
 [base][upperleft] overlay=shortest=1 [tmp1];\
 [tmp1][uppermiddle] overlay=shortest=1:x=640 [tmp2];
 [tmp2][upperright] overlay=shortest=1:x=1280 [tmp3];\
 [tmp3][lowerleft] overlay=shortest=1:y=480 [tmp4];\
 [tmp4][lowermiddle] overlay=shortest=1:x=640:y=480 [tmp5];\
 [tmp5][lowerright] overlay=shortest=1:x=1280:y=480"\
 -c:v libx264 output.mkv

You will need large monitor to enjoy your mosaic. Again you will have audio from the first video in input. If combining audio makes sense, I would rather extract audio and mix tracks in Audacity where levels can be adjusted. Now with example at ffmpeg wiki and this one I guess one can easily create different overlays like large video left and two half size ones on the right and similar.