OpenMP, file tree walk and map-reduce example

All happens on Linux and we will use GCC and built in OpenMP support and also thread local storage. We will see how easy is to spread workload across threads using OpenMP.
In man pages we find complete example for nftw or file tree walk. That example will print to standard output about everything what it finds while walking directories. Instead of having it printing everything I made it count files:

#define _XOPEN_SOURCE 500 #include <ftw.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdint.h> int counter = 0; int count_files(const char *fpath, const struct stat *sb, int tflag, struct FTW *ftwbuf) { if(tflag == FTW_F){ counter++; } return 0; /* To tell nftw() to continue */ } int main(int argc, char *argv[]) { if (nftw((argc < 2) ? "." : argv[1], count_files, 20, 0) == -1) { perror("nftw"); exit(EXIT_FAILURE); } printf("There was %d files in %s\n", counter, (argc > 1)?argv[1]:"."); exit(EXIT_SUCCESS); }

To compile it I used:

gcc -O1 -o simpleftw simpleftw.c

In order to count files across multiple directories we will pass few directories in argument list to program and also turn our static variable counter into thread local storage. We will use pragma OpenMP parallel for directive combined with reduction which is in our case simple addition:

#define _XOPEN_SOURCE 500 #include <ftw.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdint.h> __thread int counter = 0; int count_files(const char *fpath, const struct stat *sb, int tflag, struct FTW *ftwbuf) { if(tflag == FTW_F){ counter++; } return 0; /* To tell nftw() to continue */ } int main(int argc, char *argv[]) { int total_fc = 0; #pragma omp parallel for reduction(+:total_fc) for (int i=1; i< argc; i++){ if (nftw((argc < 2) ? "." : argv[i], count_files, 20, 0) == -1) { perror("nftw"); exit(EXIT_FAILURE); } total_fc += counter; printf("Thread %d %d\n", i, counter); } printf("There was %d files in ",total_fc); for (int i=1; i< argc; i++) printf("%s ", argv[i]); printf("\n"); exit(EXIT_SUCCESS); }

That print inside for loop should demonstrate that thread local storage works as expected. C99 switch is required, or we will have to specify that variable i is private. Also switch for OpenMP is required. To compile it I used:

gcc -O1 -fopenmp -o dwalker dwalker.c -std=c99

Execution produces on my box:

./dwalker ~/Documents/ ~/Downloads/
Thread 1 3894
Thread 2 7784
There was 11678 files in ...

Everything works as expected.

C pointers, assembler and GDB

Idea here is to do some multitasking, find out how C pointer looks like in assembler and use Linux system calls, GCC and GDB. We start with Hello World:

int main() { char *c = "Hello World!\n"; write(1, c, 13); return 0; }

Execute man 2 write in terminal to find out more about write, 1 is standard output and 12 is number of bytes to be written. I saved it as hello.c and this is how it was compiled:

gcc -O0 -m64 -save-temps -o hlo hello.c

GCC will emit assembly code as temp, I like 64 bit version and no optimizations. Content of temp file is:

.file "hello.c" .section .rodata .LC0: .string "Hello World!\n" .text .globl main .type main, @function main: .LFB0: .cfi_startproc pushq %rbp .cfi_def_cfa_offset 16 .cfi_offset 6, -16 movq %rsp, %rbp .cfi_def_cfa_register 6 subq $16, %rsp movq $.LC0, -8(%rbp) movq -8(%rbp), %rax movl $13, %edx movq %rax, %rsi movl $1, %edi movl $0, %eax call write movl $0, %eax leave .cfi_def_cfa 7, 8 ret .cfi_endproc .LFE0: .size main, .-main .ident "GCC: (Debian 4.7.2-5) 4.7.2" .section .note.GNU-stack,"",@progbits

It pushes base pointer onto stack, there is q at the end of the push and bp is called rbp, so it must be 64 bit code. Loads string into memory, later into rax, prepares call to write and so on. Now we take that temp file and compile it with -g switch so that we can use GDB.

gcc -g -O0 -o hlo hello.s
gdb hlo

We type in start at prompt and here is the GDB session:

Temporary breakpoint 1, main () at hello.s:16 16 subq $16, %rsp (gdb) n 17 movq $.LC0, -8(%rbp) (gdb) i r rbp rbp 0x7fffffffe360 0x7fffffffe360 (gdb) x 0x7fffffffe358 0x7fffffffe358: 0x00000000 (gdb) n 18 movq -8(%rbp), %rax (gdb) x 0x7fffffffe358 0x7fffffffe358: 0x004005ec (gdb) x/13c 0x004005ec 0x4005ec: 72 'H' 101 'e' 108 'l' 108 'l' 111 'o' 32 ' ' 87 'W' 111 'o' 0x4005f4: 114 'r' 108 'l' 100 'd' 33 '!' 10 '\n'

Label .LCO is our string, -8(%rbp) is where pointer is. After info registers in short form, I also restricted info to rbp, I find out that rbp points to  0x7fffffffe360 and at  0x7fffffffe360 - 8 = 0x7fffffffe358 is our pointer. No, 0x60 - 8 is not 0x52. After we send next instruction and line 17 is executed we can see our string in memory, slash 13 c is we want 13 characters from that address.
Since write is Linux system call, we can use strace to see what is going on.

strace -tt ./hlo 08:13:04.407694 execve("./hlo", ["./hlo"], [/* 37 vars */]) = 0 ... 08:13:04.412892 write(1, "Hello World!\n", 13Hello World! ) = 13 08:13:04.413108 exit_group(0) = ?

After quite few lines were replaced with three consecutive dots we see that we managed to write all 13 characters to standard output.
Using system calls is so easy that one may even attempt writing to real file:

#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> int main() { char *c = "Hello World!\n"; int fd; fd = open("deleteme.blog", O_WRONLY | O_CREAT | O_TRUNC | O_APPEND, S_IRUSR | S_IWUSR); if (fd == -1){ write(1, "Open failed!", 12); return 1; } if(write(fd, c, 13)!=13) write(1, "Write failed!", 13); if(close(fd)==-1){ write(1, "Close failed!", 13); return 1; } return 0; }

We added more system calls. We applied common flags and modes in open, checked for all possible errors, after writing string to file we have closed file. If we compile it using

gcc -O0 -m64 -save-temps -o hello2 hello.c

We have binary and also assembly code. With help of strace

strace -tt ./hello2 09:24:49.906151 execve("./hello2", ["./hello2"], [/* 37 vars */]) = 0 ... 09:24:49.911601 open("deleteme.blog", O_WRONLY|O_CREAT|O_TRUNC|O_APPEND, 0600) = 3 09:24:49.911842 write(3, "Hello World!\n", 13) = 13 09:24:49.912018 close(3) = 0 09:24:49.912386 exit_group(0) = ?

Executing cat deleteme.blog we see that write really worked and strace didn't cheat. Examining assembly code will be interesting to those who are curious how if works on low level. Finally every project manager will take a note that from 08:13:04.407694 to 09:24:49.906151 I have written only 21 line of code.

Extending PostgreSQL with own C functions

Power of open source, if you are not happy with what PostgreSQL currently offers, you write own extension in C. Compile code with your functions into shared library, install it and they will become available from PostgreSQL. OK there are some rules and procedures. Once we are inside PostgreSQL we are using its types, interfaces and utilities. Let us do Hello World example. To build example you will need postgresql-server-dev-9.1 or whatever version you are using, installed on your Linux box.

#include "postgres.h" #include <string.h> #include "fmgr.h" #include "utils/builtins.h" #ifdef PG_MODULE_MAGIC PG_MODULE_MAGIC; #endif PG_FUNCTION_INFO_V1(say_hello_n); Datum say_hello_n(PG_FUNCTION_ARGS) { text *arg1 = cstring_to_text("Hello "); text *arg2 = PG_GETARG_TEXT_P(0); int32 new_text_size = VARSIZE(arg1) + VARSIZE(arg2) - VARHDRSZ; text *new_text = (text *) palloc(new_text_size); SET_VARSIZE(new_text, new_text_size); memcpy(VARDATA(new_text), VARDATA(arg1), VARSIZE(arg1) - VARHDRSZ); memcpy(VARDATA(new_text) + (VARSIZE(arg1) - VARHDRSZ), VARDATA(arg2), VARSIZE(arg2) - VARHDRSZ); PG_RETURN_TEXT_P(new_text); }

As we can see we are using new V1 spec. That is modified concat_text example from PostgreSQL documentation, look for 35.9.4. Version 1 Calling Conventions. We have load of different VAR macros. For example:

VARHDRSZ is the same as sizeof(int4), but it's considered good style to use the macro VARHDRSZ to refer to the size of the overhead for a variable-length type.

That is from PostgreSQL documentation. SET_VARSIZE we can find in includes postgres.h

#define SET_VARSIZE_4B(PTR,len) \ (((varattrib_4b *) (PTR))->va_4byte.va_header = (((uint32) (len)) << 2))

Unless you are on big endian. Going through header file one also can read in comments more about Datum and varlena datatypes. Then we got palloc which corresponds to malloc, memcpy you already know and GET and RETURN macros. It is obvious that for writing extensions one needs to familiarize himself with PostgreSQL internals. Power without knowledge and responsibility exists only in fery tails told by “software evangelists” at annual developers developers developers meetings.
Variables passed around by PostgreSQL may be on the disc, do not change them.
To build shared library I used the following Makefile:

MODULES = say_hello PG_CONFIG = pg_config PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS)

Rather long story to get location of pgxs. That pgxs is location of makefiles for building extensions. It is not trivial build and using provided mk files is right way to do it. After that we can copy say_hello.so to some reasonable location or give full path to it in create function declaration.

CREATE FUNCTION say_hello(text) RETURNS text AS '/opt/pg/ext/say_hello', 'say_hello_n' LANGUAGE C STRICT; ------- select say_hello('Filip');

PostgreSQL already allows Python through untrusted language PL/Python. One can utilize power of Python for functions or triggers without learning much about PostgreSQL internals. But again if you need power and speed, you can use what PostgreSQL speaks internally and that is C.

SQLite parameterized query in C

Still very angry at Discovery Health but that is not reason to stop using C. To execute parameterized query we prepare SQL statement with one or more placeholders. Placeholder could be question mark, alone or followed by number, column, dollar sign or at sign followed by alphanumeric. For example:

select title, full_name from jane where id = @id;

We prepare such statement using sqlite3_prepare_v2, later we bind our parameter and finally execute query. To do binding we will use appropriate function, there is few of them:

int sqlite3_bind_blob(sqlite3_stmt*, int, const void*, int n, void(*)(void*)); int sqlite3_bind_double(sqlite3_stmt*, int, double); int sqlite3_bind_int(sqlite3_stmt*, int, int); int sqlite3_bind_int64(sqlite3_stmt*, int, sqlite3_int64); int sqlite3_bind_null(sqlite3_stmt*, int); int sqlite3_bind_text(sqlite3_stmt*, int, const char*, int n, void(*)(void*)); int sqlite3_bind_text16(sqlite3_stmt*, int, const void*, int, void(*)(void*)); int sqlite3_bind_value(sqlite3_stmt*, int, const sqlite3_value*); int sqlite3_bind_zeroblob(sqlite3_stmt*, int, int n);

All bind functions will return SQLITE_OK if binding is successful or error code if it fails. The first argument is handle to prepared statement. The second argument is index of parameter. The third argument is value to be set. For blob, text we have the fourth argument, size in bytes and the fifth – destructor function. Instead of destructor function we can pass constants SQLITE_STATIC - do nothing or SQLITE_TRANSIENT – make local copy and free it when done. To find out what is index of parameter we are using this function:

int sqlite3_bind_parameter_index(sqlite3_stmt*, const char *zName);

We pass prepared statement and parameter name it returns zero if there is no such parameter or parameter index if it exists. Even if we know that our parameter must have index one, we will still look for it to demonstrate how it is done. Here is the code:

#include <sqlite3.h> #include <stdio.h> #include <string.h> int main(int argc, char **argv) { int rc, i, v, col; sqlite3 *db; sqlite3_stmt *stmt; char *sql; rc = sqlite3_open_v2("surogat.db", &db, SQLITE_OPEN_READWRITE, NULL); if(rc != SQLITE_OK) { printf("Can't open database: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } sql = "select title, full_name from jane where id = @id"; rc = sqlite3_prepare_v2(db, sql, strlen(sql), &stmt, 0); if(rc != SQLITE_OK) { printf("SQL error: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } i = sqlite3_bind_parameter_index(stmt, "@id"); v = 3; rc = sqlite3_bind_int(stmt, i, v); if(rc != SQLITE_OK) { printf("SQL error: %s\n", sqlite3_errmsg(db)); sqlite3_finalize(stmt); sqlite3_close(db); return 1; } rc = sqlite3_step(stmt); col = sqlite3_column_count(stmt); for(i=0; i < col; i++) { printf("%s\t",sqlite3_column_name(stmt, i)); } printf("\n-----------------\n"); while(rc == SQLITE_ROW) { for(i=0; i < col; i++) { printf("%s\t", sqlite3_column_text(stmt, i)); } printf("\n"); rc = sqlite3_step(stmt); } sqlite3_finalize(stmt); sqlite3_close(db); return 0; }

Database should be loaded with required values in previous examples, if not here is sql to create it:

CREATE TABLE jane ( id INTEGER PRIMARY KEY NOT NULL, title TEXT, full_name TEXT NOT NULL )
INSERT INTO jane VALUES(1,'Mr','Smith');
INSERT INTO jane VALUES(2,'Mrs','Doe');
INSERT INTO jane VALUES(3,'Mr','Doe');

After we build it using:

gcc -g test.c -lsqlite3 -o test

We execute test and see the following output:

title full_name ----------------- Mr Doe

We could also misspell parameter name and rebuild to check is error handling working.

PostgreSQL, libpqxx and prepared statement example

As I promised, if people are finding interesting introductory article,  I will write more about PostgreSQL and libpqxx. But before I start with programming – rant.
I am still looking for work and finding none. Today went to Discovery Health, had very pleasant 45 minutes chat with their architect and BA and promise that we will talk again. Later guy from employment agency, who arranged meeting, calls and says that they do not want me since I do not have ANSI C in CV?! Like somebody from management decided to override decision of interviewers. If ANSI C in CV is precondition, why they wasted my time and invited me for interview? I wish them the same from their customers. BTW I am usually employed as senior developer and not as C developer or, Perl developer.
Back to programming, this time C++, had enough of ANSI C for today. Environment is Linux, I am using the same PostgreSQL and libpqxx as last time and to compile example we will use:

g++ hello_prep.cxx -o hello_prep -I/usr/local/include/ -lpqxx -lpq

This time I used test092.cxx, run dpkg with -L switch on libpqxx3-doc to see where is it. It tests passing binary parameter to prepared statement. Test macros are replaced with printing of tested values to standard output and setup of connection/transaction is included. Here is the code:

#include <iostream> #include <pqxx/pqxx> using namespace std; using namespace pqxx; int main(int argc, char** argv) { connection C("dbname=testpgdb user=testuser password=testpass hostaddr=127.0.0.1 port=5432"); if (C.is_open()) { cout << "We are connected to" << C.dbname() << endl; } else { cout << "We are not connected!" << endl; return 0; } work T(C); const char databuf[] = "Test\0data"; const string data(databuf, sizeof(databuf)); cout << "data.size() = " << data.size() << endl; cout << "databuf size = " << sizeof(databuf) << endl; const string Table = "pqxxbin", Field = "binfield", Stat = "nully"; T.exec("CREATE TEMP TABLE " + Table + " (" + Field + " BYTEA)"); if (!T.conn().supports(connection_base::cap_prepared_statements)) { cout << "Backend version does not support prepared statements. Skipping." << endl; return 0; } T.conn().prepare(Stat, "INSERT INTO " + Table + " VALUES ($1)") ("BYTEA", pqxx::prepare::treat_binary); T.prepared(Stat)(data).exec(); const result L( T.exec("SELECT length("+Field+") FROM " + Table) ); cout << "data.size() = " << data.size() << endl; cout << "L[0][0].as<size_t>() = " << L[0][0].as<size_t>() << endl; const result R( T.exec("SELECT " + Field + " FROM " + Table) ); const binarystring roundtrip(R[0][0]); cout << "roundtrip.str() = " << roundtrip.str() << endl; cout << "data = " << data << endl; cout << "roundtrip.size() = " << roundtrip.size() << endl; cout << "data.size() = " << data.size() << endl; T.exec("CREATE TEMP TABLE tuple (one INTEGER, two VARCHAR)"); pqxx::prepare::declaration d( T.conn().prepare("maketuple", "INSERT INTO tuple VALUES ($1, $2)") ); d("INTEGER", pqxx::prepare::treat_direct); d("VARCHAR", pqxx::prepare::treat_string); pqxx::prepare::invocation i( T.prepared("maketuple") ); const string f = "frobnalicious"; i(6); i(f); i.exec(); const result t( T.exec("SELECT * FROM tuple") ); cout << "t.size()= " << t.size() << endl; cout << "sz = " << 1u << endl; cout << "t[0][0].as<string>() = " << t[0][0].as<string>() << endl; cout << "l = " << 6 << endl; cout << "t[0][1].c_str() = " << t[0][1].c_str() << endl; cout << "f = " << f << endl; return 0; }

After connection is successfully obtained, transaction T is constructed. Temp table is created and after confirmation that prepared statements are supported, testing goes on. We have prepare::declaration and prepare::invocation, available in reference generated with doxygen. Adding parameters is in iterative fashion and feels natural, as they say in documentation like varargs from C. Library is well designed and easy to use, documentation, tutorial and tests are supplied. Lengths and contents should match and test succeeds.

SQLite C API another convenience routine example

Last time we presented sqlite3_exec example. Beside sqlite3_exec we can use wrapper around  sqlite3_exec and that is sqlite3_get_table. We will get, if call is successful, array of UTF-8 zero terminated strings and we have to free that array at the end. Here is interface for sqlite3_get_table and sqlite3_free_table:

int sqlite3_get_table( sqlite3 *db, /* An open database */ const char *zSql, /* SQL to be evaluated */ char ***pazResult, /* Results of the query */ int *pnRow, /* Number of result rows written here */ int *pnColumn, /* Number of result columns written here */ char **pzErrmsg /* Error msg written here */ ); void sqlite3_free_table(char **result);

It is quite self-explanatory. Number of rows doesn't count column names and we need to add one to it. Cleanup is required in the any case, if all is ОК we need to call sqlite3_free_table, if there is failure we need to do cleanup of error message, like with  sqlite3_exec. We will retrieve content of that jane table from sqlite3_exec example, then we inserted three rows into table. Here is the code:

#include <sqlite3.h> #include <stdio.h> int main(void) { sqlite3 *db; char *err_msg = NULL, *sql = NULL, **pazResult = NULL; int row, col,i,j; int rc = sqlite3_open_v2("test.db", &db, SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, NULL); if (rc != SQLITE_OK) { printf("Failure to open database: %s\n", sqlite3_errmsg(db)); sqlite3_close(db); return 1; } sql = "select * from jane;"; rc = sqlite3_get_table(db, sql, &pazResult, &row, &col, &err_msg); if (rc != SQLITE_OK ) { printf("Get table failure: %s\n", err_msg); sqlite3_free(err_msg); sqlite3_close(db); return 1; } for(i=0;i<row+1;i++){ for(j=0;j<col;j++) printf("%s\t",*(pazResult+i*col+j)); printf("\n"); } sqlite3_free_table(pazResult); sqlite3_close(db); return 0; }

Very simple and very user friendly API. If we build and execute binary, we should get error:

Get table failure: no such table: jane

Our table is in surogat.db, we repair example, recompile and we should see table printout:

id title full_name 1 Mr Smith 2 Mrs Doe 3 Mr Doe

That is legacy interface and usage is not recommended, though it is very user friendly and that is the reason I supplied example.
As usual, example is built on Linux using gcc and successfully tested. Didn't try on different OS or compiler.

Tests, pointers, arrays and GDB

While I was looking for work, actually I am still looking for work, they sent me to do some tests. Those are some “tech check” rubbish tests which are testing how much of man pages you know by hart. Not do you have logic of programmer and real working knowledge but how well have you memorized help files. So let me explain how you are going to deal with those test and real life problems in sensible way. While agile approach is very desirable in project management, memorizing help files is what industry expects from programmers. Everything further happens on Linux and we will do some debugging to find out answers.
About every book teaching C contains story how one can declare array and access array elements via pointer arithmetic. Something like this:

int arr1d[] = {1, 2, 3, 4, 5}, i; for(i=0;i<5;i++) printf("i = %d; %d = %d = %d\n", i, *arr1d+i, *(arr1d+i), arr1d[i]);

Expression *arr1d+i is not really pointer arithmetic since dereferencing will happen before addition, and everybody who worked in C longer than two weeks knows it, but it will also produce desired result. I also omitted array length and gcc managed to read it from initializer. Now we can declare some pointers and assign address of our array to them.

int *p1d = arr1d; for(i=0;i<5;i++) printf("i = %d; %d = %d\n", i, *(p1d+i), p1d[i]);

Array is just pointer to its first element, we got type and everything right. If we now take address of array we will have double pointer? Not really.

int **p1d12 = &arr1d;

Produces this warning:

warning: initialization from incompatible pointer type [enabled by default]

Since we do not know what is wrong, what type for pointer to array we are getting instead double pointer to integer, we will ask GDB. This is the code:

#include <stdio.h> int main(){ int arr1d[] = {1, 2, 3, 4, 5}, i; for(i=0;i<5;i++) printf("i = %d; %d = %d = %d\n", i, *arr1d+i, *(arr1d+i), arr1d[i]); int *p1d = arr1d; for(i=0;i<5;i++) printf("i = %d; %d = %d\n", i, *(p1d+i), p1d[i]); int **p1d12 = &arr1d; /* not really */ int *p1d11 = &arr1d[0]; return 0; }

and we will save it as untitled.c and build using

gcc -g -Wall -o untitled untitled.c
untitled.c: In function ‘main’:
untitled.c:12:16: warning: initialization from incompatible pointer type [enabled by default]
untitled.c:13:7: warning: unused variable ‘p1d11’ [-Wunused-variable]
untitled.c:12:8: warning: unused variable ‘p1d12’ [-Wunused-variable]

This is together with output. Switch -g means that we want debugg info and -Wall that we want all warnings. Now we start interactive session and ask GDB what we want to know:

gdb untitled GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04 Copyright (C) 2012 Free Software Foundation, Inc. … more advertising stuff … Reading symbols from ~/workshop/lrhw/untitled...done. (gdb) start Temporary breakpoint 1 at 0x4004fc: file untitled.c, line 4. Starting program: ~/workshop/lrhw/untitled Temporary breakpoint 1, main () at untitled.c:4 4 int arr1d[] = {1, 2, 3, 4, 5}, i; (gdb) info locals arr1d = {-6136, 32767, 4195877, 0, 1} i = 0 p1d = 0x0 p1d12 = 0x400410 p1d11 = 0x7fffffffe7f0 (gdb) next 5 for(i=0;i<5;i++) (gdb) info locals arr1d = {1, 2, 3, 4, 5} i = 0 p1d = 0x0 p1d12 = 0x400410 p1d11 = 0x7fffffffe7f0 (gdb) print arr1d $1 = {1, 2, 3, 4, 5} (gdb) print &arr1d $2 = (int (*)[5]) 0x7fffffffe6d0

It printed few lines of messages about license, where to report bugs and similar and loaded symbols for untitled. On prompt (gdb) we type in start and it starts and breaks on the first possible line. We try info locals and see that array is not initialized yet, so we execute next. Now array is initialized and we print it. Finally we ask it to print &arr1d and we learn what is the type of our “double pointer”.

int (*p1d12)[] = &arr1d;

This is what address of array returns and how “double pointer” should be declared, really ugly question on some idiotic test.
Things are becoming more interesting with multidimensional arrays. For example:

int arr2d[][4] = {{1, 2, 3, 4}, {5, 6, 7, 8}};

We can not omit everything even if we are supplying initializer, just the first square bracket may be empty. How about asking some questions? Start GDB session and ask all what you need to know:

(gdb) print &arr2d $1 = (int (*)[2][4]) 0x7fffffffe630 (gdb) print &arr2d[0] $2 = (int (*)[4]) 0x7fffffffe630 (gdb) print &arr2d[1] $3 = (int (*)[4]) 0x7fffffffe640 (gdb) print *arr2d $4 = {1, 2, 3, 4} (gdb) x 0x7fffffffe640 0x7fffffffe640: 0x00000005 (gdb) print **arr2d $5 = 1 (gdb) print **arr2d+1 $6 = 2 (gdb) print **(arr2d+1) $7 = 5 (gdb) print *(*arr2d+1) $8 = 2

Simple as that. There is one more question left, what will happen with double pointer initialized to address of array, why it not working? Again we are agile and write code:

int arr2d[][4] = {{1, 2, 3, 4}, {5, 6, 7, 8}}; int **p = arr2d; printf("*p = %d \n", *p);

That will execute and print *p = 1?! Start GDB session and check what is happening:

(gdb) p **p Cannot access memory at address 0x200000001 (gdb) p *p $1 = (int *) 0x200000001 (gdb) p sizeof(*p) $2 = 8 (gdb) p p $3 = (int **) 0x7fffffffe6e0 (gdb) x 0x7fffffffe6e0 0x7fffffffe6e0: 0x00000001 (gdb) x 0x7fffffffe6e4 0x7fffffffe6e4: 0x00000002 (gdb) x 0x7fffffffe6e8 0x7fffffffe6e8: 0x00000003 (gdb) p **arr2d $4 = 1 (gdb) p *arr2d $5 = {1, 2, 3, 4} (gdb) p sizeof(*arr2d) $6 = 16

Abbreviated print is p and x will print content of memory at some address. Array is not just pointer, there is size of it what counts. If we have used ld format in printf, we would see slightly bigger output than just one ;-)
That would be such lovely question for test, what would be output if we replace %d with %ld? Naturally it will be *p = 8589934593!
Ask yourself stupid questions for fun and for profit.