About ffmpeg tutorial

If you want to work with ffmpeg from your code, you may want to build it from source. How to build it on your Linux distro, or possibly some other operating system, is described in Compilation Guide at ffmpeg wiki. For updated tutorial code one will also want to install git. I was working on LinuxMint, so commands used in this article are Debian-centric.
Latest ffmpeg tutorial is written by Dranger and it is severely outdated. If you visit his website you will see that his tutorial is remake of older tutorial written by Martin Bohme. Few programmers have made attempts to keep code for this tutorial in sync with current ffmpeg API. One can find updated code in git repository git@github.com:chelyaev/ffmpeg-tutorial.git. So, open terminal, cd where you want to clone repo and execute

git clone git@github.com:chelyaev/ffmpeg-tutorial.git
cd ffmpeg-tutorial
git log

Last line will print log and it will start from freshest commits, to advance towards oldest ones we press space. In order to see what changes have been made on tutorial01.c between initial import and working version we execute in terminal:

git log -u 3e1426115418ae50cb9a4357d57afbcec69342fe..69b7f32f5edb38cc4e9c47711f100998404485d1 tutorial01.c

Those hashes separated by double dot we got from previously executed git log and what set of changes is hash for we know from comment bellow hash. This one I used to see what was changed in tutorial01.c to make it work.
If we go through commits we will find in few comments that sound is not being decoded as expected, that is actually main reason why I am fiddling with this tutorials. You go through only available tutorial for ffmpeg and end up debugging sound problems, not very encouraging.

SDL

Original Dranger tutorial is called An ffmpeg and SDL Tutorial, so let us take a look at SDL. We will output our video to SDL and I prefer to familiarize with SDL before I use it.
If we have done what Compilation Guide for ffmpeg says, we already have SDL 1.2 installed. In order to find header files, man pages and similar we use:

dpkg -L libsdl1.2-dev

That will list installed files and help with includes for our hello world code.

#include <SDL/SDL.h> int main(int argc, char * argv[]) { SDL_Event event; int quit = 1; SDL_Surface* screen; if(SDL_Init(SDL_INIT_VIDEO)) { fprintf(stderr, "Could not initialize SDL - %s\n", SDL_GetError()); exit(1); } screen = SDL_SetVideoMode(640, 480, 0, SDL_SWSURFACE); if(!screen) { fprintf(stderr, "SDL: could not set video mode - exiting\n"); exit(1); } SDL_FillRect(screen,NULL,SDL_MapRGBA(screen->format,255,246,143,255)); SDL_UpdateRect(screen, 0, 0, 320, 480); while (quit) { SDL_WaitEvent(&event); switch(event.type) { case SDL_QUIT: SDL_Quit(); quit = 0; } } return 0; }

There is uninitialized SDL_Surface pointer, why I didn’t set it to NULL? Because SDL_SetVideoMode will set it to NULL if it fails. Path to include file is sorted out within code, linker instructions are added and we can build it and run it.

gcc helloSDL.c -lSDL -o helloSDL
./helloSDL

And half painted window should show up. Simple use API program where SDL is initialized, later surfice created and quit event handler resource cleanup is done. To better understand what code does please execute in terminal man followed by function name.

Building ffplay

Tutorial relies on ffplay.c which is part of ffmpeg source and currently only (partly) working example how to use ffmpeg to create simple video player. That is very simple but there may be some of us who are not very familiar with GNU make. We follow Compilation Guide and set environment variables, cd to ffmpeg source directory and in terminal execute:

PKG_CONFIG_PATH="$HOME/work/ffmpeg_build/lib/pkgconfig"
export PKG_CONFIG_PATH
./configure --prefix="$HOME/work/ffmpeg_build" --extra-cflags="-I$HOME/work/ffmpeg_build/include" \
   --extra-ldflags="-L$HOME/work/ffmpeg_build/lib" --bindir="$HOME/work/bin" --extra-libs="-ldl" --enable-gpl \
   --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus \
   --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree --enable-x11grab

Config will take a while and doesn’t provide feedback about progress, so be patient. I decided to build it and install it in ~/work folder instead in suggested $HOME, if you like to follow original just grep and remove work/. Now to get commands required to compile and link cmdutils.c and ffplay.c execute in terminal:

make -n > ~/dry_run

I outputed it to file since it produces quite large output and it may be interesting to look at it again. Simple grep for cmdutils.c and ffplay.c will return required instructions. Skip start of the line:

printf "CC\t%s\n" cmdutils.o;
printf "CC\t%s\n" ffplay.o;
printf "LD\t%s\n" ffplay_g;

and use compile commands from gcc -I. -I./ ... and link command from gcc -Llibavcodec -Llibavdevice ...
Link command will produce executable ffplay_g and it is located right under compile command for ffplay.o. Now you can nicely modify, build and debug ffplay.

Passing double to printf in XMM register

As last time all this happens on Linux and processor is 64 bit AMD. We want to call C function printf and to print float. Last time we used up to six registers to pass integers and %rdi for format string to printf and now we are going to use XMM registers and %rax to specify how many XMM registers we want printed. Compiling and linking is described in previous blog entry. I will actually use double which is 64 bit float. Here is the code:

.section .data
whatever:
    .int 3
double1:
    .double -1.234, 5.6789
double2:
    .double 123.456789
double3:
    .double 9.876, 5.4321
format:
    .asciz "have %d doubles %lf, %g, %g\n"
.section .text
.globl _start
_start:   
    nop
    movupd double3, %xmm2
    movdqu double1, %xmm1
    movsd double2, %xmm0
    movq whatever, %rsi
    movq $format, %rdi
    movq $3, %rax
    call printf
    call exit

Different MOV commands are used to load data into different XMM registers. One can use gdb to see what is loaded and where. Some will load single double into low half of register and some will load both doubles into register. Function printf will print only low half of register. XMM registers are in this call behaving as stack so the first float param is in %xmm0, the second one is in %xmm1 and so on.

Linux assembler, 64 bit, for beginners

Huge majority of literature about programming in assembly is written for 32 bit architecture. So beginner will have difficulties translating and linking examples on 64 bit machine. To be more precise the first major hurdle is calling C function printf. That goes from notorious error message “Accessing a corrupted shared library” to different calling convention. To solve linking problem we need to link using 64 bit version of ld-linux.so.2. For example Richard Bloom gives following solution:

ld --dynamic-linker /lib/ld-linux.so.2 -o [name] -lc [name].o

When we apply it we have “Accessing a corrupted shared library”. I am using Linux Mint 13, based on Ubuntu 12.04, and /lib/ld-linux.so.2 is symlink to 32 bit library:

$ ls -l /lib/ld-linux.so.2
lrwxrwxrwx 1 root root 25 Sep 30 16:38 /lib/ld-linux.so.2 -> i386-linux-gnu/ld-2.15.so

Since we are on 64 bit architecture we need to modify linking to use appropriate architecture:

ld --dynamic-linker /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 -o [name] -lc [name].o

That should eliminate “Accessing a corrupted shared library” problem. Now we have calling printf problem. On 32 bit architecture we simply push on the stack parameters and on 64 bit machine that just doesn’t work. Calling convention for 64 bit architecture is the following: result will end up in %rax, parameters, in order of appearance, will go into %rdi, %rsi, %rdx, %rcx and so on. There is quite good article about it GNU x86_64 assembler - Calling printf by Aleksandar Mitrevski. Now we can try some Hello World examples.

.section .data
hws: 
    .asciz "Hello World!\n"
.section .text
.globl _start
_start:
    mov $hws, %rdi
    call printf 
    call exit

We place our zero terminated string into register %rdi and call printf. To compile, link and execute we execute the following from terminal:

$ as -o helloworld.o helloworld.s
$ ld --dynamic-linker /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 -o helloworld -lc helloworld.o
$ ./helloworld

Now slightly more complicated example:

.section .data
whatever:
    .int 1234
format:
    .asciz "Our integer is %d\n"
.section .text
.globl _start
_start:
    movq whatever, %rsi
    movq $format, %rdi
    movq $0, %rax
    call printf
    movl $1, %eax
    movl $0, %ebx
    int $0x80

The last three lines can be replaced with call exit, but this is more like examples from book Professional assembly language by Richard Blum. Final example with even more parameters and some addition:

.section .data
a:
    .int 1234
b:
    .int 766
format:
    .asciz "%d + %d = %d\n"
.section .text
.globl _start
_start:
    movq a, %rsi
    movq b, %rdx
    movq $format, %rdi
    movq %rsi, %rcx
    addq %rdx, %rcx
    movq $0, %rax
    call printf
    call exit

If you managed to build and execute all three examples, quickly go and update your CV with freshly acquired assembly on Linux experience.

Quick HAProxy install and TCP load balancing on Debian

Had to test modified echo server and one of requirements was to create cluster. Wanted simple round robin with three servers at backend. After trying balance from repositories I find out that it doesn’t do well on new distros, used to work nicely on Ubuntu 8.04. There was significant loss of packages. So, after some googling decided to go with HAProxy. Only problem is that HAProxy doesn’t have quick start or I couldn’t find it. So here is quick start for TCP load balancer.
Downloaded source from HAProxy website. Installed requirements:

# apt-get install build-essential zlib1g-dev libpcre3-dev libssl-dev

Unpacked sorce and cd to root of sorce directory. Make is for many targets and many options, Debian Wheezy with installed libs would be:

$ make TARGET=linux2628 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1

When build is successfully executed we can install it:

# make install

Then I created config file called hapconfig with following content:

global
    daemon
    user tcpstuff
    group tcpstuff
    chroot /home/tcpstuff
    maxconn 1024

defaults
    mode tcp
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend tcp-in
    bind *:7000
    default_backend servers

backend servers
    balance roundrobin 
    server s1 127.0.0.1:7001
    server s2 127.0.0.1:7002
    server s3 127.0.0.1:7003

And now to run HAProxy:

# /usr/local/sbin/haproxy -f /home/tcpstuff/work/haproxy-1.4.24/hapconfig

It daemonizes itself, as requested in config file, and goes into background, to verify it is up we can execute:

ps aux | grep haproxy

To test echo server cluster I used:

$ echo ‘Hello world!’ | nc -q 1 192.168.1.101 7000

Why -q 1? If we omit it on Debian it will default to waiting for timeout to expire, on Linux Mint we can skip -q. And that was really quick.

Where is Mozilla Firefox Cache

This one is not about programming. Mozilla Firefox is popular web browser and it comes as default browser on many Linux distros. OK it may be called Iceweasel instead of Firefox, but that is about the same. Cache used to be in $HOME/.mozilla/firefox/(profile)/Cache but now it is not there. During the time one get used to delete or copy files from cache to some safe location, for example:

find (path to Cache) -type f -size +100k -exec cp --parents {} (where to copy) \;

But now cache is gone and nothing works. Initial reaction is google for solution, we are turning into web search addicts, and when that doesn’t bring us joy we are searching some more.
So, how we find cache? For the beginning we start Firefox and terminal. In terminal we execute:

ps aux | grep firefox

That will tell us what is process ID for Firefox. Should look something like this:

(user name)   16025 19.6  5.9 1065852 226284 ?      Sl   16:10   0:13 /usr/lib/firefox/firefox

That 16025 is process ID or pid. Knowing pid we can easily find out what files that proces is keeping open:

ls -l /proc/16025/fd | grep cache

We execute that in terminal, no need to be root. On Mint it is in $HOME/.cache/mozilla/firefox/(profile).
That was so simple, how is possible that I couldn’t find it on Google ;-)

Using POSIX MQ from Java - another part

In introduction we managed to create our message queue and to close it. Only thing which is unclear is where is it? Message queue is maintained by kernel and it is created in virtual file system. If we execute in terminal:

cat /proc/filesystems

we will see entry:

nodev    mqueue

If we want to take a closer look at our JNIMQ_1 we will have to mount mqueue. It is nicely described in man pages, just look for mq_overview entry. For lazy people here are commands for mounting message queue, you must do that as root:

# mkdir /dev/mqueue
# mount -t mqueue none /dev/mqueue

After this we should exit root level. Again using man pages we find out that second line is standard form for mount and looks like this mount -t type device dir. Once mounted we can use our standard set of tools for examining file content on Linux, for example:

ls -l /dev/mqueue
cat /dev/mqueue/JNIMQ_1

Now we can go on sending and receiving messages. If we do not specify time-out, those messages and queues will be persisted until system reboots.
Since we have functional message queue, we can nicely start chat. For that reason we will send some messages. We already have queue from last time and we will not create it:

public class SendMessage{ public native int send(String name, int flags, String text); public static void main(String[] args) { int status = new SendMessage().send("/JNIMQ_1",1,"Hello World!"); if(status!=-1) System.out.println("Message dispatched."); else System.out.println("Sending message failed."); } static { System.loadLibrary("SendMessage"); } }

That flags = 1 is open in write only mode. Boring part with javah we are skipping and here is implementation:

#include "SendMessage.h" #include <stdlib.h> #include <stdio.h> #include <string.h> #include <unistd.h> #include <sys/stat.h> #include <sys/types.h> #include <mqueue.h> JNIEXPORT jint JNICALL Java_SendMessage_send (JNIEnv *env, jobject obj, jstring name, jint flags, jstring text){ mqd_t mq; const char *cstr = (*env)->GetStringUTFChars(env, name, 0); mq = mq_open(cstr, flags); if((mqd_t)-1 == mq){ (*env)->ReleaseStringUTFChars(env, name, cstr); perror("mq_open"); return -1; } const char *ctxt = (*env)->GetStringUTFChars(env, text, 0); int res = mq_send(mq, ctxt, strlen(ctxt)+1, 0); if(res==-1){ perror("mq_send"); return -1; } if((mqd_t)-1 == mq_close(mq)){ perror("mq_close"); return -1; } (*env)->ReleaseStringUTFChars(env, name, cstr); (*env)->ReleaseStringUTFChars(env, text, ctxt); return 0; }

We compile it as described in previous blog entry and send some messages. If we cat /dev/mqueue/JNIMQ_1 we will see that it is growing.
Receiving is equally simple, here is the Java code:

public class GetMessage{ public native String getMessage(String name, int flags, int priority); public static void main(String[] args) { System.out.println(new GetMessage().getMessage("/JNIMQ_1",0,0)); } static { System.loadLibrary("GetMessage"); } }

We want queue opened as read only and we want messages with priority 0. After applying javac and javah we write implementation:

#include "GetMessage.h" #include <stdlib.h> #include <stdio.h> #include <string.h> #include <unistd.h> #include <sys/stat.h> #include <sys/types.h> #include <mqueue.h> JNIEXPORT jstring JNICALL Java_GetMessage_getMessage (JNIEnv *env, jobject obj, jstring name, jint flags, jint priority){ unsigned int up = priority; char ret[1025]; const char *cstr = (*env)->GetStringUTFChars(env, name, 0); mqd_t mq; mq = mq_open(cstr, flags); if((mqd_t)-1 == mq){ (*env)->ReleaseStringUTFChars(env, name, cstr); perror("mq_open"); return (*env)->NewStringUTF(env,"dud"); } ssize_t bytes_read = mq_receive(mq, ret, 1024, &up); if(bytes_read==-1){ perror("mq_receive"); return (*env)->NewStringUTF(env,"dud"); } ret[bytes_read] = '\0'; if((mqd_t)-1 == mq_close(mq)){ perror("mq_close"); return (*env)->NewStringUTF(env,"dud"); } (*env)->ReleaseStringUTFChars(env, name, cstr); return (*env)->NewStringUTF(env,ret); }

This 1024 should be retrieved from mq attributes, but I am lazy to do that and I still remember how it was created.
In this two examples I deviated from passing mq descriptor to Java, that is what jtux does. Not sure what is more expensive, crossing managed-native border or opening file, so do not ask.

Using POSIX MQ from Java - Introduction

All this is taking place on Linux Mint but should work on any other Linux. JDK is Oracle JDK, I am doing Android programming and it requires Oracle JDK, OpenJDK should do fine but path will differ. Goal of this introduction is to create new message queue and to close it - nothing else.
As usually in JNI workflow we write Java code:

public class PosixMQ { public native long mq_open(String name, int flags); public native int mq_close(long desc); public static void main(String[] args) { PosixMQ pmq = new PosixMQ(); long desc = pmq.mq_open("/JNIMQ_1",64); if(desc!=-1L){ System.out.println("We have MQ open."); System.out.println("Close returned " + pmq.mq_close(desc)); }else{ System.out.println("Open failed."); } } static { System.loadLibrary("PosixMQ"); } }

Idea about storing pointer to data structure into long is from jtux project, otherwise it should be message queue descriptor or (mqd_t)-1 in case of failure. Queue name must start with slash and 64 is O_CREAT | O_RDONLY we want queue to be created and to be read only for us. Now we compile PosixMQ.java and we run javah on class to get header for C implementation. On my box it looks like this:

/usr/lib/jvm/java-7-oracle/bin/javah -jni PosixMQ

Interesting part of PosixMQ.h looks like this:

/* * Class: PosixMQ * Method: mq_open * Signature: (Ljava/lang/String;I)J */ JNIEXPORT jlong JNICALL Java_PosixMQ_mq_1open (JNIEnv *, jobject, jstring, jint); /* * Class: PosixMQ * Method: mq_close * Signature: (J)I */ JNIEXPORT jint JNICALL Java_PosixMQ_mq_1close (JNIEnv *, jobject, jlong);

Finally we write trivial C implementation, being careful to include all required imports:

#include "PosixMQ.h" #include <stdlib.h> #include <stdio.h> #include <string.h> #include <unistd.h> #include <sys/stat.h> #include <sys/types.h> #include <mqueue.h> JNIEXPORT jlong JNICALL Java_PosixMQ_mq_1open (JNIEnv *env, jobject obj, jstring name, jint flags){ struct mq_attr attr; attr.mq_flags = 0; attr.mq_maxmsg = 10; attr.mq_msgsize = 1024; attr.mq_curmsgs = 0; const char *str = (*env)->GetStringUTFChars(env, name, 0); mqd_t desc; desc = mq_open(str, flags, 0644, &attr); (*env)->ReleaseStringUTFChars(env, name, str); return (intptr_t)desc; } JNIEXPORT jint JNICALL Java_PosixMQ_mq_1close (JNIEnv *env, jobject obj, jlong desc){ return mq_close((mqd_t)(intptr_t)desc); }

I was too lazy to pass attributes around, it should be done at later stage. Permissions are also conveniently hardcoded, though passing octal through parameter list is not difficult. To compile this I used:

gcc -I/usr/lib/jvm/java-7-oracle/include -I/usr/lib/jvm/java-7-oracle/include/linux -o libPosixMQ.so -shared -fPIC PosixMQ.c -lrt

Switch -fPIC will not be required on 32 bit Linux and -lrt instruction to link against the real-time is required everywhere. To run it we do:

java -Djava.library.path=. PosixMQ

in directory where are binaries and we see:

We have MQ open.
Close returned 0

Not very impressive but again very simple. Maybe next time we send and receive some messages, who knows.