Easy light polluted night sky workflow

I got four 120 seconds ISO 800 RAW frames. Better option would be to go for 60 seconds ISO 1600 and minimize tracking errors, stacking should eliminate noise anyway. Since pictures are taken under suburban sky, they are overexposed and have ugly orange background color. When we open them in Darktable default white balance preset is camera white balance and that looks like this.

Now we want to eliminate that ugly orange background and we switch from camera white balance to spot white balance.

If images were saved as JPEG, camera white balance would be applied, colors shifted to allow better compression and we won’t be able to do much processing. Before exporting images to TIFF, to export hit Ctrl+e, we will tweak exposure as on picture:

I export them as 16 bit integer per channel TIFF, if you do not know how to manage export settings it is explained in previous blog entries.
Now to do stacking I will open terminal and execute magic formula:

align_image_stack -a tif *.tiff

gmic tif0000.tif tif0001.tif -div 256 -gimp_blend 3,1,0 -mul 256 -c 0,65536 -type ushort -output one.tiff
gmic tif0002.tif tif0003.tif -div 256 -gimp_blend 3,1,0 -mul 256 -c 0,65536 -type ushort -output two.tiff
gmic one.tiff two.tiff -div 256 -gimp_blend 3,1,0 -mul 256 -c 0,65536 -type ushort -output tutorial.tiff

If you compare it with previous blog entries about G’MIC stacking you will see that new version of G’MIC is not completely backward compatible. Some people would maybe like to use DSS instead and that is also OK. Finally we open image in GIMP bump up contrast and LAB color decompose image. We duplicate A and B component, set copy mode to overlay and then merge them down. Do not flatten layers, there should be L, A and B layers. L layer can be slightly stretched or left how it is. When we LAB compose layers back into RGB image we will have nice saturated colors. Now, some people will proceed playing with curves but I will just add background gradient. I am not really using it as intended. I switch from divide to overlay and reduce opacity to 50%. That background gradient is part of astronomy plugin for GIMP by Georg Hennig and 2.8 compatible version is available from here git://gitorious.org/gimp-plugins-ambulance/gimp-plugin-astronomy.git. Building plugin is trivial. Here is result:

 
 Complete damage control was done in Darktable in three straightforward steps and that is why I call it easy. If level of black was lower and and for example exposure was reduced only -0.5EV, we could further increase contrast and get more of that Flame nebula. Though it will be more fiddling in GIMP and may be not so simple as it sounds.

Why to use RAW?

Quite frequently that question is repeated in different forms on Google+ Open Source Photography community https://plus.google.com/u/0/communities/110647644928874455108.
For me argument that RAW encodes data into 14 bit per channel vs 8 bit per channel, for JPEG, is more than sufficient. But for non-technical person that is not very convincing. If we add to it fact that internal camera processor in 90% of situations does great job when it creates JPEG and you need quite high level of processing skills to achieve the same starting from RAW, rock solid RAW argument doesn’t look so good. People advocating RAW start to look like Groucho Marx: “Who are you going to believe, me or your lying eyes?”.
So, here is simple example why RAW is good. Camera handling of great difference in luminosity is nowhere near to human eye. With automatic exposure on we will get or dark is too dark or bright is too bright. Offending photo is stored as RAW and I will open it in Darktable. It should be opened by default in darkroom mode. After doing sharpening and lens correction, in correction group (one with broken circle symbol) I switch to basic group. Here is what we got:

After activating overexposed plugin all overexposed parts will become red, like here:

To remedy that I switch on exposure plugin and push slide to unreasonably low -2.61EV. Now everything what is underexposed is blue.

After adjustment of exposure to reasonable -1.42EV we still have some underexposed areas but that is in shade and we can safely ignore it.

Now we can export it and do further processing in GIMP or even we can try exporting different levels of exposure and later do exposure blending.

More about stacking

In the last tutorial we stacked few frames using GIMP after manually aligning them. Since we were setting layers to screen mode it was at the end quite bright. What we could do to make it more natural is to make additional layers transparent. Opacity of bottom layer 100%, next 50%, next 25% and so on.

Automated stacking

Here Windows users have really wide choice of free or paid programs. For example Deep Sky Stacker and RegiStax. If there is dozen or more frames to stack it pays to start KVM or VBox and struggle with Windows for for few minutes in order to use Deep Sky Stacker. There is no real equivalent which will do the same task on Linux, register, align and stack frames using GUI. For smaller number of frames we can use ale - The Anti-Lamenessing Engine written by David Hilvert. Typically it is not compiled with ImageMagick support enabled, so we have to prepare images to be processed an to convert them into ppm format. We can install ImageMagick and use mogrify and convert to convert photos from terminal, but in order to make things easier we will use GUI.
If we are using DSLR camera or some of those new compact cameras we may be able store images in RAW format. It allows us to do significant amount of processing on such image. There is few really good programs for RAW processing, like UFRaw, RawTherapee or Darktable. We are going to use the last one Darktable. After installation we will right click on RAW file and select from context menu Open With Darktable. User interface is non-conventional so here is quick explanation. We want our RAW to be converted to ppm and we want to remove hot pixels.
Proper removal of hot pixels would be taking “dark frames” at the end of the session. That is place cap on lens and take picture using the same ISO value and exposure time as data frames. After that we can add dark to data frame as layer in GIMP and subtract it to remove hot pixels.
When Darktable shows-up we will see in the right pane tab which says more plugins. Clicking on it we open it and select hot pixels plugin. Again clicking on it we close it and under correct tab is hot pixels plugin, which is "switched off". We "switch it on" and it removes hot pixels.

We are happy with all default processing so far. Now we want to export image to ppm. For that we hit key “l” what brigs us to lighttable mode. On the right pane we locate export selected and make it to look like this:

Export could be achieved via export button or keyboard shortcut Ctrl+e. To go back to darkroom mode we hit “d”. We repeat process on all frames. If we added some exposure, EV or maybe two, it is likely that we have generated noise. Also if we do not want full size picture we may want to resize it, ale will finish processing much faster. In GIMP we do Filters -> Enhance ->Wavelet Denoise and here is how it looks after and before denoising:

If we want to resize that is Image -> Scale image, and we export image back to original ppm.
Now we can open terminal and cd to folder with ppms. This is what I did and what was output in terminal

ale --md 64 --dchain fine:box:1 *.ppm stacked.ppm
Output file will be'stacked.ppm'.                                 
Original Frame: 'img_0001_01.ppm'. 

Supplemental Frames: 
'img_0001_02.ppm'***** okay (88.869874% match). 
'img_0001_03.ppm'***** okay (90.897255% match). 
'img_0001.ppm'***** okay (91.984785% match). 
Iterating Irani-Peleg. 
Average match: 90.583972% 

Here is explanation --md Set element minimum dimension x. (100 is default),  --dchain fine:box:1 approximates drizzling. and two remaining params are input and output. To get more info execute ale --hA.
Result was rather cold and dark. In order to bring some warmth and dynamics I opened stacked.ppm in GIMP and did Colors -> Components -> Decompose where from drop-down LAB is desired. Now for each layer I made only one which I am currently working on visible and others invisible, clicking on eye in layers docking window. Then after duplicating layer and setting copy layer to Overlay mode, I merged visible layers accepting default option expand as necessary. That was repeated for L, A and B layer.

After that Colors -> Components -> Compose, again selecting LAB. At the end Colors -> Auto -> White Balance and Edit -> Fade Levels with default Replace mode and Opacity 33. Here is result:

Those are the same four frames from last tutorial, if you do not have own to process, download them, convert them to ppm and you can try ale stacking and post-processing with them.