Everything Is An Integer
Today I found myself plumbing the murky depths of C++. My latest task: file IO.
C++ provides a type fstream. Being a proper object-oriented system, the class hierarchy is completely indecipherable. However, the actual use of this type is fairly simple. An instance of fstream has a “get pointer” and a “put pointer”. To write to the file, you must seek the put pointer to a location, then write. To read, you must seek the get pointer, then read. Fairly simple, right?
The seek methods: seekp and seekg are overloaded:
ostream& seek[pg] (streampos pos);
ostream& seek[pg] (streamoff off, ios_base::seekdir way);Should be no problem, right? Let’s de-typedef-ify that:
ostream& seek[pg] (int pos);
ostream& seek[pg] (int off, int way);Ok, still no big deal. This is C++ after all, enums are integers. So, I carry on my merry way. I implemented the following functions:
template <class to_write>
static void write_out(fstream * fio,
offset location,
const to_write & value)
{
fio->seekp(location, ios_base::end);
write_out<to_write>(fio, value);
}
template <class to_write>
static void write_out(fstream * fio,
const to_write & value,
streampos position)
{
fio->seekp(position);
write_out<to_write>(fio, value);
}
template <class to_read>
static void read_in(fstream * fio,
to_read & into_location)
{
...stuff...
}The idea: wrap the seeking and writing into one operation, so my high level code need not concern itself with it. The behavior:
write_out<long>(foo, 8, ios_base::beg);
…and…
offset bar = 0;
write_out<long>(foo, bar, 0);
…both result in the first function being called! Were this a language with a proper type system, all those types would be distinct and the function overloading would work as advertised. Unfortunately, this is C++, where there is only 1 type: byte.
Let this serve as a warning, so you don’t spend half your day troubleshooting such a stupid problem.
The Smallest Things…
For the last few weeks I’ve been banging my head against a problem. I need my Photo Booth application to actually take a photo. It seems like such a simple thing, but it has actually been one of the most difficult ones I’ve encountered. Like the Great PostScript Debacle and the Mystery of the GAsyncQueue Ref before it, I spent a good week banging my head against the wall. I even took a day off to write an entire Lua Camera Module just so I could shell out and call a command line utility to try to work around it.
The best part? If you’ve been following the blog, you know I’m kind of a crybaby about poor documentation. While libgphoto2 is certainly a repeat offender on this count, no amount of documentation could have prepared me for what was to come.
But first, let’s go over my now-working implementation.
Taking A Picture With Libgphoto2
To take a picture, we need 3 functions:
gint dmp_cm_camera_init()gint dmp_cm_camera_finalize()gint dmp_cm_camera_capture(gchar * location)
dmp_cm_camera_init
gint dmp_cm_camera_init()
{
context = gp_context_new();
gp_log_add_func(GP_LOG_ERROR,
(GPLogFunc) dmp_cm_log_func,
NULL);
if (gp_camera_new(&camera) != GP_OK)
{
//error handling
}
if (gp_camera_init(camera, context) != GP_OK)
{
//error handling
}
return DMP_PB_SUCCESS;
}There are two main structs: Camera and GPContext. A Camera represents, shockingly, a camera attached to the system. A GPContext represents work to be done. Callback functions, data, and other things of that nature.
First we create a new context. Next we can add a log function to accept log messages from libgphoto2. In my experience, no matter what you do you will get a lot of useless garbage output from libgphoto2. For this reason, I recommend you don’t just let this spew to the console or some other user-facing output. At first, I was going to send this to the console queue, but I’ve since decided against using this feature. It is good to know about though in case you need it for troubleshooting.
After all of that is done, we need to create our camera object, and initialize libgphoto2.
dmp_cm_camera_finalize
gint dmp_cm_camera_finalize()
{
gp_camera_unref(camera);
gp_context_unref(context);
return DMP_PB_SUCCESS;
}Nothing particularly tricky there. We need to ensure we free our memory when we’re done, so we unref our camera and context. Having seen these two functions, you may be wondering to yourself: “Are we dealing with GObjects here?” Luckily for us, there is a simple test for this:
g_assert(G_IS_OBJECT(camera));I’ll spare you the effort of running this test: the assertion fails. Too bad really, but it is what it is. Libgphoto2 just uses function names similar to GObject.
dmp_cm_camera_capture
gint dmp_cm_camera_capture(gchar * location)
{
CameraFile * file;
CameraFilePath camera_file_path;
gint fd;
CameraEventType event_type;
void * event_data;
if (gp_camera_capture(camera,
GP_CAPTURE_IMAGE,
&camera_file_path,
context) != GP_OK)
{
//error handling
}
if ((fd = g_open(location, O_CREAT | O_WRONLY,
0644)) == -1)
{
//error handling
}
do
{
gp_camera_wait_for_event(camera,
1000,
&event_type,
&event_data,
context);
if (event_type ==
GP_EVENT_CAPTURE_COMPLETE) break;
}
while(event_type != GP_EVENT_TIMEOUT);
if (gp_file_new_from_fd(&file, fd) != GP_OK)
{
//error handling
}
do
{
gp_camera_wait_for_event(camera,
1000,
&event_type,
&event_data,
context);
}
while(event_type != GP_EVENT_TIMEOUT);
if (gp_camera_file_get(camera,
camera_file_path.folder,
camera_file_path.name,
GP_FILE_TYPE_NORMAL,
file,
context) != GP_OK)
{
//error handling
}
if (gp_camera_file_delete(camera,
camera_file_path.folder,
camera_file_path.name,
context) != GP_OK)
{
//error handling
}
gp_file_free(file);
do
{
gp_camera_wait_for_event(camera,
1000,
&event_type,
&event_data,
context);
}
while(event_type != GP_EVENT_TIMEOUT);
return DMP_PB_SUCCESS;
}This function is where the meat of the process is. First we need to do some housekeeping. We create a CameraFile pointer to represent the actual image file, and a CameraFilePath struct to represent the path to the file. We also create an int for use as a file descriptor, a CameraEventType and void pointer for our calls to gp_cmaera_wait_for_event
Next we call gp_camera_capture which triggers the camera to take a picture. After that is done, we’ll open a file descriptor to save the image. You’ll notice that the call to g_open is enclosed in parentheses. THIS STEP IS 100% MANDATORY Don’t omit it, you’ll be sorry. More on this in a bit.
Next, we wait for the camera to finish working. The camera uses an event system; it will emit events when things happen. After releasing the shutter, the camera has other work to do before it is “done taking the picture”. If you try to do the next step before the camera is ready libgphoto2 will spew garbage to your STDOUT and you’ll have to ctrl+c to fix it. To avoid this, we call gp_camera_wait_for_event while event_type != GP_EVENT_TIMEOUT || GP_EVENT_CAPTURE_COMPLETE Capture complete is obviously the event we care about, but it may have happened while we weren’t listening for it. In that case, we’ll settle for a timeout.
Next up is instantiating our CameraFile. We use our File descriptor that we just opened to call gp_file_new_from_fd. Unfortunately there is no gp_file_new_from_file_pointer which means that this call is POSIX only, and there’s no portable substitute.
After creating our CameraFile we download the image we just took by calling gp_camera_file_get and then delete the file from the camera using gp_camera_file_delete
Finally we make sure no events are pending, then return.
Why Are You Yelling At Me?
Good question. The block in question of course is
if ((fd = g_open(location, O_CREAT | O_WRONLY,
0644)) == -1)
{
//error handling
}Inside of that if block, I’m assigning a value and testing the result inside of the if statement. This operation is about a 2 out of 10 on the cleverness scale. Normally, you could omit the parentheses around (fd = g_open(location, O_CREAT | O_WRONLY, 0644). However, if we do it here, things go off the rails. Not right away, of course, but a few function calls later we get to:
if (gp_camera_file_get(camera,
camera_file_path.folder,
camera_file_path.name,
GP_FILE_TYPE_NORMAL,
file,
context) != GP_OK)
{
//error handling
}As soon as gp_camera_file_get(...) is evaluated, this is spewed to the console:
…and you have no choice but to kill the process.
Why does this happen? I have no idea. Why does enclosing the call to g_open in parenthesis fix it? Again, no idea. And it only happens here too. I just tried to modify the examples that come with libgphoto2 to reproduce the error and get that screenshot for this post, but it works fine there. Knowing my luck, if you download and build the program, it’ll work fine for you.
As long as it works, I guess…
DMP Photo Booth: Deep Magick
After working on DMP Photo Booth for a few months, the day came when I needed to implement actual functionality. It’s the day we all dread, but for me this was no longer some looming menace; it was time to stop fiddling around in my framework and actually build on it.
More specifically, it was time to figure out how to turn ~5 images and a background into a photo strip. After checking to make sure GLib and GTK didn’t provide this functionality (GdkPixbuf almost cuts it, but as far as I can tell, it can’t layer images over each other), I turned to google. After some time, I settled on my library: ImageMagick.
ImageMagick bills itself as sort of a command-line PhotoShop. I was suspicious as well, but that’s neither here nor there. The thing about ImageMagick that interested me is its language bindings. ImageMagick provides a library for many languages, including two for C: MagickCore for “Wizard-level developers” and MagickWand for us chumps. Being a chump, I decided to go with MagickWand.
NetBeans configuration
This was relatively straightforward. If you got GTK set up, this should be no problem for you. First, ensure you have the WagickWand development headers. On Ubuntu, this can be accomplished by the following command:
sudo apt-get install libmagickwand5 libmagickwand-devOn my system, libmagickwand5 was already installed, so I downloaded the headers and got to work.
Next, in NetBeans click Tools->Options, click on the C/C++ tab, and click on Code Assistance. Add the location of the ImageMagick headers (/usr/include/ImageMagick for me). Click OK.
Next, we need to set up our project. Right click your project and click Properties. Click Build->C Compiler. Under Additional Options, add MagickWand to your pkg-config --cflags string. Click Build->Linker and do the same with your pkg-config --libs string.
You are now ready to conjure some magick!
Conjuring Some Magick
Now, let us put on our robes and wizard hats; it’s time to do some magick! Let’s go over a function that will overlay a resized image over a larger background. You can find the actual production photo strip function on Github.
void cast_magick_spell()
{
MagickWandGenesis();
...Before we can do anything, we must initialize MagickWand. “MagickWandGenesis()”, you ask? Of course the function would be called MagickWandGenesis, what kind of silly question is that?
...
MagickWand * background_wand = NewMagickWand();
MagickWand * working_wand = NewMagickWand();
MagickWand * final_wand = NULL;
...The MagickWand * is the main object passed around in a MagickWand application. Working with MagickWand requires some juggling of these pointers, which is why we have 3 of them.
...
if (!MagickReadImage(background_wand,
"/tmp/background.jpg")
{
...This function reads an image from a file. It returns MagickTrue on success, and MagickFalse on failure. If it returns MagickFalse, we have some clean-up to do…
...
ExceptionType exception_error_code;
char *
exception_message = MagickGetException(
background_wand,
&exception_error_code);
...MagickWand makes use of “Exceptions” throughout, so if a function fails, you can most likely pull an exception out of it using MagickGetException. Like any exception, it is up to you what to do with them. Since I’m using GLib, I’ve been wrapping them in a GError and propagating them up. This is actually very easy to do; your error code and message are already there. All you need to do is G_DEFINE_QUARK your error quark and throw it in there. For the purposes of this function, I’m just going to use printf, do some cleanup, and return.
...
printf("Oh no! Exception %d: %s\n",
exception_error_code,
exception_message);
MagickRelinquishMemory(exception_message);
DestroyMagickWand(background_wand);
DestroyMagickWand(working_wand);
return;
}
...Nothing particularly shocking here. We printf our message, free the exception_message string using MagickRelinquishMemory, free our MagickWands using DestroyMagickWand, and return.
Assuming we make it past this block, background_wand now contains the background image. Next, we load the foreground image:
...
if (!MagickReadImage(working_wand,
"/tmp/foreground.jpg")
{
/*
* Exception handling omitted
* for brevity. You should still
* do it here...
*/
}
if (!MagickResizeImage(working_wand,
[WIDTH],
[HEIGHT],
LanczosFilter,
[BLUR]));
{
/* More error checking... */
}
...After loading the foreground image, and doing our error checking we attempt to resize the image using MagickResizeImage. This function takes several parameters:
MagickWand * working_wand: The MagickWand to operate onsize_t [WIDTH]: the width to set the image tosize_t [HEIGHT]: the height to set the image toFilterTypes LanczosFilter: the filter to use to resize. There are a list of them in the API documentation. Discussion of these is outside the scope of this post.double [BLUR]: The blur factor to apply. 1.0 is no change. The further 1.0, the blurrier the resulting image.
Like many calls in this library, this function can return MagickTrue or MagickFalse. If it returns MagickFalse, something threw… Next, we adjust the position of the image…
...
if (!MagickResetImagePage(working_wand,
[RELATIVE_PAGE_SPECIFIER]))
{
/*
* You'd think one of these wizards
* could have written a function that
* doesn't throw...
*/
}
...This function takes some explaining. The second parameter: char * [RELATIVE_PAGE_SPECIFIER] is what’s doing the work here. This is a Magickally formatted string that looks like this: "100x100+15+15!". Let’s examine this as a printf formatting string:
"%dx%d+%d+%d!"
We have 4 integer tokens here. The first token is canvas width, and the second is the canvas height. Note that these will not resize the image, so don’t get any bright ideas about eliminating the previous resize call. The third and fourth tokens are offset from X and Y respectively. These are what we’re really concerned about here. This will allow us to position our foreground image over the background.
Also, don’t forget to check for exceptions!
...
MagickSetLastIterator(background_wand);
if (!MagickAddImage(background_wand,
working_wand))
{
/* exceptional! */
}
...We’re almost there! Now we have to add the images from working_wand to background_wand. These MagickWand objects are lists of images. Like most lists, they have iterators. The call to MagickSetLastIterator sets background_wand’s iterator to the last image in the list. Any images add will be added after this image. next we call MagickAddImage which adds copies of working_wand’s images into background_wand. As before, don’t forget to check for exceptions.
...
final_wand=MagickCoalesceImages(background_wand);
if (!final_wand)
{
/* You guessed it! */
}
MagickSetLastIterator(final_wand);
...Now we need to combine all our images into one single image. This is accomplished by calling MagickCoalesceImages which returns a new MagickWand with all of our images combined into 1. The MagickWand used for this call remains unaffected. Obviously, if final_wand == NULL, something threw.
After this is done, we need to set the iterator of final_wand to the Last Iterator, or the next step doesn’t work as advertised…
...
if (!MagickWriteImage(final_wand,
"/tmp/final.jpg"))
{
/* wait for it! */
}
...Shockingly, this function writes your new image to a file. Make sure you check your exceptions.
...
DestroyMagickWand(background_wand);
DestroyMagickWand(working_wand);
DestroyMagickWand(final_wand);
MagickWandTerminus();
}…and were done! Clean up your pointers and call MagickWandTerminus to finalize the MagickWand library. If you browse to /tmp, you should have a newly created final.jpg if all was well!
Would I Do It Again?
Sure.
ImageMagick was a decently easy to work with library. The documentation wasn’t amazing, but it was tolerable. I’m still a little hazy on the use of the [RELATIVE_PAGE_SPECIFIER], but it’s working so far. One nice thing about the docs is that there are many examples. If the docs don’t explain something, you can look up an example and get an idea of how things work.
The only really big issue I have with this library is how it handles exceptions. This is an issue that I’ve touched on before; it is all too easy to forget to check a return code. I went down that road with DMP Photo Booth, and I’ve since rejected it. I spent an entire day refactoring my program to use GError.
MagickWand has exceptions, they’re just really easy to ignore. While writing this blog post, I caught several instances of unchecked return values in my 1 function that uses MagickWand. Tomorrow I plan to fix this, but it’s time I could be spending on something else.
If GError is Java’s checked exceptions with all it’s order and verbosity, then MagickWand’s exceptions are the Wild Wild West of C++’s unchecked exceptions. If you’ve spent any time working with C++, this has almost certainly bit you; some function doesn’t document if it throws, and your program magickally starts crashing because you didn’t catch something. Bad times are had by all. Sure, you could throw a try/catch block in main() and catch all exceptions to keep from crashing, but at that point your program is a dead man walking. Best to put it out of its misery…
Personally, if I ever write a personal project in C++ again, I’m likely to disable exceptions in my program; they’re just more effort than they’re worth. Maybe I’d even use GError in my C++ app if I could convince myself that I’m not a bad person for using a C library in C++.
Regardless, one blemish on an otherwise pleasant experience is no big deal. Here’s to a successful foray into the land of High Adventure!
Imagine: A World Without Strcmp
Recently, I’ve been working on implementing the config file for DMP Photo Booth, and luckily for me, GLib has the answer for this: GKeyFile. GKeyFile is GLib’s implementation of those flat config files you tend to see in Linux applications:
#Some comment!
[section]
value=words
some_bool=true…like that. I happily set to work, satisfied that today was not the day that I’d have to begin dealing with FILE pointers. Unfortunately, though I don’t have to deal with straight up file IO, I do still have to deal with functions that will fail if you so much as look at them funny. All the functions of GKeyFile take a group and key name string to look up, and if you pass in a bad string, things go off the rails. At first I had planned to use #defined strings, but halfway through implementing input validation in my first access function, I thought that there must be a better way. Strings are “difficult” to compare against known values, and it all seemed like so much work!
Next, I thought of using magic integers, but I’d have to come up with some way to resolve them to a string. Since this was all starting to sound like the beginnings of implementing a hash table, I turned to GLib for its implementation. This also seemed like more effort than it was worth. As I was poking around the docs, I found my answer!
Introducing Quark!
I’m talking about the GQuark. A GQuark functions in a similar manner to a hash table (in fact, if you dig through the source, you’ll see that GQuark uses a hash table.) GQuarks are a two-way association between a non-zero guint32 and a gchar * (A GQuark value of 0 resolves to NULL.) Basically, once you register a string as a quark, you can pass the GQuark object around and use it to obtain the string. You can also attempt to resolve a string to get its integer value if it has been registered. The best part about a GQuark is you can directly compare the GQuark object using operator==, avoiding a call to strcmp(), reducing boilerplate code.
Notable Functions
GQuarks do not have a lot of moving parts. Below are some important functions for using GQuark:
G_DEFINE_QUARK(some_string, function_name)
This macro defines a function named function_name_quark that returns a quark for “some_string”. Basically, this is your replacement for:
#define MAGIC_STRING_NAME "magic_string_literal"Where you might use the MAGIC_STRING_NAME before, you can now just call your newly defined quark function! Notice up above that in the macro parameter, some_string is not in quotation marks. This is important; the function defined by the macro will strigify the first parameter. For example, if you enter this:
G_DEFINE_QUARK(The quick brown fox jumped over the lazy
dog's back\n, dog_fox)You’ll get a function:
GQuark dog_fox_quark()…that returns a GQuark for the string:
"The quick brown fox jumped over the lazy dog's back\n"The second thing to note about this macro is that it declares AND implements the function. In other words, it expands to something like this:
GQuark name_quark()
{
...code...
}As you can see, it would be an error to place a semicolon behind the macro.
All of that may seem like a lot to remember, but if you want to define several quarks, it will save you hours of typing.
g_quark_to/from_string and friends
These functions are your GQuark manipulation functions. They will let you convert strings and GQuarks to each other. For instance, if you enter:
g_printf("%s\n", g_quark_to_string(some_quark));You will print the string contained in some_quark. Pretty straight-forward.
Thread Safety
You may be thinking: “gee, this sounds like it uses a whole lot of globals! Is this thread safe?” I know I was, so I did a little investigation. Looking through the source for GQuark, all the functions that access global variables are protected by a mutex; therefore you may feel free to use this from multiple threads.
Dancing in the liBreadLine
So, here I was working on a program to assist with writing these very blog posts, minding my own business… I was pondering how to make a config file for my program, and after mucking about with FILE pointers for a while, I wondered why I was doing this the hard way. Why don’t I just use a lua config file?
Unfortunately, Ubuntu apparently has a version of Lua in its repository that was tainted by Debian. The binary works fine, but the source has a bunch of random Debian headers in it. Instead of trying to figure out where lua5.2-deb-multiarch.h is, I decided I should just build the official Lua from lua.org. After downloading and unravelling the tarball, I executed my sudo make linux and got this:
lua.c:67:31: fatal error: readline/readline.h: No such
file or directory
After cursing myself for not writing down how I fixed this earlier, I turned to Google. Google tells me that I need to have the package libreadline-dev to build Lua. Luckily this is easily obtained. In Ubuntu you can just:
sudo apt-get install libreadline-dev
Crisis. Averted.
Reading and Writing Files With std::fstream
Among the many things made easier in object-oriented programming is file IO. Gone are the days of manipulating FILE pointers, now there is a C++ stream class for handling file operations: std::fstream. Like all stream objects, fstream objects use operators << and >> to perform input and output. Let’s look at an example.
int main(int argc, char** argv) {
std::fstream read("/tmp/read_from",
std::fstream::in);
std::fstream write("/tmp/write_to",
std::fstream::out);
std::string buffer;
while (!read.eof())
{
read >> buffer;
std::cout << buffer << std::endl;
write << buffer << std::endl;
buffer = "";
}
read.close();
write.close();
return 0;
}
This program does 3 things: 1) reads in the contents of read_from, 2) outputs the contents to cout, and 3) writes the contents to write_to.
First, we declare two fstreams, read and write. We pass to the constructor the absolute paths of the files to read, and the magic constants std::fstream::in and std::fstream::out respectively. We also declare a buffer string.
When you attempt to read past the end of a file, it sets fstream.eof to true. We can use this to loop through a file and read it in. We do this in our while loop.
Within our loop we take input from read, we output it to cout and write, then we reset our buffer string. After this is all done, we close our two fstreams and return 0.
This function has one problem. fstream operator<< tokenizes the file contents, stripping all whitespace. Therefore, after this program is run, write_to will not be the same as read_from. This is what happens if we ldiff them.
#ldiff read_from write_to
words words
more words | more
non-indented words | words
> non-indented
> words
Probably not what you wanted to happen. So, how can we prevent this? There is a function getline(). Let’s modify our example to use getline():
int main(int argc, char** argv) {
std::fstream read("/tmp/read_from",
std::fstream::in);
std::fstream write("/tmp/write_to",
std::fstream::out);
std::string buffer;
while (!read.eof())
{
getline(read, buffer);
std::cout << buffer << std::endl;
write << buffer;
if (!read.eof()) write << std::endl;
}
read.close();
write.close();
return 0;
}
This program functions as expected, producing an exact copy of read_from…
#ldiff read_from write_to
words words
more words more words
non-indented words non-indented words
For more info on the use of fstream, see this cplusplus.com reference page.
Doing My Programming... 