Pretty Printing our Pretty Program with Pretty
Tell me if you’ve heard this one before. You’re making some complicated data structure in Haskell. Something like the following:
data Expr =
EVal Int
| EIdent String
| EFunc String [String] [Expr]
| ECall String [Expr]
| ELet String Expr
| EWhile Expr [Expr]
| EShizzy Expr Expr Expr
| EIfElse Expr Expr Expr
| EReturn Expr
deriving (Show)You throw that deriving (Show) because it’ll totally help you debug! Time goes by, and you use your fancy Expr to write a robust imperative function:
test = EFunc "testFunc" ["gatito",
"moogle"]
[ELet "foo"
$ ECall "petCat" $ [EIdent "gatito",
EIdent "moogle"],
EWhile (ECall "catsPlacated" [])
[EShizzy
(EIfElse
(EVal 5)
(ELet "foo"
$ ECall "malloc" [ECall "sizeof" [EVal 2]])
(ECall "petCat" $ [EIdent "gatito",
EIdent "moogle"]))
(ECall "petCat" $ [EIdent "gatito",
EIdent "moogle"])
(ELet "foo" $ ECall "safeLeakMemory" [])
],
EReturn $ EIdent "foo"
]You’re still good, you got Rainbow Delimiters to keep that all sorted. Now, more time goes by and you’re in the repl and something doesn’t work. What was this test thing again?
*Shizzy> test
EFunc "testFunc" ["gatito","moogle"] [ELet "foo" (ECall "petCat" [EIdent "gatito",EIdent "moogle"]),EWhile (ECall "catsPlacated" []) [EShizzy (EIfElse (EVal 5) (ELet "foo" (ECall "malloc" [ECall "sizeof" [EVal 2]])) (ECall "petCat" [EIdent "gatito",EIdent "moogle"])) (ECall "petCat" [EIdent "gatito",EIdent "moogle"]) (ELet "foo" (ECall "safeLeakMemory" []))],EReturn (EIdent "foo")]
Well that’s unfortunate. What do we do now? Aside from the fact that this is completely useless to us, this certainly wouldn’t be OK to print out to a user…
The good news is that pretty printing is a snap thanks to the HughesPJ Pretty Printer. One simply has to implement the Pretty class for one’s type, and then they can get some reasonable output:
testFunc(gatito moogle) {
let foo = petCat(gatito moogle)
while(catsPlacated()) {
+--
{
if (5) let foo = malloc(sizeof(2))
else petCat(gatito moogle)
}
+++
+ +
+++
{
petCat(gatito moogle)
}
+++
+ +
+++
{
let foo = safeLeakMemory()
}
+--
}
produceResult foo
}Let’s see how we get there, constructor by constructor.
EVal and EIdent
First up are the easy ones: EVal and EIdent, which represent bare values:
instance Pretty Expr where
pPrint (EVal v) =
int v
pPrint (EIdent s) =
text sFirst we have to declare our instance, but next we can just print the values. int pretty prints an integer, and text pretty prints a string.
EFunc and ECall
Next we have function calls and definitions:
pPrint (EFunc n a b) =
text n
<> (parens $ hsep $ fmap text a) <+> lbrace
$+$ (nest 2 $ vcat $ fmap pPrint b)
$+$ rbrace
pPrint (ECall n a) =
text n <> (parens $ hsep $ fmap pPrint a)There are some operators here that require explanation. All of these operators have type (Doc -> Doc -> Doc ), and are used to combine two Doc into one. A Doc is the type of thing that can be pretty printed, which is returned by calls to pPrint and various other primitive pretty printing functions provided by the library (like text and int)
<> glues the thing on the left next to the thing on the right with no space in between, where <+> does the same, but leaves 1 space in between. Similarly, $$ and $+$ glues the thing on the right to the bottom of the thing on the left. $+$ allows the right hand side to overlap with the left (more on this later). There are also hsep, and vcat which takes lists of Doc. These functions work similarly to <>, $$ and friends; they glue Docs together horizontally and vertically. *cat are the non-plus variants, and *sep are the plus variants.
Other functions introduced here are parens, which takes a Doc and surrounds it by parens. lbrace and rbrace insert { and } respectively. There is a braces function as well, but I didn’t use it because I want to control how the braces are shown. Finally, we have nest, which indents its argument. This function is great because it takes into account outer nest calls, so we can get arbitrary nesting.
ELet, EWhile, EIfElse, and EReturn
pPrint (ELet n e) =
text "let" <+> text n <+> equals <+> pPrint e
pPrint (EWhile p b) =
text "while" <> (parens $ pPrint p) <+> lbrace
$+$ (nest 2 $ vcat $ fmap pPrint b)
$+$ rbrace
pPrint (EIfElse p t f) =
text "if" <+> (parens $ pPrint p) <+> pPrint t
$+$ text "else" <+> pPrint f
pPrint (EReturn v) =
text "produceResult" <+> pPrint vNothing particularly fancy here. in ELet, we have equals which inserts an equals sign and we have more nesting throughout. As you can see, this is very simple once you get the hang of it; there are few abstract instances involved, and most functions do what they say.
EShizzy
As we all know, any programming language that doesn’t have Shizzy statements isn’t worth knowing, so of course I implement them:
pPrint (EShizzy u d t) =
onFire
$+$ (nest 6 (lbrace $+$ (nest 2 (pPrint u))
$+$ rbrace))
$+$ noRules
$+$ (lbrace $+$ (nest 2 (pPrint d)) $+$ rbrace)
$+$ noRules
$+$ (nest 6 (lbrace $+$ (nest 2 (pPrint t))
$+$ rbrace))
$+$ onFire
where
onFire = nest 2 $ text "+--"
noRules = nest 2 $ (text "+++" $$ text "+ +"
$$ text "+++")This produces the following output as we’d expect:
+--
{
if (5) let foo = malloc(sizeof(2))
else petCat(gatito moogle)
}
+++
+ +
+++
{
petCat(gatito moogle)
}
+++
+ +
+++
{
let foo = safeLeakMemory()
}
+--and this gives me a good opportunity to demonstrate the difference between $$ and $+$. Let’s swap all the plus variants out:
onFire
$$ (nest 6 (lbrace $$ (nest 2 (pPrint u))
$$ rbrace))
$$ noRules
$$ (lbrace $$ (nest 2 (pPrint d))
$$ rbrace)
$$ noRules
$$ (nest 6 (lbrace $$ (nest 2 (pPrint t)) $$ rbrace))
$$ onFire
where
onFire = nest 2 $ text "+--"
noRules = nest 2 $ (text "+++" $$ text "+ +"
$$ text "+++")Now, the pretty printer outputs the following:
+-- { if (5) let foo = malloc(sizeof(2))
else petCat(gatito moogle)
}
+++
+ +
+++
{ petCat(gatito moogle)
} +++
+ +
+++ { let foo = safeLeakMemory()
}
+--Basically, if something is nested far enough that it would appear after the end of the left hand side argument, it is pasted to the right of it instead of below it. Honestly, it’s pretty strange behavior to me, I say just avoid $$ for the most part and you’ll be fine. It’s not a huge stretch for me to think of a use for this, but I think I’d just use <> or <+> if I wanted this to happen.
Regardless, I’ve worked on projects that use this pretty printing library before, and now that I’ve given it a shot I know why!
Fun With Makefiles
Lately, I’ve been toying with the idea of trying my hand at some graphics programming. After spending the better part of yesterday trying to figure out how to even get started, I think I have a way ahead.
Building hello triangle using OpenGL is a fairly involved task. First, you need to settle on a graphics library. There are two choices here: OpenGL and DirectX. Obviously, I’ll be selecting OpenGL in order to avoid Microsoft vendor lock-in.
Next, you need a library to display a window for you. Sure, you could do it yourself, but then you’d get bogged down in a quagmire of platform specific issues. If you’ve been reading the blog, you know I don’t care for this sort of platform dependent nonsense, so I’ve tenatively settled on SDL 2. SDL is a cross platform multimedia library that handles sound, input, window creation, and the like. I plan to use this, in conjunction with an OpenGL context to do my work.
After you have that in order, you need an OpenGL Function Loader. Apparently the folks at Khronos were inspired by DMP Photobooth’s module system, there isn’t some opengl.h file you can just include and get your functions: you get to call dlopen and use dlsym to get function pointers. This wouldn’t be a huge issue if there were just a few functions, but there are thousands of them. In light of this, I’ve elected to go with GL3W for the time being. GL3W is a simple python script that generates a .c file containig the function pointers, and a .h to include.
All of this leads us to the topic of today’s post. How do we build this mess of libraries and random .c files?
We’ll Make it Work
The obvious answer here is that we need to use some sort of build system. Given my past experience with the abominations produced by NetBeans, I’ve elected to roll my own. Let’s take a look:
.DEFAULT_GOAL := allFirst, we have the default goal. By default, the default goal is the first one in the file. However, I like to make things explicit. Here, we set the default goal to “all”, which builds the code for all targets.
Next, we define some variables:
CC = gcc
COMPILE_FLAGS = -c -g -Wall -Wextra -std=c11 $(OPTIMIZE_LEVEL)
LINK_FLAGS = -g -Wall -Wextra -std=c11 $(OPTIMIZE_LEVEL)
OPTIMIZE_LEVEL = -Og
LINKER_LIBS = -lSDL2 -ldl -lGL
RM = rm -f
UNIVERSAL = gl3w gl_includes.hThe first variable, CC is built-in, and defaults to gcc. Again, I’m redefining it here to be explicit. After that, I define COMPILE_FLAGS and LINK_FLAGS, which are the flags I want to pass when I’m compiling someing to be link at a future time, and when I’m compiling and linking respectively. I define OPTIMIZE_LEVEL separately, because I want to potentially change it, and I don’t want to have to worry about if the two are in sync.
LINKER_LIBS are the libraries I’m going to be using. RM is the rm command, with flags, to be used in the clean target. UNIVERSAL is a list of files and targets that all buid targets depend on.
all : chapter1 chapter2 chapter3 chapter4 chapter5 chapter6 chapter7 chapter8 \
chapter9 chapter10 chapter11 chapter12 chapter13 chapter14 chapter15 \
chapter16 chapter17
chapter1 : $(UNIVERSAL) chapter1.c
@echo "Building chapter 1:"
$(CC) -o chapter1 $(LINK_FLAGS) chapter1.c gl3w.o $(LINKER_LIBS)
...
chapter17 : $(UNIVERSAL)
@echo "Building chapter 17:"Here we have the meat of our makefile. The tutorial I’m following has 17 chapters, and I’ll be building code from each. We have an “all” target that builds each chapter, and we have a target for each chapter that builds an executable. Each chapter target depends on UNIVERSAL and its own files.
gl3w : gl3w.c GL/gl3w.h GL/glcorearb.h
$(CC) $(COMPILE_FLAGS) gl3w.cHere we build the source files that GL3W produces. I’m compiling it into a .o file so that it can be linked into the code for the various chapters.
clean:
@echo "Deleting .o files..."
$(RM) *.o
@echo "Deleting core..."
$(RM) core
@echo "Deleting chapters..."
$(RM) chapter1
$(RM) chapter2
$(RM) chapter3
$(RM) chapter4
$(RM) chapter5
$(RM) chapter6
$(RM) chapter7
$(RM) chapter8
$(RM) chapter9
$(RM) chapter10
$(RM) chapter11
$(RM) chapter12
$(RM) chapter13
$(RM) chapter14
$(RM) chapter15
$(RM) chapter16
$(RM) chapter17Finally, I have my clean target. Here we delete all the cruft that builds up in the build process.
It’s a simple makefile, but I feel it’ll make this process easier. I can just do the exercises and hopefully spend less time fiddling with gcc.
On ArrayLists and Vectors
Long ago, when I first started doing Java, I struggled against Java’s limitations regarding the resizing of arrays. Eventually after poking around for solutions, I found the ArrayList. Before that, I found the Vector, but for whatever reason, NetBeans was saying that Vector was deprecated, so I settled on ArrayList, and it was fine.
A few years later, I took my first class on C++, and learned about the std::vector. I was also introduced to the concepts of the vector and the linked list. Needless to say, I was confused. If vectors and linked lists aren’t the same thing, then what is this ArrayList thing? At the time, I elected not to pursue the question, needing to focus on figuring out these “pointer” things they were making us learn. I put it out of my mind.
Flash forward to today. This morning I was travelling the internet, and I came across a forum thread. The topic of the ArrayList came up. Curiosity finally overcame me, and I looked into the issue.
So, About Those ArrayLists…
The Java Collections framework is broken up into various datatype interfaces. There is an interface for lists, queues, maps, sets, and deqeues.
You’ll notice that there is no vector interface. While a list and an array have different semantics and use cases, you can use them roughly in the same way. “Array indexing” can be implemented on a list by iterating to the nth node, and arrays can be traversed like a list. Since there’s nothing a list can do that a vector can’t and vice versa, it follows that they can both implement the same interface. Sun had to make a choice: name that interface “list” or “vector”. They went with list.
Given that Java’s vector type implements the list interface, it follows that they would reflect that in the name: ArrayList. For all intents and purposes, ArrayList is equivalent to C++’s std::vector. ArrayList implements the Collections framework’s list interface, but it is a growable array. It behaves like a vector, indexing is cheap, resizing is expensive. Java also provides a LinkedList that behaves like a linked list.
So, what about Java’s Vector? It seems that Vector predates the Collections framework. After the introduction of the Collections framework Vector was retrofitted to be a part of it, also implementing the list interface. So, what’s the difference between ArrayList and Vector? Vector is synchronized, and therefore is safe to use in threaded code. Due to this it is significantly slower than ArrayList.
Simply put: use ArrayList in single-threaded code, and Vector in multi-threaded code.
Why Names Matter
This is why names matter. One might not think much of it, but imagine how many hiring managers would be out an interview question if ArrayList and Vector had been named Vector and SynchronizedVector respectively?
DMP Photo Booth 1.0
Well, the day has come and gone. DMP Photo Booth’s final test on June 21st went off without issue, and DMP Photo Booth has left Beta and is now considered “production ready”. The initial 1.0 release can be found on GitHub.
The significance of June 21st is the very reason DMP Photo Booth was created; the 21st is the day of my wedding. My wife wanted a photo booth for the reception. We looked into renting a photo booth, but it turns out that they run around $1,000. I turned to open source. Some quick googling turned up some options, but they were all personal projects or out of date. Sure I could get somebody else’s project working, but what’s the fun in that? I decided that we didn’t need to rent one, or download one, I could build it!
In late 2013, I set to work in earnest. I had a couple of months of downtime in school, and since I’m not currently working it was the perfect time. I decided I had three main objectives for this project: get some arduino experience, get some GTK+ experience, and do this all as portably as possible. I had initially decided to mostly ignore GLib and focus on GTK, but slowly I grew to appreciate GLib for what it is: the standard library that C never had. First I used GModule to handle shared libraries in a portable manner. Next I decided to use GLib primitives to keep from having to deal with cross-platform type wonkiness. Next, having grown tired of dealing with return codes, I refactored the project to use GLib’s exception replacement: GError.
Lessons Learned
It’s not all roses and puppies though. There are certainly things I’d do differently. DMP Photo Booth is developed in an Object Oriented style, passing opaque structs with “method” functions that operate on them. Each component of the program are organized into their own source file with file scoped globals scattered throughout. Said globals are protected by mutexes to create a semblance of thread safety. That said, threading issues have been a major thorn in my side. Long story short: I regret this design choice. While I still feel that this is the correct way to structure C code, and that if globals are required, this is the correct way to handle them; I feel that I should have made more of an effort to limit side effects. Recently, I’ve spent some time doing functional programming, and if I could do it again I’d try to write in a more functional style. Fortunately for me, this is something that a little refactoring could help with.
Additionally, one thing I thought would be a major help is something that began to be a major thorn in my side: NetBeans. As the size of the project grew, NetBeans got slower and slower. It seemed that I spent more time fiddling with IDE settings than actually coding. Even worse is that the IDE-generated makefile is so convoluted that it’s extremely difficult to modify by hand in a satisfying way. I’ve always coded with and IDE so I wouldn’t have even considered not using one, but then I spent some time with Haskell. One of Haskell’s “problems” is that it doesn’t have good IDE support. It doesn’t seem like any IDE really handles it well, so most people use Emacs. Personally, I haven’t really warmed up to Emacs, but GEdit has syntax highlighting for Haskell and a built-in terminal for GHCI. GEdit also has syntax highlighting for C. Next time, I will seriously consider using a lighter-weight text editor for a C project. All this said, I think NetBeans for Java remains the way to go.
What’s Next
Like any program, version 1.0 is just one of many versions. There certainly remains a lot of work to do with DMP Photo Booth. Some major items you are likely to see whenever I get around to working on DMP Photo Booth some more:
Options Dialog
I think anybody who has seen it will agree: the options dialog in DMP Photo Booth is bad. It’s poorly organized, and kind of wonky. Personally, I modify settings using the .rc file, which is telling. This is certainly a high-priority improvement.
Functional Refactor
Like I said above, the code could use a pass to limit side effects. Funtions need to have their side effects limited, and globals need to be eliminated unless absolutely necessary. However, C is not a functional language. While one could argue that function pointers enable functional programming in C, this is a very pedantic argument. I won’t be going crazy with functional programming techniques. There will be no Monads, or for loops being turned into mappings of function pointers.
Optional Module API
An idea I’ve had on the back burner for a while is an optional module API. This would be used for very specific quality-of-life things. For instance, a module could provide a GTK widget to be shown in the options dialog. Any module that doesn’t want to implement any or all of the optional API can just ignore it. The module loading function will gracefully handle the dlsym failure, just treating it as it is: declining to implement the API. I have no plans to change the current existing API, so all you module developers can rest easy!
User Interface Module
It occurred to me that it might be good to have a UI module. This would provide the UI, and wouldn’t be tied to the trigger/printer/camera module start/stop system. This module would be loaded at startup and unloaded on shutdown. This would allow the Photo Booth to use different widget toolkits: QT, Curses, Cocoa, WinForms, or whatever else. Under this scheme, the current GTK+ interface would be abstracted into the reference UI Module.
List as a Monad
Much ink has been spilled on the subject of “Lists as Monads”. Just the fact that they are of course, not what it means for you. This doesn’t really seem surprising to me; there are better monads to use to describe how monads work. Additionally, lists have their own special functions for working with them; you don’t fmap a list, you just call map.
Presumably, Lists as Monads is something that seasoned Monadimancers understand. Maybe it’s some sort of rite of passage. But no more. Today I’ll shed light on the arcane mysteries of Lists as Monads.
Motivation
But first we need a problem to solve. For my example I’ll be using the cartesian product of two lists. The cartesian product of two sets (we’ll be using lists) is defined as:
A X B is the set of all ordered pairs (a, b) such that:
a is a member of set A and
b is a member of set B…translated into Haskell we should get a function with the following signature:
cartProd :: [a] -> [b] -> [(a, b)]How might this function look? Well, we’d need three functions. The first function should have the signature:
cartProd'' :: a -> b -> (a, b)This function is pretty straight forward, it pairs an a and a b. Next we’d need the function:
cartProd' :: [b] -> a -> [(a, b)]This function maps (cartProd'' a) over b, producing the list [(a, b)]. Finally, we need a function to tie it all together:
cartProd :: [a] -> [b] -> [(a, b)]This function maps (cartProd' b) over a, then concats the resulting list of lists. An implementation might look like this:
cartProd :: [a] -> [b] -> [(a, b)]
cartProd l r = concat $ map (cartProd'' r) l
where cartProd' :: [b] -> a -> [(a, b)]
cartProd' r l = map (cartProd''' l) r
where cartProd'' :: a -> b -> (a, b)
cartProd'' l r = (l, r)Did you go cross-eyed? Not exactly the most concise function that’s ever been written. Surely there is a better way…
Binding Our Lists
You may remember a few weeks back I talked about using the Maybe Monad. The gist of that post being that if you’re inside a do block, you can treat your Maybe as if it were just a plain value. It turns out that we can do something similar with Lists.
The Monad instance for [] is somewhat complicated, but it’s operation is straight forward. If >>= takes a Monad a, a function that takes an a and returns a Monad b, and returns a Monad b, what happens if we bind a simple function to [1]?
Prelude> [1] >>= (\a -> return $ a + 1)
[2]…ok, so it added 1 to 1, and stuffed it into a list. Seems pretty straight forward. Let’s try something a little more complicated:
Prelude> [1, 2] >>= (\a -> return $ a + 1)
[2,3]…so this time it added 1 to each list node, as if we’d called map ((+) 1) [1, 2]. Let’s try something else:
Prelude> [1] >>= (\a -> [(a + 1), (a - 1)])
[2,0]…this time we tried it in reverse. We bound [1] to a function that returns a list with two elements. The resulting list contained two elements. Again, nothing ground breaking here, but what if we do both?
Prelude> [1, 2] >>= (\a -> [(a + 1), (a - 1)])
[2,0,3,1]…now we get a list with four elements: 1+1, 1-1, 2+1, and 2-1. To replicate this behavior we can’t just map the lambda over the original list. We need to add a call to concat. Let’s expand this our a bit further:
Prelude> [1, 2] >>= (\a -> [(a + 1), (a - 1)]) >>= (\b -> [b, b])
[2,2,0,0,3,3,1,1]…all simple functions, but if we were to try to do this without the use of List’s monadic functions it’d become a mess like my cartProd function. Speaking of which…
The Better Way
Getting back to our original topic. Now that we have a feel for how List’s monadic interface works, how could we re-implement cartProd to not be terrible? Simple:
cartProd :: [a] -> [b] -> [(a, b)]
cartProd l r = do l' <- l
r' <- r
[(l', r')]I’m often hesitant to toot my own horn and call something I wrote “elegant”, but it’s hard to argue that this function isn’t. In three short lines, I managed to re-implement that nested monstrosity I wrote before. There is one fairly massive gotcha here…
In the first and second lines, we bind our two lists to a label. It’s important to note that the order of these matters. The lists will be cycled through from the bottom up. Meaning that for each element of l, all elements of r will be evaluated. For example, using our cartProd:
Prelude> cartProd [1,2] [3,4]
[(1,3),(1,4),(2,3),(2,4)]1 is paired with 3, then 1 is paired with 4, then 2 is paired with 3 and 2 is paired with 4. Were we to swap the order in which we bind l and r to look like this:
cartProd :: [a] -> [b] -> [(a, b)]
cartProd l r = do r' <- r
l' <- l
[(l', r')]Then the output would look like this:
Prelude> cartProd [1,2] [3,4]
[(1,3),(2,3),(1,4),(2,4)]1 is paired with 3, then 2 is paired with 3, then 1 is paired with 4, then 2 is paired with 4. Before the first element of the tuple was grouped together where here the second element is. For our purposes, both results are valid per the definition of cartesian product, but that may not hold for you so be careful.
Yet Another Aeson Tutorial
Lately I’ve been trying to make sense of Aeson, a prominent JSON parsing library for Haskell. If you do some googling you’ll see two competing notions: 1) Aeson is a powerful library, but it’s documentation is terrible and 2) about 10,000 Aeson tutorials. Even the Aeson hackage page has a huge “How To Use This Library” section with several examples. These examples usually take the form of:
“Make a type for your JSON data!”
data Contrived = Contrived { field1 :: String
, field2 :: String
} deriving (Show)“Write a FromJSON instance!”
instance FromJSON Contrived where
parseJSON (Object v) = Contrived <$>
v .: "field1" <*>
v .: "field2"“Yay!!”
I’ll spare you the long form, other people have done it already and I’m sure you’ve seen it already. What I quickly noticed was that this contrived example wasn’t quite cutting it. I’ve had some challenges that I’ve had to overcome. I’d like to share some of the wisdom I’ve accumulated on this subject.
Nested JSON
The first Problem I’ve run into is the nested JSON. Let’s take a look at an example:
{
"error": {
"message": "Message describing the error",
"type": "OAuthException",
"code": 190 ,
"error_subcode": 460
}
}That is an example of an exception that can get returned by any Facebook Graph API call. You’ll notice that the exception data is actually contained in a nested JSON object. If passed to a parseJSON function, the only field retrievable by operator .: is “error”, which returns the JSON object. We could define two types and two instances for this like:
data EXTopLevel = EXTopLevel { getEx :: FBException
} deriving (Show)
data FBException = FBException { exMsg :: String
, exType :: String
, exCode :: Int
, exSubCode :: Maybe Int
} deriving (Show)
instance FromJSON EXTopLevel where
parseJSON (Object v) = EXTopLevel <$>
v .: "error"
instance FromJSON FBException where
parseJSON (Object v) = FBException <$>
v .: "message" <*>
v .: "type" <*>
v .: "code" <*>
v .:? "error_subcode"In this case, you could decode to a EXTopLevel, and call getEx to get the actual exception. However, it doesn’t take a doctor of computer science to see that this is silly. Nobody needs the top-level object, and this is a silly amount of boilerplate. The solution? We can use our friend the bind operator. Aeson Objects are instances of Monad, and it turns out that it’s bind function allows us to drill down into objects. We can re-implement that mess above simply as:
data FBException = FBException { exMsg :: String
, exType :: String
, exCode :: Int
, exSubCode :: Maybe Int
} deriving (Show)
instance FromJSON FBException where
parseJSON (Object v) = FBException <$>
(e >>= (.: "message")) <*>
(e >>= (.: "type")) <*>
(e >>= (.: "code")) <*>
(e >>= (.:? "error_subcode"))
where e = (v .: "error")Much better right? I thought so too.
Types With Multiple Constructors
The next problem I’d like to talk about is using types with multiple constructors. Let’s take a look at an example:
{
"value": "EVERYONE"
}When creating a new album of facebook, the API needs you to set a privacy on the album. This setting is set using this JSON object. This would seem to be a trivial case for Aeson:
data Privacy = Privacy { value :: String
} deriving (Show)
instance FromJSON Privacy where
parseJSON (Object v) = Privacy <$>
v .: "value" <*>Unfortunately, the following is not a valid privacy setting:
"{
"value": "NOT_THE_NSA"
}However, our Privacy type would allow that. In reality, this should be an enumeration:
data Privacy = Everyone |
AllFriends |
FriendsOfFriends |
SelfBut how would you write a FromJSON instance for that? The method we’ve been using doesn’t work, and parseJSON takes and returns magical internal types that you can’t really do anything with. I was at a loss for a while, and even considered using the method I posted above. Finally, the answer hit me. Like many things in Haskell, the answer was stupidly simple; just define a function to create the privacy object, and use that in the parseJSON function instead of the type constructor! My solution looks like this:
instance FromJSON Privacy where
parseJSON (Object v) = createPrivacy v .: "value"
createPrivacy :: String -> Privacy
createPrivacy "EVERYONE" = Everyone
createPrivacy "ALL_FRIENDS" = AllFriends
createPrivacy "FRIENDS_OF_FRIENDS" = FriendsOfFriends
createPrivacy "SELF" = Self
createPrivacy _ = error "Invalid privacy setting!"If the parseJSON needs a singular function to create a type, and you have more than one type constructor for your type, you can wrap your type constructors in one function that picks and uses the correct one. Your type function doesn’t need to know about Aeson; Aeson magically turns it’s parser into whatever type your function calls for.
Doesn’t Play Nice With Others
For the last few weeks, I’ve been looking into making a Printer Module in Haskell. I must say, it’s been a pretty miserable experience. Not the Haskell part, that was ok. No, my issue is more basic. It seems that Haskell doesn’t like to share.
My plan was to build a module in Haskell to do the printer logic, then link that module as a library into C, which will be imported by the Core as normal. A preliminary look about the internet confirms that this is supported behavior. There are a few trivial examples peppered throughout the internet; so I set to work, confident that this was a solvable problem.
Giving Cabal A Shot
Cabal is Haskell’s package management program. In addition to this, it serves as Haskell’s answer to make. With a simple call to:
cabal initYou are presented with a series of questions about your package. After filling out the form (and selecting library), Cabal creates a Setup.hs file. Calling:
runhaskell Setup.hs configure
runhaskell Setup.hs build…produces a .a library for your package. Success, right? Unfortunately when you try to load that you get a linker error stating something to the effect of “can not be used when making a shared object; recompile with -fPIC“. After hours of research I have determined that this is because the Haskell’s libraries have not been compiled using the -fPIC, which prevents them from being used with a static library.
Trying GHC
The Glasgow Haskell Compiler can be used to compile libraries directly. Having given up on cabal, I decided to try to cut out the middle man and use GHC. After much tinkering, I came up with a Makefile that worked, which I will preserve here for posterity:
COMPILER=ghc
HS_RTS=HSrts-ghc7.6.3
OUTPUT=dmp_printer_module.so
all:
"${COMPILER}" --make -no-hs-main -dynamic \
-l${HS_RTS} -shared -fPIC dmp_printer_module.c \
DmpPrinterModule.hs -o ${OUTPUT}This makefile compiles any required Haskell scripts, as well as a C “glue” source file that initializes and finalizes the Haskell environment. More on what goes in that file can be found in the GHC documentation.
Cool, good to go, right? Wrong.
Couldn’t Find That Dyn Library
So, I started adding things to the module to make sure it doesn’t break. After adding some dependencies, and trying to recomplie, I start seeing this error:
Could not find module `[SOME_MODULE]'
Perhaps you haven't installed the "dyn" libraries
for package `[SOME_PACKAGE]'?After more hours of research, it turns out that for a module to be used in a shared library, it must be compiled as one. Seems logical, but that would imply that all module developers have to go through this nightmare. And the developers of any dependencies they use have to have done so. And so on…
Since even Prelude hadn’t done so, I set off to figure this out. After poking about, it turns out that Debian provides a package ghc-dynamic, which provides the dyn libraries from Base. I installed it, and things were checking out. However, the dependencies I was using still did not work.
After some more reasearch, I found a suggestion that I re-install all my Cabal packages using the --enable-shared flag, which would provide me with my dyn libraries. I gave it a shot, but since my dependencies’ dependencies hadn’t done so, I got the same errors.
Some more research suggested that I could delete the .ghc folder in my home folder, then re-install all Cabal packages. This would force them to rebuild. However, I encountered the same issues.
The Man In Black Fled Across The Desert…
I’m beginning to feel a bit like Roland, ceaselessly chasing after the Dark Tower. Every time I get there, my journey starts over again. I clear one roadblock, and there’s another there waiting for me.
It seems like there isn’t any real interest in calling Haskell from C, and I must say that I am extremely disappointed. Calling C from Haskell works great, but when asked to share its toys with C, Haskell takes its ball and goes home.
I’m sure it’s possible to do meaningful work in Haskell, and call that from C. However, the amount of work I would have to do to attain that goal is not something I’m willing to accept. For this reason I am shelving Haskell for the time being. Maybe I’ll pick it again for some other project, but it’s not a good fit for DMP Photo Booth.
C With Classes
No, I’m not taking an ignorant view on the nature of C++, I mean exactly that: C, with classes. “But C doesn’t have classes” you say. Technically you would be right.
One of the things I find great about C is that while it is very simple, you can build anything with it. An analogy that I’m fond of is Lego. If programming is like Lego bricks, then C is the basic tub of square bricks. They are simple, and universally useful.
Meanwhile C++, Java, C#, and whatever other fancy, “feature rich” langauge you can think of are like the specialized kits. They come with all sorts of fancy curved and bendy pieces. These pieces can be used to make some cool things but are only useful in very specific situations. If you’ve ever played with Lego, you know that the fancy kits still come with a few regular square bricks.
One might think you’d need the fancy slanted bricks to make a truly intricate model, but one would be wrong. If you scale things up, your basic square bricks can create just as intricate models as any fancy kit.
Back on the subject of programming, many large projects these days make heavy use of object-oriented features. Data is logically grouped into objects, which have behaviors. This model of programming is facilitated by “Object-Oriented” languages, which have these types of structures built into the language.
C does not have a class keyword, and classes are not built into the language. This doesn’t mean that you can’t do object-oriented programming in C.
What Constitutes a Class
Books have been written about this topic, and given the audience of a programming blog, you’ve probably already read one or two of them. The main goal of a class is to centralize related data and logic into a single unit, and to insulate the rest of the program from it. A class contains data describing what it is and what it does, and has methods that act on that data. The class keeps this data hidden as much as possible from the rest of the program. A printer class might need to keep a bunch of state data, but the rest of the program doesn’t need to know about that. All the rest of the program needs to know is if the printer class is ready to accept a job, and how to tell the printer class to print.
This is called separation of concerns, and it’s good because it prevents the user of a class from getting bogged down in the nitty-gritty of an operation.
Meanwhile, C has structs. A struct has no access control, and no methods. Anybody who has ever attempted to deal with something like termios know what a nightmare this can become.
I’m here to tell you that it doesn’t have to be this way.
What Could Be
What should object-oriented programming look like in C? To answer that question, let’s really break down the idea of a class. At the core, a class has three parts: methods, data, and access control. Data is state that a class maintains, methods are functions that act upon a class, and access control is how a class decides if the outside world should be able to see data or a method. All three of these things can be accomplished in C.
Data
This is the easiest part. Data can be grouped in C using a struct, and this is how we will be doing it.
Methods
Now things are getting a bit trickier. While C has function pointers, and these can be stored in a struct, this is not how we will be doing it. In most object-oriented languages, you can access a this within the body of a method to access data within the class. You can think of this as an implicit extra argument to the function. Similarly, to approximate methods within C, we will be using functions. The first argument to any “Method” should be a pointer to the data struct.
Access Control
This is accomplished by way of an “opaque struct”. That is, a struct that is forward declared in the header, but without a definition. This struct can be passed around, but its members can’t be directly accessed because they haven’t been forwardly declared. In this manner, we approximate private data. If we want our class to have public fields, we can forwardly declare a transparent struct. This struct should have a pointer to a second opaque struct that will serve as its private data.
A Contrived Example
This is all a lot clearer with an example. First let’s take a look at the header for class contrived:
typedef struct _contrived
{
int first;
struct c_priv * priv;
} contrived;
contrived * dmp_contrived_new(int first,
int second,
int third);
void dmp_contrived_free(contrived * to_free);
int dmp_contrived_sum(contrived * ths);
int dmp_contrived_get_second(contrived * ths);
void dmp_contrived_set_second(contrived * ths,
int to_set);First, we have our class struct. Class contrived has 1 public field, first, and a pointer to an opaque private struct. Users of this class can directly access first, but they can’t access anything within the private area.
Next we have a constructor and a destructor. If a user of the class wants to create or delete a contrived, they would not use malloc and friends, they would use the constructor and destructor. This solves another problem of C, complicated manual memory management.
After that, we have three methods: dmp_contrived_sum, a method on contrived, and a getter and setter for second, a private field. Similar to any other object-oriented language, we are free to break our encapsulation by providing getters and setters.
Now, let’s take a look at our implementation:
struct c_priv
{
int second;
int thrid;
};First we have the definition of the private area. Since this is only in the .c file, it is not visible to the rest of the program.
contrived * dmp_contrived_new(int first,
int second,
int third)
{
contrived * working = malloc(sizeof(contrived));
working->first = first;
working->priv = malloc(sizeof(struct c_priv));
working->priv->second = second;
working->priv->thrid = third;
return working;
}Next we have our constructor. It takes care of calls to malloc, and data initialization. When it’s done, it returns the newly created pointer.
void dmp_contrived_free(contrived * to_free)
{
free(to_free->priv);
free(to_free);
}The destructor ensures all memory is freed. This way, a user never needs to worry about freeing data. Since this pointer has a pointer in it, it would be very easy to forget the nested pointer and leak memory. This problem is solved by taking the responsibility away from the user.
int dmp_contrived_sum(contrived * ths)
{
return ths->first +
ths->priv->second +
ths->priv->thrid;
}
int dmp_contrived_get_second(contrived * ths)
{
return ths->priv->second;
}
void dmp_contrived_set_second(contrived * ths, int to_set)
{
ths->priv->second = to_set;
}Finally, we have our contrived methods. They deal with the nitty-gritty of struct manipulation, much like traditional class members.
You may have noticed that I do not have any private methods. This is also very easy to implement; just create a static function in the same .c file. This function will not be visible outside the file it resides in, making it private.
What About Inheritance?
This is also possible in C. Basically, you can just include a pointer to a base class object in a derived class object. There also frameworks such as GObject that can facilitate this in a more robust way.
However, if you find yourself making heavy use of inheritance, I cannot advocate the use of C over a traditional object-oriented language. “Inheritance” in C makes heavy use of void pointers, and this vastly degrades the type safety of your program. While it can be done, and even done safely if you try hard enough, I feel that it is no longer worth it. Object oriented languages with these features built into them will vastly improve your productivity at this point.
But if your program just makes use of a few of your own classes, and those provided by libraries, then there’s no reason you should feel obligated to use an object-oriented language. Indeed, for basic objects, C is more than enough for anybody.
DMP Photo Booth: Crunch Time
Over the last few months, I’ve become distracted. As anybody who’s ever committed to one project can probably tell you: it stops being exciting. What was once your pride and joy becomes the daily grind. Things get stale. As was the case with me, I suspect that this happens for most people when development of new features ends and the focus shifts to bug fixes.
I became distracted. My mind began to wander to the next thing, which in my case ended up being Haskell. I began learning Haskell, and it was definitely educational. I learned a lot with Haskell, and I plan to stick with it so that when I list it on my resume, I don’t get destroyed on a whiteboard test. Then came The Great Matrix Affair of 2014; I got overwhelmed at school. I spent so much time studying and doing homework that I couldn’t muster up the motivation to program. Things fell by the wayside, as you can see in my blog post output for February. Luckily for me, that is done, and I have the next two months free to program!
What Remains To Be Done?
It’s been a good 6 – 8 weeks since I’ve really focused on DMP Photo Booth, so the first order of business is the figure out what needs to get done. After doing some brainstorming I’ve settled on a list:
Finish The Trigger
I’ve mostly completed the trigger, but it doesn’t work. The button is soldered wrong, and while it was magically working for a while, it has since stopped. I need to fix the wiring issue, and then drill a hole in the box to put it through. After that and maybe a quick coat of paint it will be complete.
This particular task is due by Friday. I have a series of VIP demos coming up, the first of which is Saturday morning. A few of my cousins are coming in from out of town on Saturday to do wedding stuff, and I want to show it off then. While my cousin Allen is an engineer, and can appreciate a breadboard mockup of what will Totally Become A Real Thing, it certainly won’t be impressive. My cousin Laraine will likely be less amused, but I’m sure I’ll get a pat on the head for my “hard work”. Due to this, it’s important that the trigger be done before then.
Facebook Printer Module
The reference suite of modules was planned to be: a Trigger Module using my Arduino implementation, a Printer Module using CUPS, a Camera Module using LibGPhoto2, and a Lua module for each so that modules can be created using Lua. Of these, only the Lua Printer Module remains to be done. Since creating a Lua module is a trivial task (and not terribly important to be honest), this is a very low priority item.
However, the current Printer Module requires a physical printer. This might not always be ideal, since printers are big and heavy. What if you just want to bring a laptop and a camera and have a photo booth? My fiancée is having just this sort of situation; she wants to use the photo booth at her bachelorette party, but who wants to lug a huge printer to a hotel room? To solve this, I’ve promised her a Facebook Printer Module.
The idea is that when dmp_pm_print() is called, the photo strip will be uploaded to facebook. While I know this sort of thing can be done, I haven’t really researched it much. If it turns out that you have to pay facebook money to get this sort of access, I will find a hosting service that is free. Maybe Photobucket, or Dropbox, or whatever. The important thing is that the photo strips will end up in some central location on the internet so that anybody at the party can download the strip later. Ideally, this central location would be a facebook gallery so people can tag themselvs and be all Web 2.0.
My fiancée’s bachelorette party is in May, so this project isn’t a burning priority. However, this represents the most substantial addition of new functionality remaining to be done, so I plan to work it sooner rather than later. Code will be posted on my Github as it evolves, and like DMP Photo Booth will be available under the GPLv3.
Mac Support
To this point, all my development has been done in Linux. First using Ubuntu, and now using Debian. However, most people don’t use Linux. While Linux is the main target platform for DMP Photo Booth, I have been coding as portably as possible, so it should be little effort to port the application to Mac. Over the next few months, I’ll be making sure it compiles and runs correctly my old Macbook. My Macbook is vintage 2010, as such it is only running Snow Leopard. Therefore if anybody in the audience is a Mac user, and has a current version of OSX, feel free to give it a shot as well and let me know how it goes.
Ideally, my fiancée will bringing the Macbook to her bachelorette party and not my development laptop, therefore this project is due at the same time as the Facebook Printer Module, in May.
Bugs!
Finally, bugs. I need to identify them. I need to squash them. And I need unit tests.
After making a big show about being a good person and doing unit tests, I promptly lost the faith. Soon after implementing those first tests, I made a major change to how I handled errors in my code. Suddenly, all my tests were broken, and I was faced with the choice: rewrite them all, or delete them. After some thought I decided that my tests weren’t that great and that I’d probably change something again and break them all again. So I deleted them.
I’ve got to say, I miss those tests. There has been more than a few times where I’d refactored something and wasn’t sure if it still worked. Sure, it seems to work, but does it really? If I had unit tests in place, I could say with a greater degree of certainty that they do. Plus, “it sounds like a lot of work” is not a good reason not to do something, so it won’t be one for me.
On top of that, I’ll be identifying and squashing bugs the old fashioned way: by trying stuff. I’ve already found a few doozies, and I’m sure I’ll find more. As I find them I’m going to post them on the bug tracker for DMP Photo Booth on Github. I do this for a few reasons: 1) it will provide a public centralized repository of open issues so that I don’t lose or forget about them. 2) I would like others to post bugs here. If I post them here and show that I am fixing them, I feel it will establish confidence that it is looked at and acted upon.
This is due when DMP Photo Booth goes to version 1.0. That is planned to be on June 21, the day of my wedding. DMP Photo Booth will get a good solid night of work as the 80 or so people attending put it through its paces. Assuming all goes to plan with minimal issues, DMP Photo Booth will be declared to be out of Beta. That said, to be truly version 1.0, unit tests must be in place and “all bugs must be fixed”.
Packaging
Currently, DMP Photo Booth is available in source form only. No end-user ever had to compile Microsoft Office; I don’t feel they should have to compile DMP Photo Booth.
To that end, binary distributions will be available for Linux and Mac when DMP Photo Booth goes to version 1.0.
Got My Work Cut Out For Me
It’s a long list to be sure, but I have 4 months to focus on it. However, I’ve decided that I should spend some time focusing on other things too, so that I don’t burn out. To that end, I plan to spend 1 day per week focusing on learning new technologies, and 1 day per week keeping my old skills sharp.
For new technologies, this means things like learning more Haskell, and other languages or frameworks. Whatever strikes my fancy. For old skills this means practicing with Lua and Java, and maybe even C++ if I’m feeling particularly masochistic that day. This will likely take the form of blog posts on topics relating to these, so stay tuned!
The Smallest Things…
For the last few weeks I’ve been banging my head against a problem. I need my Photo Booth application to actually take a photo. It seems like such a simple thing, but it has actually been one of the most difficult ones I’ve encountered. Like the Great PostScript Debacle and the Mystery of the GAsyncQueue Ref before it, I spent a good week banging my head against the wall. I even took a day off to write an entire Lua Camera Module just so I could shell out and call a command line utility to try to work around it.
The best part? If you’ve been following the blog, you know I’m kind of a crybaby about poor documentation. While libgphoto2 is certainly a repeat offender on this count, no amount of documentation could have prepared me for what was to come.
But first, let’s go over my now-working implementation.
Taking A Picture With Libgphoto2
To take a picture, we need 3 functions:
gint dmp_cm_camera_init()gint dmp_cm_camera_finalize()gint dmp_cm_camera_capture(gchar * location)
dmp_cm_camera_init
gint dmp_cm_camera_init()
{
context = gp_context_new();
gp_log_add_func(GP_LOG_ERROR,
(GPLogFunc) dmp_cm_log_func,
NULL);
if (gp_camera_new(&camera) != GP_OK)
{
//error handling
}
if (gp_camera_init(camera, context) != GP_OK)
{
//error handling
}
return DMP_PB_SUCCESS;
}There are two main structs: Camera and GPContext. A Camera represents, shockingly, a camera attached to the system. A GPContext represents work to be done. Callback functions, data, and other things of that nature.
First we create a new context. Next we can add a log function to accept log messages from libgphoto2. In my experience, no matter what you do you will get a lot of useless garbage output from libgphoto2. For this reason, I recommend you don’t just let this spew to the console or some other user-facing output. At first, I was going to send this to the console queue, but I’ve since decided against using this feature. It is good to know about though in case you need it for troubleshooting.
After all of that is done, we need to create our camera object, and initialize libgphoto2.
dmp_cm_camera_finalize
gint dmp_cm_camera_finalize()
{
gp_camera_unref(camera);
gp_context_unref(context);
return DMP_PB_SUCCESS;
}Nothing particularly tricky there. We need to ensure we free our memory when we’re done, so we unref our camera and context. Having seen these two functions, you may be wondering to yourself: “Are we dealing with GObjects here?” Luckily for us, there is a simple test for this:
g_assert(G_IS_OBJECT(camera));I’ll spare you the effort of running this test: the assertion fails. Too bad really, but it is what it is. Libgphoto2 just uses function names similar to GObject.
dmp_cm_camera_capture
gint dmp_cm_camera_capture(gchar * location)
{
CameraFile * file;
CameraFilePath camera_file_path;
gint fd;
CameraEventType event_type;
void * event_data;
if (gp_camera_capture(camera,
GP_CAPTURE_IMAGE,
&camera_file_path,
context) != GP_OK)
{
//error handling
}
if ((fd = g_open(location, O_CREAT | O_WRONLY,
0644)) == -1)
{
//error handling
}
do
{
gp_camera_wait_for_event(camera,
1000,
&event_type,
&event_data,
context);
if (event_type ==
GP_EVENT_CAPTURE_COMPLETE) break;
}
while(event_type != GP_EVENT_TIMEOUT);
if (gp_file_new_from_fd(&file, fd) != GP_OK)
{
//error handling
}
do
{
gp_camera_wait_for_event(camera,
1000,
&event_type,
&event_data,
context);
}
while(event_type != GP_EVENT_TIMEOUT);
if (gp_camera_file_get(camera,
camera_file_path.folder,
camera_file_path.name,
GP_FILE_TYPE_NORMAL,
file,
context) != GP_OK)
{
//error handling
}
if (gp_camera_file_delete(camera,
camera_file_path.folder,
camera_file_path.name,
context) != GP_OK)
{
//error handling
}
gp_file_free(file);
do
{
gp_camera_wait_for_event(camera,
1000,
&event_type,
&event_data,
context);
}
while(event_type != GP_EVENT_TIMEOUT);
return DMP_PB_SUCCESS;
}This function is where the meat of the process is. First we need to do some housekeeping. We create a CameraFile pointer to represent the actual image file, and a CameraFilePath struct to represent the path to the file. We also create an int for use as a file descriptor, a CameraEventType and void pointer for our calls to gp_cmaera_wait_for_event
Next we call gp_camera_capture which triggers the camera to take a picture. After that is done, we’ll open a file descriptor to save the image. You’ll notice that the call to g_open is enclosed in parentheses. THIS STEP IS 100% MANDATORY Don’t omit it, you’ll be sorry. More on this in a bit.
Next, we wait for the camera to finish working. The camera uses an event system; it will emit events when things happen. After releasing the shutter, the camera has other work to do before it is “done taking the picture”. If you try to do the next step before the camera is ready libgphoto2 will spew garbage to your STDOUT and you’ll have to ctrl+c to fix it. To avoid this, we call gp_camera_wait_for_event while event_type != GP_EVENT_TIMEOUT || GP_EVENT_CAPTURE_COMPLETE Capture complete is obviously the event we care about, but it may have happened while we weren’t listening for it. In that case, we’ll settle for a timeout.
Next up is instantiating our CameraFile. We use our File descriptor that we just opened to call gp_file_new_from_fd. Unfortunately there is no gp_file_new_from_file_pointer which means that this call is POSIX only, and there’s no portable substitute.
After creating our CameraFile we download the image we just took by calling gp_camera_file_get and then delete the file from the camera using gp_camera_file_delete
Finally we make sure no events are pending, then return.
Why Are You Yelling At Me?
Good question. The block in question of course is
if ((fd = g_open(location, O_CREAT | O_WRONLY,
0644)) == -1)
{
//error handling
}Inside of that if block, I’m assigning a value and testing the result inside of the if statement. This operation is about a 2 out of 10 on the cleverness scale. Normally, you could omit the parentheses around (fd = g_open(location, O_CREAT | O_WRONLY, 0644). However, if we do it here, things go off the rails. Not right away, of course, but a few function calls later we get to:
if (gp_camera_file_get(camera,
camera_file_path.folder,
camera_file_path.name,
GP_FILE_TYPE_NORMAL,
file,
context) != GP_OK)
{
//error handling
}As soon as gp_camera_file_get(...) is evaluated, this is spewed to the console:
…and you have no choice but to kill the process.
Why does this happen? I have no idea. Why does enclosing the call to g_open in parenthesis fix it? Again, no idea. And it only happens here too. I just tried to modify the examples that come with libgphoto2 to reproduce the error and get that screenshot for this post, but it works fine there. Knowing my luck, if you download and build the program, it’ll work fine for you.
As long as it works, I guess…
Doing My Programming... 