Appledoc became quite popular amongs Cocoa developers, especially since 2.0. It also seen many contributions from various users. But it has become very hard to maintain. In this blog post I’ll describe the reasons and future directions.

Introduction

Appledoc 1.0 was created after a similar Pyhton tool doxyclean. It used doxygen to create intermediate xml files and then transforming these into html. But using doxygen proved to be a bottleneck, especially due to it’s bad support for Objective C.

To make it better, I first checked doxygen source code, but it seemed too messy for my taste, so I decided to create my own subset of Objective C parser. I always liked programming parsers, so it seemed like fun if nothing else :)

Another issue when preparing for 2.0, was how to make source code documentation simpler to create. One solution that stood out of the rest was Markdown. It allows clean and easily readable documentation and control of generated output, so it seemed obvious choice.

The problem

In order to create usable documentation, the tool that compiles it needs quite a bit control over the process - i.e. the “hooks” that are called during various stages so that it can customize the output. Therefore appledoc 2.0 featured it’s custom Objective C and Markdown parsers, completely custom built. Although this nicely covered initial features, expanding the feature set brought it to its limits.

On the Markdown level, compiling source code documentation was fine, but after introducing “static” documents, limitations became obvious - there was no support for images and many other useful constructs. I was standing before a choice: either add all those to appledoc processing code or use third party Markdown processor. I opted for later and following user’s suggestions and digging for a while, settled on Discount. It nicely boosted functionality, but also brought new problems. Most significantly: how to handle appledoc cross references. To deal with these, appledoc needs to pre-process text, detect cross references and convert them to standard Markdown link notation. So far so good, but this must not be done inside code blocks for example - Markdown doesn’t process code there! So appledoc would require a hook from Discount, messaging it when it starts processing some part of text as code block and when it ends. But due to discount being more or less closed library - you simply give it Markdown formatted string and it spits out html counterpart - there are no such hooks appledoc. To circumvent, I used several hacks, but they resulted in various bugs. I think these are more or less cleared by now, but there’s door wide open for improvements. To make the code really flexible, it would have to re-implement partial Markdown parsing. But that seems like an overkill if there’s already build in Markdown processor…

Objective C parsing code is also becoming hard to maintain and extend. As appledoc only needs to parse a subset of Objective C, using fully fledged parser would seem like an overkill. But it must be able to detect and handle various formatting styles as the minimum. Although parsing itself isn’t that hard, there’s another side of the coin with handling parsed data (for example merging from header and implementation files). As a consequence, there were several bugs related to parsing code. Some due to me forgetting implementing certain aspect, some due to decisions I made. The net result is extending the tool for one of the single most requested feature would require substantial refactoring of the code. Which would most likely bring whole set of new issues with it, possibly also reopening things already fixed.

So as you can see, appledoc has been through some major refactorings already, and it’s starting to feel like patched pieces of code thrown together in haste…

The solution

Facing above mentioned challenges and my very limited time at this point, I was thinking hard about possible solutions. At the end, I’m leaning towards a substantial refactoring of the appledoc source code. In the mean time, I discovered different Markdown parsers, which not only allow generation of html, but also provide hooks that are called during parsing. For example libupskirt or its more known fork sundown. They don’t seem to be as feature rich as Discount (in terms of Markdown extensions), but they would play much nices with a tool like appledoc.

I was also checking out libclang for doing parsing, but it seemed too complicated for my simple usage. I currently have a short window of time and am experimenting with a different approach to custom parsing. I’ve come up with a nice solution that works well and is nicely structured.

And finally (but perhaps most important of them all) - in the mean time I became more experienced developer. Appledoc was one of my first serious endeavors to Objective C language. Consequently I was not only learning the new syntax but also underlying functionality and libraries. That and new APIs available in newer releases of OS X, give me more tools to complete my job.

Conclusion

I will continue to update appledoc, but will concentrate on above mentioned update. Consequently I might not do as much maintence to current branch but instead focus on larger picture layed out above. It may take quite a bit to see the light of day due to my time constraints though, I’ll do my best :) The good news is - you can follow along: I plan to create a separate branch on appledoc GitHub repository which will be updated with new code as it comes.

I might also publish smaller experimental spikes as separate repositories (for example just the parsing code). I will mention this either by linking on appledoc issues related to the solution or Twitter and/or this blog. I’d appreciate your feedback on those so that I can spot problems early on act on them.

Thanks for all the feedback you’ve provided so far and looking forward to hear more from you - together we can make it better!