Shedding commercial attitudes towards documentation
Off the Beat: Bruce Byfield's Blog
Six years ago, I made my living as a technical writer. I wouldn't want to return to the profession, but, when Esther Schindler recently blogged about the importance of detailed code comments and Carla Schroder about the need for better documentation in free software generally, I noticed.
But, as much as I agree with Schindler and Schroder, I wonder how much of the community is about to give it the attention it deserves. Too much of the community still seems to cling to attitude about documentation inherited from commercial development.
Attitudes to writers (and why they deserve them)
Part of the problem is that technical writers are held in low regard. There is a simple reason for this reputation: Most technical writers know nothing about technology, and the documentation they produce is worthless. The average technical writer is an English major, and is as apt to be attached to the marketing department of a company as to development or quality assurance. In other words, they have little feel for technology, and are encouraged to think of what they produce as a cosmetic extra rather than a necessity.
Of course, technical writers could overcome these handicaps if they wanted. I have an English degree myself, yet I had few problems finding work because I realized early that I needed an expert understanding of what I wrote about.
But this is a minority opinion. The majority of technical writers are under the illusion that all they need are language skills. They see no need to know technology and code. If anything, they look down on this knowledge as less important than grammar and spelling.
With this attitude, the documentation that most technical writers produce may be clear and concise, but it is also superficial and incomplete, and displays no more than a rudimentary knowledge of the software or hardware they are writing about.
So, naturally, developers despise writers and what they produce. In the eight years that I was a technical writer, I got used to having to prove my competence at the start of each new contract. Experience had taught me that other writers had muddied the waters before me.
Just in time documentation
At the same time, commercial developers and project leaders have very little sense of how useful documentation is produced. For a long time, pundits on the subject of technical writing have insisted that writers have to be involved in the development process as early as possible. Yet, in my experience, not one company in six has adopted this practice.
Instead, documentation begins near the end of the development process. Often, it begins ridiculously late -- I have worked on projects in which I was expected to document software with a dozen modules in less than a week, and the best I could do was triage.
Yet, even when the time allotted for documentation is more reasonable, bringing writers in near the end of the project rarely works. Already pushing deadline, the writers are under still more pressure because they have to learn about what they are documenting. Even if they only learn the bare minimum that most writers settle for, their productivity is impaired. To do a proper job and actually understand their subject is next to impossible.
Moreover, at the same time that writers are gearing up, developers -- their main source of information -- are winding down. Often, the developers feel that educating the writers is not their job, or they are ready to relax and disinclined to have writers pick their brains. At times, they go on vacation, leaving writers to struggle on alone.
These same attitudes apply when developers do their own documentation in the form of comments or FAQs. Just like a student writing an essay, who pounds out the bibliography as an afterthought, the average commercial developer writes documentation only after their main work is done, expending minimal thought and less effort, and not thinking how it might be useful to others.
In other words, the prevailing attitude in commercial development is that documentation is of secondary importance. Developers might be fond of advising people to RTFM (Read the Fucking Manual), but everybody still jokes that nobody really reads documentation. The marketing department might want it, but everyone knows that those drones are obsessed with the superficial, right?
Searching for a free software alternative
The division between commercial and free software is not firm. Most free software developers have also worked on commercial software. In fact, these days, free and commercial software can even be the same work. One of the results of this inter-connection is that much of the free software community views documentation as an extra and not a part of their job.
True, many projects, including Drupal, Fedora, and KDE, have struggled to change this attitude. But at times the old commercial attitudes still seem to faintly linger. How often, for example, do you see writers blogging alongside developers on Planet feeds for major projects? At least to some extent, writers and other non-developers still seem to have less status than coders, and to receive less credit as well. Unofficially, writers often seem less part of the community than developers.
This inheritance from the commercial world is not only counter-productive, but needless. Free software technical writers may not be coders any more than their commercial counterparts are, but, given that they are involved in free software, they are unlikely to think they don't need to understand the project they are working on. And, unlike in the commercial world, where technical writing is the last refuge of English majors who can't find a teaching gig, free software technical writers presumably like what they are writing about.
Similarly, despite the move towards twice-yearly releases in many large projects, there is no reason for documentation to be a rushed effort at the last minute. So long as writers can compile source code, they can learn about the software while it is being revised. Nor, in most cases, is there any reason why a release couldn't be delayed until the documentation is ready.
In the same way that the free software has produced superior applications, its projects are positioned to produce superior documentation as well. But, to do so, projects have to shed attitudes that do not apply to their situation. The only question is how many are willing to do so.
online marketingAffiliate Marketing is a performance based sales technique used by companies to expand their reach into the internet at low costs. This commission based program allows affiliate marketers to place ads on their websites or other advertising efforts such as email distribution in exchange for payment of a small commission when a sale results.
hai friendAffiliate Marketing On The Internet
Affiliate Marketing is a performance based sales technique used by companies to expand their reach into the internet at low costs. This commission based program allows affiliate marketers to place ads on their websites or other advertising efforts such as email distribution in exchange for payment of a small commission when a sale results.
online markettingAffiliate Marketing is a performance based sales technique used by companies to expand their reach into the internet at low costs. This commission based program allows affiliate marketers to place ads on their websites or other advertising efforts such as email distribution in exchange for payment of a small commission when a sale results.
floss manualshave you looked at FLOSS Manuals? come join us! we are a community of 1500 people writing free docs about free software. its odd you have not heard about us since we collaborate with the fsf, google, mozilla, civicrm, sugar, etc etc etc
We have been going now for about 2 years and we have many many excellent docs about free software mostly in English (http://en.flossmanuals.net) but also we have an Finnish community (http://fi.flossmanuals.net) and a Farsi community (http://fa.flossmanuals.net) and also many manuals being translated into various languages (see http://translate.flossmanuals.net/write).
for the linux community there is much of interest, but probably most interesting is the excellent introduction to the command line manual which we produced in collaboration with the FSF:
You can buy it here:
We produce community authored content and we often do this in 2-5 days. For example the following were produced in 5 days :
and the following were written in 2 days:
We also produce books. All of the above are available in printed form (see bookstore on the right of the front page).
We also have produced books in other languages including the recent 'How to Bypass Internet Censorship' book ( http://objavi.flossmanuals....nslate-2009.12.01-12.02.55.pdf ) which was printed in Arabic for the recent Arabic Bloggers conference (http://advocacy.globalvoice...05/2nd-arab-bloggers-meeting/) held in Beirut.
General Issue - Many Find it Hard to See Things From Others PerspectiveHaving been in commercial development, and had experience one time of my documentation efforts, being "improved" by a technical author to make things flow; this was very frustrating as without necessary application knowledge the author changed the meaning of things, in very carefully constructed technical phrases which aimed at non-wordy brevity & clarity (influed by UNIX man(1) style). The Documenation was aimed at Electrical Engineers, of course I then ended up with more support issues and complaints about the manual, because it was b***sed up by the contracted specialist Author. I would simply accidentally send a copy of the original, which would generally clarify things; though they wanted (of course) more examples, more deeply worked (hard for me as I wasn't an expert in what they were doing). A better answer to me, would have been to spend time with one of the clueful end users rather than technical author and get the reference material sorted that way.
The General problem, and there's been a lot of slagging of "coders" as illiterate unable to write English gone on in these Documentation threads, is the difficulty people have of understanding and putting themselves in other's position. Perfect Docs. for one user, is poor for another; those complaining about UNIX man pages not making sense, to me simply show laziness and lack of effort they normally make perfect sense to those familiar with UNIX concepts.
MS Windows documentation & help, available on the machine, is to my mind an example of outstandingly poor work, it just repeats the obvious, normally what is on the screen; and one has to access Google & Knowledge Base for true enlightenment. Why repeat what the application is telling you anyway, and give no more information than what someone with a clue can easily extrapolate.
Different documentation level templatesWriting technical documentation is hard to do right, because you must consider who will read it and for what purpose. Last, but by no means least - what are they doing when they read it?
A good friend tried to explain how a large network was connected using drawings connected to further documentation below. The picture showed the components, but failed completely at showing the true relationship between them. Using the standard network "lightning bolt" between the firewalls failed, because it did not show in which direction traffic was allowed between the components. The drawing explained very little to new network engineers in the company, but it had pretty colours and made the administration happy.
Talking about code or code comments as documentation is rubbish. The code is one of several possible ways of explaining to the computer what you want it to do for you. Documentation is what the computer needs from you to do it. We write documentation for our future selves or for other people. We write documentation to explain our intent, not what the code actually does - which might be faulty. If it does what we intended it to do, there is no need to look at the code at all. However, this is not true for the documentation, because our intent must be explained to new users over and over again until they know what to do.
I am willing to argue the true reason for the success of projects like MySQL and PHP is the documentation. This is true of companies like Cisco in the networking industry. All of these allow users to comment on every single aspect of the products. Use them as templates for how to do it right, borrow their structure - a short introduction about the intent, a short tutorial, and then more advanced stuff, and finally references if any.
I'm running Kubuntu 9.10 Netbook remix right now. Please, do this short exercise: find the documentation that explains how to add and remove your favourite applications from the Favourites container on the plasma workspace. I'll explain it first, so you have several clues you can use to find the documentation: click on the yellow star on the program icon to add the software, click on the red minus sign to remove it from the Favourites container.
The best of luck!
No difference between commercial and open sourceI have worked as a systems engineer for 25 years on both commercial and open source projects. Documentation is always a problem and the blame can be spread around equally. From developers with poor writing skills to technical writers and managers with poor technical knowledge. I know very few software/hardware developers/engineers who have decent writing skills (myself included.) In most cases documentation is an afterthought because this is not what developers/engineers like to do. They do the bare minimum needed while working on the project and at the end have lost interest and would rather start something new and much more interesting than documentation. Commercial projects usually have an advantage of a budget and can attach a technical writer to the project in the early stages. Open source projects sometimes have better documentation because this is where the developers can 'show off' their product. When it comes to making deadlines, documentation has low priority - no point in documenting something that doesn't work. It is really up to the project manager to force the issue - making sure they know the project is not finished until the documentation is also finished.
primary documentation to developersTo most developers, the primary documentation is the source code. This describes exactly and precisely what the code is, what it does, and how it does it. Unfortunately, it is written in an artificial language comprehensible only to developers and software tools.
Secondary documentation is the comments in the source code, which is dismal. Most of the time, it is not there, or is a rehash of the program code in pseudo-english, and is often wrong, it does not accurately describe the real code.
Knuth tried, inventing literate programming, Sun tried, with javadoc sections in source code files. Most programmers are simply not literate and articulate enough to write lucid, accurate descriptions.
Someone should have taught them how to write English (or Chinese or Hindi or whatever human language they use) declarative sentences and paragraphs when they were about 12 years old. Now it is too late, the only grammatical tense they can speak or write is second person imperative: (computer) do this !
FOSS documentationIt's true that FOSS software doesn't differ that much from commercial software. Good documentation is hard to get apart from major projects like PHP, MySQL and the lot. I know that - although I liked Forth - I had to buy a ton of books to do anything but the most trivial thing. That is: both commercial and FOSS. On the other hand, TurboC v2.x was decently documented although it did not come with a tutorial - which can be defended in some way.
When I started my 4tH project I was dedicated to produce excellent documentation on each and every level, because it would boost the usefulness of the project. I got comments from my peers and produced a generalized Forth tutorial which still comes with several other commercial and FOSS products on the subject. It did even inspire others to make similar documentation. A good example makes people follow you in that respect, I think.
I know that people producing documentation are not in high regard, which is unjust, because making good documentation is an art, just like software. It would absolutely help the cause of FOSS if they would do better in that regard.
As far as I'm concerned, no release EVER came with an updated manual, but then again I can choose when I make a new release. When making software at work, it at least has user documentation - time can get tight - but the rest WILL follow. No product can simply do without.
The wrong approach to (technical) documentation5 times ACK! I have pointed this out many times myself, and I have still no clue why projects and companies won't implement the process of documenting as a tool to improve quality. Note: If you encounter problems when documenting your code/app/device in a proper way, this usually points to problems within the implementation.
Mozilla’s product think tank sinks silently into history.
TODO group will focus on open source tools in large-scale environments.
New tool will look like GParted but support a wider range of storage technologies.
New public key pinning feature will help prevent man-in-the-middle attacks.
Carnegie Mellon researchers say 3 million pages could fall down the phishing hole in the next year.
The US government rolls new best-practice rules for protecting SSH.
Klaus Knopper announces the latest version of his iconic Live Linux system.
DARPA and NICTA release the code for the ultra-secure microkernel system used in aerial drones.