A recent article in First Monday describes five things that are often wrong with open source projects. Beautifully stated, and a nice summary of why and how so much open source software is difficult to use. (Mark Surman reminded me about First Monday, a peer-reviewed electronic journal covering technology and society. It's worth reading.) There's no inherent reason that open source software can't be robust and user-friendly, though – it depends on the approach taken by the software developers.

Today in ZDNet, there's an article about 'shared source' software. In a new initiative called Avalanche, large corporations are joining together to share software upgrades and customizations to reduce the cost of managing software. This is an interesting model for nonprofits as well; it could make sense for a consortium to join this group (at a cost of $30,000 US/year) on behalf of nonprofit organizations.

 Posted on Charity Village April 7, 2004.

Most nonprofits providing human services get financial support from some formal funding body such as government, a foundation or a centralized fundraiser like the United Way. All of these funders are beginning to demand evidence that their financial support is worthwhile. Each one of them has a different set of priorities (such as effectiveness or community impact or cost efficiency), but most of them are demanding more data than ever before.

The effect of this pressure isn't entirely positive:

• Very few agencies have a good management information process that tracks service delivery. Almost none have an information system that evaluates the effectiveness of their services. As a result, data quality is generally poor or nonexistent. Funders shouldn't rely on the data for decision-making, and in fact the most sophisticated and truthful agencies may be penalized for submitting data based on reality rather than crude estimates.
• Every funder has different priorities and asks different questions. Even when there are overlaps of information (like annual revenue or agency address), every funding form is different. The increased demand for data is creating huge administrative burdens on agencies. Data collection and reporting is cutting into service delivery time without necessarily leading to improvements in service quality.
• Few funders know how to handle the data they do collect. Many funders collect data that they think might be important, but don't use the information to make better decisions. The cost of collecting and analyzing data is not acknowledged, so a whole lot of useless information is collected and filed.

Many agencies are developing management information systems that collect client data in an effort to meet funder requirements. This is a good thing, in principle. It is also a good thing that funders are determined to increase the effectiveness of the services they support. However, if not well designed, information and evaluation systems can cause more problems than they solve.

Deciding what data to collect

Before you develop a data reporting system, you need to figure out what data you should be collecting. Here are a few tips:

1. Start out by ignoring funder requirements (for a while), and just define the information that will be most helpful in carrying out your agency's mission. I like the vocabulary of key performance indicators, which involves defining the most important indicators that tell you whether you are doing a good job, and that will give you the feedback that will enable you to improve. Other people use balanced scorecards or program logic models for the same process. There's a lot of evaluation jargon that makes this whole area very complicated and difficult to understand, but basically, you have to figure out what you are trying to achieve, what success looks like, how you tell the difference between success and failure in the short term, and which information will be the most helpful in keeping you on track.
2. As a second step, define the data that your funders require. This is not always the same as the data that will make you more effective. For example, your funder may want to know how many individual clients you serve, even though you primarily do community development or advocacy. Or you may need to collect data on your human resources to manage your agency effectively, but funders don't need that kind of detail. Try to define the funders' requirements as broadly as possible, so that one data element can meet the needs of many different funders. Some funders are extremely legalistic in their definitions of acceptable data elements. Those legalistic definitions may be good enough to use as data elements for other funders that aren't so picky. Or you may be able to negotiate the definitions with funders if their data requests are unreasonable.
3. Figure out the simplest way to define and collect both kinds of data – funder requirements and key performance indicators. For example, do not send out surveys to all of your clients if you are just trying to collect information on service quality. Think about how you can design a suggestion process and a complaints resolution process to collect and resolve relevant information about your services. Supplement it with phone interviews with a small sample of your clients. Don't create a client management system that tries to collect everything you might possibly need – it's way too expensive and will not lead to better services unless you have built in processes to analyse and act on the resulting information.
4. Design a process that allows you to collect and analyse your data while minimizing administrative costs. For example, say that you need to report various bits of financial data to most of your funders. Instead of manually copying numbers from your annual report, perhaps you can automatically export numbers from a financial database into report templates for each of your major funders. This step generally involves going through a process improvement exercise. It's a common mistake for agencies to create an information system by designing a database that replicates all of the bad, inefficient processes that they already use. The design of a management system offers an opportunity to redesign the way that the agency collects information, to dramatically reduce errors, and to increase the quality of the data, and reduce the cost of administration.
5. Finally, you can design a management information/data tracking system that will make sense and actually help you improve services. If you haven't gone through the steps above, you will almost certainly end up with a system that is far more expensive and far less useful than it should be. And by the way, this doesn't have to be a big expensive software development project. You can have a beautiful, effective information system using a simple spreadsheet or even paper forms; technology is just a way to facilitate processes. A database can be very useful, but only if you're collecting the right information in the right way.

RealWorld Systems is currently involved in helping funders and agencies identify key performance indicators to guide data collection. We are hoping that over the next few years, funders will develop a common data dictionary and a common data protocol that will dramatically reduce administrative time in collecting and reporting data. But that's another article.

Process improvement

Here are some comments on process improvement from Lori Criss Powers, an expert in this area who is also my business partner at RealWorld Systems. Her writing is rather more technical than mine, so brace yourselves. I quote:

Don't assume you will have massive efficiency gains when you develop your new client management system unless you undertake a process improvement exercise while you implement. If you don't, your actual efficiency gains may be small because the technical solution was superimposed on top of manual processes and/or because process redundancies were never eliminated (e.g., people continue to photocopy and route paper documents from desk to desk even though an electronic version of the data is available because they are most comfortable using paper). Sometimes barriers to efficiency are not obvious and a quick process analysis does not uncover them. Sometimes problems are identified but are ignored under the pressure to implement a solution. All this may mean that people end up actually adding to their workloads to accommodate the technology.

It is quite common for an organization undertaking such an initiative to suffer from an 'information junkie' condition in which unnecessary data is collected (defined as data that neither leads to improved services nor meets core accountability requirements). Unnecessary data costs a great deal of time and expense to collect, store and analyse, and affects staff morale because data collection is so unpleasant. It also decreases data quality because unnecessary data lacks legitimacy in the organization, and staff have little incentive to ensure that it is accurate. Conversely, without clarity on data requirements, essential information may not be collected because people aren't exactly sure what is important and what is not.

A process is a series of interrelated activities which bring about a result or which are directed toward a particular aim. Process improvement is the analysis and redesign of processes to eliminate organizational problems and inefficiencies in small increments, over time, by improving one or two processes at a time.

Process improvement is done on an operational level (as opposed to radical re-engineering which is done on a strategic level) and is carried out primarily by the people most involved in the process.

The incremental process improvement approach builds in small successes that motivate teams to continue. Failure, if it occurs, has less potential to do serious damage because scope is limited to one or two processes at a time. Incremental process improvement can be a good way to realize (sometimes substantial) benefits in quality and organizational performance while bringing people together over a common goal. It can stimulate morale and can inspire employees to look for innovative ways to deal with challenges.

In future articles, we may get further into specifics on process improvement and the development of useful client management/information systems. Please send us questions if you'd like us to address any particular issues.

**********
Gillian Kerr, Ph.D., C.Psych.
Lori Powers
RealWorld Systems

gkerr or lpowers at realworldsystems.net

All of the consultants on RealWorld Systems projects work out of home or satellite offices – we don't have a central office, and many of us have never met despite several years of working together. We're linked together by various distance collaboration tools, mainly email, phone conferences, instant messaging, and a private extranet.

For years, I've been looking for an application that makes project file sharing easier. A consulting project might involve 10 different team members, each generating dozens of documents and files. By the end of a complex project, there may be hundreds of documents, culminating in a set of final reports. Our intranet, which is based on Microsoft Sharepoint Services, is excellent but it’s not ideal for sharing working files. It’s a hassle to upload documents, and we only do it for the most important ones that everyone needs to use as references. Typically, we don’t upload the working documents that are revised by many people over a period of days or weeks.

Like most collaborative groups, we send documents back and forth by email, resulting in massive email storage and a lot of confusion about which documents are current. I’ve tried many solutions, including Groove (which has just released version 3), Sharepoint Services, Critical Path, Punch Networks, Xdrive, Tenix, ReBOL, and so on, and all of them have fatal flaws for our purposes.

• Some only work on Windows XP or higher, which leaves out team members who use different operating systems.

• Some take up massive amounts of computer memory. I really like Groove, for example, but my team flatly refuses to use it because it slows down their system too much.

• Some are so complex they interfere with the running of other programs (in technical terms, they screw up the Windows registry) even after they are uninstalled. Groove is another culprit here.

• Most require the laborious uploading of documents onto an online service. Even if the process is fairly easy and quick, it just doesn’t seem to get done when people are in a hurry.

Ideally, a file sharing application should work on different operating systems (Windows 98, Windows XP, Linux and Mac), should enable secure encrypted file sharing, should be as easy to use as saving documents to your own drive, and should make group document revisions as simple as working on a Local Area Network in a regular office. It should also be lightweight and simple to uninstall.

I’ve recently been using Shinkuro, and I’m pretty excited about it. Shinkuro is a small software program that enables teams to securely share files, exchange instant messages, and do simple screen sharing. It works with the four operating systems listed above (though it requires a small workaround for Win98 systems), and it’s currently in free beta. [Update from May 10 2004: It looks like there may be problems with some Win98 systems, so it may not work with yours.] The developer, Jeff Kay, has promised that it will be reasonably priced even for nonprofits when it’s formally released.

Its best feature is its incredibly easy file sharing; you designate a folder on your computer for sharing, and every document that you drag into that folder is shared with your designated Shinkuro group. You can do free backups that way (automatically backing up files on another server by saving them into the shared folder), and you don’t have to remember to upload the files onto a separate extranet. The files are encrypted and sent through the internet in small packets, so you can share huge files without needing to send them as attachments in email messages. And when your team members make revisions on your documents, their revisions are stored as separate documents so your versions don’t get overwritten. It looks great, though we haven’t fully tested it, and it seems perfect for project document sharing.

Shinkuro also offers instant messaging and ‘presence awareness’ to team members. Presence awareness means the ability to see when team members are online and at their computers – it’s similar to working in a central office and being aware when colleagues are at their desks by peeking over the cubicle. With presence awareness, it’s much easier to exchange information quickly or set up ad hoc meetings. Shinkuro’s version, unlike many of the free IM services like MSN, Yahoo and AOL, offers totally private, encrypted instant messaging with only the people you have chosen to invite.

Optional but increasingly important in team collaboration is screen sharing, in which you can see what’s on another colleague’s computer (e.g., PowerPoint slides, a document or a live software program) and discuss it during a meeting. Shinkuro offers a simple form of screen sharing in which users can show what is on their screen to other members of the same private group. It’s useful for document revision, to make a presentation, to show a graphic, or to carry on a software training session. There are other services that do a better job of screen sharing (I’ve already talked about glance.net and ASAP on this blog) but it’s a nice little feature that saves time when you’re trying to explain something visually.

Besides Shinkuro, collaborative groups will still need a variety of tools including email and group teleconferencing. Email is simple; everyone working online already has an email program, and it’s possible to get good free email services that work just as well as the most expensive.

Group teleconferencing has been expensive until recently. Skype, a Voice over IP application, has just released a version that supports free encrypted teleconferencing for up to five participants. You need a headset, a computer and a highspeed internet connection, and then you can make unlimited long distance or group ‘phone calls’ for free. Skype is expanding quickly – seven months after its launch it has been downloaded over 10 million times and it's available in many languages, including Chinese. Apparently the voice quality is very good, though I haven’t tested it extensively. We use a stand-alone teleconference service by Enunciate because most people are still reluctant to use computer headsets for their phone calls. Skype offers presence awareness and instant messaging as well, and will soon (supposedly) be able to link to regular telephone networks for an extra charge.

Plone Improves Usability, Flexibility according to an eWeek review. Plone is an open source (i.e., free) web development program that's designed to be extremely easy and flexible. Version 2 includes more work flow processes, better ways to link to databases, and is accessible to people with disabilities. Looks very good, and worthwhile considering for web sites or intranets. See plone.org for more information.