Cognetics - Papers - Charles B. Kreitzberg

This article explores the role of chaos in business computing and recommends how IS departments can improve the return on the investment (ROI) their companies make in computer technology.

Before the world began, according to ancient Greek mythology, there was chaos. In Chaos, everything was present but in a state of disorder. To those of us who have been living with the breakneck evolution of information technology over the past 15 years, chaos may seem like the most appropriate analogy for the current state of the field.

In some ways, chaos is a good analogy because like the ancient Greek myth, most of the pieces are present, but they are far from ordered or even stable.. Asked to describe the state of information technology, Manny Fernandez, CEO of the Gartner Group said:

We pay a price for this chaos. Software development is painful for both users and developers. And the product which results is less than ideal.

Perhaps it is too cheap a shot to say that the information technology industry is in chaos. Clearly some work is getting done in exchange for the $500 billion that is spent on IT in the United States annually.

And while the "productivity paradox: which suggested that investments in information technology were not yielding return was hot news a couple of years ago, since 1995 the payoff on information technology is becoming more apparent.

Still, any user and any IT professional will tell you that there are serious problems in the industry which we need to understand and respond to. To overcome the chaotic elements in information technology we need to understand their nature. With this understanding we can take steps to reduce them.

The data available on the nature of chaos is less rigorous and complete than would be ideal. Corporations have little time to measure their internal performance nor is there an outside agency that has this responsibility. But consulting groups periodically publish studies which shed some light on the problem. These studies suggest that we can characterize the chaos as centering on two areas:

The first area of concern is the software development process. Any programmer can tell you that software development is very difficult and any manager can tell you that it is slow and uncertain. My personal experience as a consultant has been that many projects move slowly, start and then need to be restarted, or run into serious technical difficulties. A 1995 study by the Standish Group, reported in Forbes and available on the web reveals just how bad this problem may be.

The Standish Group conducted surveys of 365 respondents representing a total of 8,380 applications. They looked at software development activities in small (annual revenue of $100 - $200MM), medium (annual revenue of $200 - $500 MM) and large-sized companies (annual revenue > $500MM).

Their major (and quite distressing) finding was that only 9%-16% of software development projects are completed on-time and on-budget. The 9% figure is for large companies (in which the average cost of a project is $2,322,000); the 16% figure is for medium companies whose projects average $1,331,000). Small companies did better, with 28% of their projects succeeding (average cost $434,000). One possible conclusion is that smaller projects are easier to manage and are therefore more successful, another is that small companies have better communications and therefore manage projects better. Whichever conclusion you choose, the results are not encouraging. This is surely a major reason that outsourcing has become so popular among IT departments.

Given the small number of successful projects, the question is what happens to the remainder, which constitute the vast majority of development efforts. According to the Standish Group study, 31% of software development projects are cancelled before completion. These are projects which were started and which represent lost opportunity and lost investment.

Another 52% of projects eventually do get completed but end up costing 189% of their original budget. In terms of time, they take between twice and three times as long as originally anticipated. Perhaps of more concern is the fact that many of these project end up as "weak visions" of the original concept implementing only 42% of the features originally intended (for large companies). This is of concern because features not implemented are business functions that are not available to the corporation. They are a strategic handicap and reduce productivity.

A comment from one of the focus groups is indicative of the chaos in project development. A project manager at an insurance company said:

To the extent that the development process is in chaos, it is not surprising that the products which result are difficult to use. Problems with software usability affect productivity and a corporation’s ability to respond to its customers.

Many software engineers and CIO’s are unaware that there exists a well-developed body of knowledge regarding software usability. Usability professionals are specialists in this area. Often drawn from the field of psychology, these individuals specialize in studying business process, designing the user interface and testing the quality of their designs so that flaws which impair usability can be corrected.

The myth that GUI’s have made software easy to use is dispelled by a monumental study by Thomas Landauer, published by MIT Press. Landauer looked at the interfaces of a large number of software products, using a technique known as meta-analysis (actually a study of studies). He found that the average user interface has 40 design flaws which can be identified by usability professionals through various quality control techniques. Of these design flaws, half may be classified as superficial and are easily corrected. The other half are structural, meaning that they are embedded in the navigational flow and data representation of the program and cannot be corrected without significant redesign and reprogramming. The superficial problems can therefore be corrected if action is taken before the software is released while the deep problems can only be addressed during a major revision cycle.

Landauer also looked at the return on correcting the design problems in the user interface. Correcting the 20 superficial problems in the interface, yielded an average productivity gain on the part of the end user of 50%.

If the remaining 20 design flaws were corrected, returns were an order of magnitude higher -- up to 720%. To achieve this type of improvement, user-centered design needed to be integrated into the software development process from the beginning.

There is remarkably little data available about end-user performance -- how people actually use computers in their jobs. Such data would be of great value because it would help software designers understand how serious the usability problems that face users are. One measure which has been refined since 1987 is the Gartner Group’s analysis of the total cost of ownership of computers (TCO).

Gartner suggests that the end user operations can be reduced by a factor of 40% if well thought out policies and procedures are in place. (The figures in this section were obtained from an article in Network Magazine, May 16, 1997 by Steve Steinke).

Another study of end-user operation was a survey of 6,000 workers conducted by SBT Accounting Systems in San Rafael, CA and reported in Forbes. This study found that users spend five hours a week "futzing" with their PCs. According to the report, the number one time-waster was waiting for programs to run, reports to print, repairmen to show up or technical support folks to pick up the telephone.

It seems clear from these studies that chaos in software development and in end-user computing is a serious problem. Addressing the problem is the key to increasing the return on information technology investment.

The antithesis of chaos is order. Some IT departments have attempted to reduce chaos by returning to highly centralized, even draconian control of information assets. While there are some environments in which such an approach may be successful, there are likely to be unfortunate side effects of achieving order with an iron fist. One such effect is user resentment. For many years, the relationship between IT and its users has been strained. Authoritarian approaches are not likely to promote a harmonious relationship.

For example, recently a client wanted to create a new analysis tool which it considered central to its business strategy. The IT department insisted that it alone had the authority to authorize new software. When the division asked for an estimate of when the software could be designed, the IT department offered to look at it in three years.

This story shows a common conflict between IT and the user community. The IT department was concerned with maintaining the integrity of the desktop systems and not allowing the users to simply install new software with unknown impact. The client of course felt that its basic business strategy was being compromised by an external group which was imposing unacceptable risk for its own satisfaction.

What is needed in most environments is a vision which is both orderly and flexible. To achieve such a vision will require change on both the IT and business side. I propose that there are four factors which are contributing to chaos and which, if properly managed, will open up the opportunity for increased returns on the information technology investment.

There are four identifiable factors which are contributing to chaos in both the software development process and in end-user productivity. These factors are:

1. A flawed software development model which is technocentric rather than user-centric. This model is leading to inadequate specification of requirements and poor fit to the business needs of the users.

2. An IT mission and organizational competence which does not conform to technology’s core role within the organization.

3. Immature development tools and jockeying by vendors for market share with a resultant lack of clear standards.

4. Inadequate technical competence on the part of business users which does not allow them to function in partnership with the IT department.

To reduce chaos, each of these factors can be addressed. While the third one is not under the direct control of the corporation, the remaining factors can be directly addressed.

Factor 1: A Flawed Software Development Model
Despite the existence of software development methodologies, the basic software development model is flawed. The problem is that development methodologies focus on the technical aspects of software design but do not provide a great deal of guidance in analyzing the business process and designing the user interaction with the product. The failure to adequately address the business and interaction aspects of software leads to products which do not fully satisfy the strategic and operational needs of the organization.

Some of the problem is historical. Because in the past, software was developed for use within the IT department, the culture of software development does not focus on interaction design. Developers don’t separate the design of the user interaction from technical design but tend to deal with it as a small part of the overall technical design. As a result interaction design is performed by programmers who are not qualified to do it well. Since the design of the GUI is not technically complex, it is often deferred until late in the project, when more serious technical challenges have been addressed.

The problem with this approach, is that the design of the user interface offers developers a clear specification of what the software is to do and how it is to work. As a result, programmers approach the design of products with less than clear specifications. If they had a prototype of the design available, the development process would be far less chaotic.

The figure below shows a traditional development cycle. What is missing from the flow is an opportunity for users to see how their input is being translated into product at a stage early enough for corrections to be made.

Software should always be developed in two phases. In the first phase, designers work with users to analyze requirements and create a prototype. In the second phase the prototype is translated into a working system by software engineers. With clear specifications and solid user support, the development process is streamlined. There is less churn, less argument, and far less risk of needing to unravel code to incorporate functional requirements that surfaced mid-project. The figure below shows how a two-phased approach is structured:

The management of the business analysis and interaction design should be the responsibility of usability professionals who are expert in these areas. Every software development effort should begin with a clear, crisp product concept. Business analysis is often conducted by constructed scenarios in conjunction with the users. A prototype needs to be constructed and reviewed by users. It should also be subjected to quality assurance in the form of heuristic reviews (similar to code walk-throughs) or usability tests, in which users are observed interacting with the prototypes.

By separating programming projects into a design/prototyping phase and employing talented user-centered designers to prototype, the quality of the software will be vastly improved. Users will be far more satisfied. And the cost of development should decrease. It’s a win-win approach.

Recommendation 1
Rethink Current Software Design Practice to
Incorporate User-Centered Design

Phase 1: user-centered design and prototyping
Phase 2: technical design and build
Bring in usability professionals to manage phase 1 and create user/IT partnerships
Use multi-disciplinary teams
Change incentive structure to reflect user-centered and strategic priorities

Factor 2: Need for an Updated IT Mission
When it was first formed, the IT department was highly centralized. It owned and managed both hardware and software. In the 1980’s with the emergence of the personal computer, the traditional centralized model of the IT department became obsolete as elements of information processing became increasingly decentralized.

The shift from mainframes to personal computers created a need for IT to create a new technology infrastructure. This new infrastructure required extensive investment in both new hardware and software. It also was made especially difficult by the rapid evolution of both hardware and software which tended to obsolete assets soon after their acquisition.

The shift from mainframe to PC was a major shift in information technology models. The emergence of the World Wide Web in the 1990’s created yet another major shift. The web, and the intranets which adopted its technology, challenged local area networks, operating systems, and client-side software. From a business operations perspective, the shift changed the relationship of corporations, government, vendors, consumers; creating new opportunities to share data. In addition, the web’s use of simple graphic/hypertext interfaces created a new set of expectations from users.

Traditionally, IT was regarded as separate from the core business. Often looked at as the "transaction factory," IT operated independently and in an advisory role. With the new technology, information technology has permeated all aspects of the corporation. It is now central to virtually all activities. The mission and resulting organizational structure of IT needs to be updated to reflect its current core position within the organization. To be effective, IT must emerge from its "silo" and become an active player in creating and supporting corporate strategy.

That this need is increasingly being recognized by CIO’s is evident from a survey published this month by Ernst & Young. According to the survey:

IT must beware of reactionary forces which are working against the emergence of the new model. One such reactionary force is the desire to reduce support costs by moving toward thin clients. Users regard network computing with well-founded suspicion and recoil at the idea that the IS department will once-again become the gatekeeper of technology. If IT thoughtlessly changes users’ desktops, their office suites and restricts their ability to influence the design of tools that are at the core of their work product, an organization may lose years in closing the gap between IS and business. It will be essential for IT to demonstrate that it can manage software centrally and still provide users with autonomy and flexibility.

IT departments are also attempting to reduce software development costs through the use of external partners for outsourcing. This is a valid and useful approach. However, no outside vendor, no matter how technically competent, can match the insight and understanding of a company insider. And even the best-intentioned vendor has its own agenda and priorities. Outsourcing increases the risk that the design process will become even more remote.

The legacy of IT carries with it some remaining cultural baggage which must be eliminated. One of these is a continuing failure on the part of software engineers to recognize the importance and complexity of user-centered design. Technical people are used to technical struggle and tend to regard it as part of their skill development. They often fail to realize that users do not see technical struggle in the same way. In fact, users tend to confuse struggle with incompetence.

To address the problem, the incentives which guide software engineers must be changed to reward for the quality of the solution not just for delivery of a product regardless of how usable and useful it is.

The need to re-think the IT mission and to retread its organizational focus and competencies leads to:

Recommendation 2
Update the IT Mission and organization to become
customer-centric and strategic

Create a true partnership with the business. Enhance IT communications skills so that it can work more effectively with senior management and operations.

Factor 3: Chaos Caused by Vendors Jockeying for Market Share
Following the runaway evolution of the 1980’s, a stable technical vision is emerging. While there will be considerable continuing development, it will most likely take place in a more structured environment than previously. Among the basic components of the technical model are:

1. Distributed client-server technology. Software will in many cases migrate from client to server where it can be more easily maintained. Programming languages like Java will support server resident code and will encourage platform independence on the client side.

2. Object-oriented code. Corporations will attempt to reduce costs by reusing code encapsulated as objects. Some organizations will undertake significant enterprise modeling efforts in an attempt to create libraries of reusable objects.

3. Applets. Monolithic applications which are difficult to develop and inflexible when change is required will yield to smaller applets which work together to perform needed tasks. Components, purchased from third parties, will form a major part of most development efforts, reducing programming and allowing specialized modules to be developed by experts.

4. TCP/IP connectivity. The emerging model is highly connected. Web-style technology will form the foundation of a highly connected business model that integrates suppliers, customers with the corporation. Directory services and security will be basic to managing connectivity.

The emergence of the new model should reduce chaos. Unfortunately, a number of corporations are relentlessly jockeying for market share as this new model emerges. The result is that standards, essential to productive development are in flux. Given the immaturity of software development tools, this lack of standards increases confusion. Among some of the areas of conflict are:

1. Server-side Programs: ActiveX vs. Java
2. Distributed Objects: DCOM vs. CORBA
3. Directories: (LDAP vs. Application Configuration Access Protocol)
4. Security: (S/MIME vs. Open PGP)
5. Push Technology: CDF (Channel Definition Format) vs. Castanet

An interesting example of the lack of standards is seen in the following quote from the September 1997 issue of Byte:

"… get rid of the idea that the label ‘ActiveX’ refers to some well-defined technology or group of technologies – it doesn’t. Instead ActiveX is a brand name, like Calvin Klein or Ford….what it is applied to can vary over time."
There is not a lot of direct control that corporations can exert over vendors who see dominance of their particular technologies as critical to their business success. However, working together, corporations can exert the pressure of the marketplace to minimize the damage done.

Recommendation 3
CIO’s should band together to create pressure on vendors to create open interoperable standards

This will not be easy to accomplish but can be effective if vendors feel there is a risk of losing business..

Factor 4: Need for End-User and Software Engineer Training
The final area in reducing chaos is education. New skills are needed by both business users and by software engineers.

The notion of computer literacy may be adequate for some users but is not appropriate for those who must work in partnership with information professionals. Such users need a higher level of expertise I call "computer competence." Computer competence goes beyond basic computer literacy which may be viewed as knowing how to operate a computer and understanding its basic principles of operation. Users with computer competence have a clear understanding of a given technology and have a clear conceptual idea of how it works, although they are not able to program. The computer competent individual can participate in decision-making with IT professionals.

Software engineers need additional training as well. Training in business process analysis will assist in requirements definition. Engineers also need to improve their non-technical communications skills. Finally, it is essential that software engineers understand the depth of user-centered design and the role of usability professionals.

The need for additional training on the part of both users and IT professionals leads to recommendation 4:

Conclusion
In Greek mythology, order emerged from Chaos. In computing the same can occur but only if corporations are willing to make the effort. The type of change required is not easy to achieve. It requires that incentives be changed and new vision be established. These are threatening to many who would prefer to maintain the current situation.To some, chaos has advantages; it can obscure responsibility and mediocre performance.

But the investment in information technology of $500 billion annually will only achieve the desired return if chaos is reduced. Reducing chaos will improve corporate productivity and quality. It is a task well worth undertaking.