Chapter 4 – Case Analysis Findings

This research investigated values conflicts in innovation from ten case studies of software development projects in large software product organizations. The findings identify the types of values conflicts experienced in development projects, and show their relationship to innovation practice in the organizations. Organizational innovation processes contributed to values conflicts, through embedding values in-use within processes. The organizational factors were analyzed independently by studying two software companies in-depth. Both the innovation project cases and the organizational study suggest values conflicts may be inherent in the official processes used in managing innovation projects.

Three research questions guided the research and maintained its direction.

What values conflicts arise within software development teams?

To what factors do product designers and managers attribute values conflict?

How do innovation processes used in product development embed organizational values?

Values and culture are frequently identified in the design literature as an underlying source of influence of both products and processes in systems and innovation projects (Kumar and Bjorn-Anderson, 1990, Grudin, 1993, Friedman, 1996, Nardi and O’Day, 1999). This study investigated values conflict to understand the influence of personal and organizational values on product design. Four consistent classes of conflict were discovered across seven projects at a large software product company, and three other projects in other companies. A central finding was that both product managers and software designers attributed values conflict to organizational innovation processes used in their organizations. Values issues attributed to design and development process located eight factors across all cases, revealing organizational process as a source of values conflict. 

Organization of Findings

This chapter presents the findings developed from all data analysis in three sections.

Models of Values Systems were derived from seven foundation case studies selected from the literature. These prior studies generated a platform for relevant analysis of organizational values systems in the product innovation domain. These models summarized values constructs drawn from the case studies, and offered the values framework developed in advance of data collection. These models were prepared as structures for data collection, analysis, and interpretation of interviews and cases.

Case Analysis Findings follows, describing the findings from data analysis, and discussing the interpretations of these findings. Findings were developed from analysis of ten projects used as the research cases. The Findings are organized in three sections.

1.    Cross case analysis of projects. This section presents the analysis of recurrent themes found in the interview transcripts, represented as claims of values conflict in projects. These claims show across nearly all the cases, revealed through analysis of categories across the cases.

2.   Values analysis of innovation projects. This section presents key findings extracted from summary analysis of cases, including the specific conflict types and the personal and process values revealed in the research. This section also discusses the results of the product and process surveys conducted with each participant.

3.   Discussion of case findings. The case analysis findings are discussed as a summary analysis following the findings.

Interpretive Analysis of Innovation Process. This third section presents a hermeneutic interpretation of the embedded values conflicts in innovation processes. This independent study followed up on emergent findings drawing from the ten project cases. This analysis describes the patterns of conflict from embedded organizational values in the innovation processes of two large software companies.

Models of Values Systems

Model Development from Cross-Case Analysis

A values framework was developed using literature case studies as qualified sources to define a structure for values research. The framework offered a means to expand upon the values discussions documented in organizational studies. Seven case studies contributed to the framework for guiding direction and establishing common definitions.

In advance of data collection, I constructed a design-specific values model from cross-case analysis of seven frequently-cited studies, as a foundation to guide the design of the study and the data collection instruments. The seven cases were selected from the literature as foundation cases, based on the sampling criteria and analysis described in Chapter 2, Research Method. These studies were all field research studies, and used interpretive analysis approaches. Four of them dealt explicitly with software product development (Curtis, Poltrock, Tang, and Walz); the other three (Orlikowski, Robey, and Sachs) were information systems studies, with organizational orientations. These cases all discuss implications for organizational process and behavior in the product innovation context.

The purpose of this case study model was to develop an initial framework of research questions and constructs for guiding data collection and interpretation. I present this model in the Findings, since it contributed new information as well as interpretive utility. Also, it was developed from a similar interpretive analysis approach used to derive findings from the PDE research cases, and so functioned as a pilot model for the research itself. An analysis of common themes and key research problems from these cases defined the scope for the values model. The seven foundation research cases are summarized in Table 4-1.

Table 4-1. Foundation Case Studies.

Development of Composite Values Model

Throughout the investigation of innovation project values I reevaluated the definition of “values” in the context of innovation. What is meant by values constructs and conflicts in products and the designing process? What have researchers found in studying personal and organizational values, and their relationship to design and development process? What are the important or relevant values constructs to consider in this specific domain of design and process? Are there general values models applicable to product design and the design process? This research accommodates these questions, but they also lead well beyond the study.

The composite values model was constructed following three stages of development supporting the research using models from across literatures. The first step reviewed values models in organizational systems and information systems case studies. The second step integrated the common attributes from these prior models, and derived a framework to encompass a range of values constructs. The goal was to support a more inclusive description than offered by a single well-known model. Finally, this composite values model was distilled into a set of specific constructs that were evaluated by the research participants. Some values were common across values the dimensions; these were interpreted as showing agreement for inclusion in the reduced set of values in this model. Even though the values framework was an interim process used to construct data collection instruments, the framework itself revealed a usefulness for design problems, and makes a contribution for future research and design use.

Tables 4-2 and 4-3 compare and summarize the values models. I found four different families of values dimensions, each representing a different research tradition and social meaning. Design values and social values are characterized as reflecting values positions or responses of individuals, and are demonstrated as responses to personal values. These two sets were defined as individual values. In the Findings, I refer to personal values as a specific participant’s values response from experience. Although many of these values dimensions are identified from the organizational domain, they are motivators and response factors for individuals.

Organizational and technical values are characterized by their reference to institutional values systems, and are demonstrated by reference to one’s organization or occupational standards. The institutional values are dimensions identified across large organizations and professions. The four families were characterized as follows.

1.      Design values – Design values were found in two literatures. Friedman (1997), Kling (1996), Kumar and Bjorn-Andersen (1990) were the key references in information systems design, and Margolin (1998), Papanek (1996) and Jones (1992) in design studies. These design values apply universal human values to designed artifacts, and they refer to values recognizable across many traditions. Being situated in the literature of design, the emphasis in this collection was on values dimensions affecting the design of systems and products, and not basic human values.

2.      Humanistic values – Humanistic values integrated the human values constructs of Rokeach (1973) known as the most widely-cited and accepted general values model. Maslow’s (1971) values framework was adopted, drawing from the B-values (“higher” values) constructs, and England’s (1967) values model was referenced.

3.      Organizational values – Organizational values constructs were drawn from the Walsham and Waema (1994) case study, which exemplified values conflicts among leadership styles and organizational processes. Values constructs were also drawn from Crosby, Bitner, and Gill (1990), which developed a structural model of organizational values.

4.      Technical/engineering values – Drawn from Kumar and Bjorn-Andersen (1990) and Banathy (1997), these values apply to systems engineering and development practice.

Tables 4-2 and 4-3 describe the specific values dimensions included in these four values families. Selected values dimensions were integrated into survey instruments used with the interviews. Product design values were investigated using six dimensions, drawn from Friedman (1996), selected from design values from the individual values dimensions. The six dimensions selected for inclusion in data collection are indicated with an asterix. Table 4-3 (institutional values dimensions) similarly indicates eight dimensions selected for use in the research survey on design process values.

Across these four families of values, redundancy was reduced among similar dimensions. The values dimensions from across sources were evaluated for commonality, and then categories were drawn from dimensions using the same terms or intentions. This was a heuristic procedure, and was not generated from a statistical approach (e.g., paired comparisons, correlation). Although these dimensions are not orthogonal, the framework was not designed for studying values dimensions, but to construct appropriate research instruments.

Table 4-2a. Individual Values Framework – Design Values.

Table 4-2b. Individual Values Framework – Humanistic Values.

Table 4-3a. Institutional Values Framework – Organizational Values.

Table 4-3b. Institutional Values Framework – Technical/Engineering Values.

Case Analysis Findings

The following sections describe the research findings from analysis of 10 product innovation projects and the two companies hosting most of these projects. The following sections organize the findings: 1) Institutional context of case studies, 2) Summary case analysis of projects, 3) Cross case findings, and 4) Analysis of organizational innovation processes.

Institutional Context of Case Studies

Seven of the ten reference case projects were drawn from one company, Data Online Corporation (DOC). Data Online (not its real name) is a large, leading online publishing company, with roughly one billion in revenues and 8,000 employees across all of its concerns and sister companies in the group. DOC offers online access to news, business, legal, and government information. It was a leading provider before the shift to Web services, and is now also a leading provider on the Web having successfully managed the transition to the Internet.

DOC grew from roots in technology, from its early installation of very large databases, to expand into online publishing. The shift in focus from information technology to information provider occurred in the early 1990’s, as the company faced competition from well-capitalized publishing firms. By the mid-1990’s, DOC made dramatic changes to staffing and organization through reengineering. Following this restructuring, DOC was sold to a global publishing firm, and was structured more closely to a publishing enterprise than a technology company.

Corporate Strategy. DOC’s strategies include growth through sales of large contracts and through acquisition of related publishing concerns in its areas of strength. Although its past strategy regarded its databases as key to competitiveness, recent strategy has emphasized innovative product and service offerings and adding value to the data. DOC has also continued to add to its depth of publications by acquisitions and mergers of other publishing firms.

Products. Data services remain the core of DOC’s product offerings. It serves the markets for online news data from thousands of newspapers, periodicals, magazines, journals and worldwide publications. It also offers government records, financial, legal, and scientific data. Its product strengths are unique, high-value bundles of content provided for information professionals in commercial industry, consulting, professional services, and government. In particular, the majority of projects analyzed in this research involved product development for improving the data, or enhancing its access through the Internet. Therefore, DOC maintains a core competency in information technology product development.

Customers. Professional services firms, universities and libraries, government organizations, and large companies are the primary customers. They typically purchase large contracts for data services and information products. DOC gathers customer feedback on products through market research and field visits, as well as through traditional channels such as the sales force.

Competitors. DOC recognizes between 5-10 traditional competitors in online data services, and over 20 competitors across the Internet in the various niche markets its also serves. DOC remains a significant vendor of content and services in all of its markets.

Structure and Culture. DOC is organized into operating companies that manage content and services for each marketplace. Within each operating company, a strong matrix organizational structure with four management levels reports to an operational executive. Staff are matrixed to projects, which require cross-functional participation for every project due to the specialized knowledge required. Software developers and design professionals report through a product management chain (business, not technology). Although work is conducted through projects, the culture is not project-base, but remains hierarchical in orientation. As a reflection of this, most project managers are not responsible for product development, as responsibilities are diffused across roles.

Role of Innovation Management. DOC continuously works toward improving both effectiveness and cost efficiency. Innovation management in the company focuses both on products and the processes used to develop products. Although DOC has traditionally innovated more on improving the usefulness of the data, recent efforts have focused innovation on product user interfaces, product features, unique packaging of databases, and internal design practices. However, no consistent management approaches have been used across the company to organize practices and learning in innovation management.

Cross Case Analysis Findings

Transcript content analysis revealed consistent recurring claims of conflict with project innovation processes. Claims clustered around two broad dimensions, values (personal vs. organizational) and processes (professional practice vs. organizational routines).

Based on the research focus, at least one significant conflict experience was described in almost every project case. In all cases except one, the conflict was attributed to a values difference. Since the purpose of the research was to investigate values interactions in product development organizations, these themes were expected. An unexpected finding showed most of the claims representing organizational process issues, and not personal values conflicts. Three of the four themes surfaced underlying difficulties in organizational or development process, and one identified conflict between personal and organizational values.

The following sections provide a summary of the findings, followed by an overview of the projects from which the case findings were drawn, and then the cross case findings for each of the claims.

Summary of Cross Case Analysis

Four recurrent distinctions or themes were found across the cases, and were coded as claims. Most of the claims represented organizational constraints on practice or behavior. Each theme was found in over half of the cases evaluated, and some were identified in every case. Only one case showed exceptions to the consistency of these findings, reporting no project conflicts; since this participant (Hal) had minimal product innovation experience (less than two years), his exposure to projects was far less than all other participants. 

1.  Cross-functional projects afford multiple points of conflict with personal and organizational values.

Project team members expressed personal values conflicts with in-use values of the organization. Multiple conflicts were found in most cases; attributions for the conflict were made to differences with organizational values and processes. Participants also cited substantial influence from hierarchy and imposed organizational process. 

2. Innovation management processes afforded conflict between professional and organizational norms and values.

Values conflicts between project team members were evident in each case study, and multiple conflicts typically occurred throughout the case. Participants attributed claims of conflict to professional or organizational cultural differences. Intra-team conflict often showed up as differences in practice used by members reporting to different organizational functions.

3. Differences between standard and practiced innovation processes afforded project conflict.

All cases identified discrepancies between official (espoused) and practiced (in-use) innovation management processes. These included project management and oversight, product lifecycle management, software development, and product/interface design. Conflict between explicit design processes and implicit, situated design was also noted. Designers often used highly situated design practices in the daily work context of designing product features. These preferred practices were contextual and not explicitly codified, as revealed through interactions with the project team. Product managers may have performed in a similar capacity, using experience instead of official published processes.

The in-use innovation practices afforded project conflict when standard processes were ignored. Conflict between development practices constrained access to sources of design knowledge. Specifically, most cases cited the separation of designers from their source of domain knowledge, typically users or customers. This difference was attributed to standard processes, embedding practices and organizational “ownership” of customers. 

4. Process conflicts interfered with team knowledge integration.

Participants indicated conflict with knowledge integration, or the requirement for design teams to intensively generate and integrate knowledge about the target domain. For example, participants suggested teams failed to adopt customer/user information supporting design requirements, substantially constraining knowledge integration among all team members.

Project Case Descriptions

Seven DOC projects and three non-DOC projects were analyzed. Of the non-DOC projects, two did not contribute significantly in cross case analysis. One DOC project did not contribute to the analysis; the summary identifies it as an exceptional case. The projects contributing to the analysis are summarized briefly below to enable the reader to better interpret the findings. Projects are listed in order of presentation in the findings, by participant name and project.

The DOC projects were primarily search and retrieval products developed for the Internet. Two projects were infrastructure technology, with product groups as internal customers. Nearly all DOC projects were large, cross-functional projects staffed with professionals from product management (business), software product development, product design/usability, data engineering, publishing, customer service, and included a project manager. The typical project was conducted within a year period, with a 2-3 month concept and requirements phase, a 3 month development cycle, and a test and release cycle. DOC projects followed Cooper’s (1996) Stage-Gate process to some extent, and other internal policies and standard processes were followed as required for project management, product testing, and software development.

Mike’s project, the WIP product, was a Web-based interface to a multiple platform engineering system used in CAD/CAM analysis and design. This was the sole project quoted in the following analysis not drawn from DOC. The other two non-DOC projects were included in the analysis, but were not used as sources for representative citations of issues. All project cases analyzed in this study are described as follows.

1. Brian, Designer - Web Enterprise

Brian was the senior interface and product designer for the Web Enterprise product, an online news and information service designed for use in large organizations. This large Internet project was considered critical to the company, and spanned a one-year period of design and initial product development (1997), requiring nearly a second year for redesign and release (1998). Brian participated on the project during a second phase of concept design, following an initial concept phase in which outside consultants developed the first prototype.

The product team initially consisted of two interaction designers, two product managers that shared responsibilities, and a project manager. Following this concept phase, a technical lead came on the project with other software engineers available as required. The project team grew to 20-30 people overall, all but five software engineers. A major company reorganization occurred during the design phase, disrupting staff assignments and roles. Following the reorganization, a senior architect role was added as well as an internal consulting group.

Two formal processes were used in the project. The project manager followed basic project management, based on the stage-gate product development process with an emphasis on schedule and deliverables. Brian used a user-centered approach to organize the user interface and to manage all detailed design decisions. He also attempted to conduct front-end analysis, iterative design/prototyping and user testing. No documented product lifecycle management or development processes were used, except “an attempt at rapid/iterative development.”

Brian indicated the project manager did not actually run the project, but that different roles held project authority – the product manager, executives, and software engineers. Brian left the project at some point during development. Brian moved into product management in 1999, and then returned to manage a major redesign of the product.

2. Karen, Product Manager - Web News products

Karen was the project and product manager for a collection of products offered as news content products. Her products constituted some of DOC’s original Web offerings in 1996 and 1997. Karen managed a group of seven Web products, some of which were released, others which were cancelled. The staff complement for the program consisted of nearly 50 people, mainly software engineers. Karen managed these projects over roughly a two-year period, from late 1995 through 1997.

Process used in Karen’s projects included a steering committee process, a project management structure, the stage-gate product process, and a standard development process. Beginning the project with the standard processes, the project encountered internal pressures and a new executive replaced the original project sponsor. With the new sponsor, change control was disregarded – “we didn't know what feature/functions were in or out from day to day, but we made our date.” Karen described the project activity as follows:

“Not after the management structure got out of whack, everyone did their own thing. There was no "reporting" back to a steering committee. The implementation team took license where they needed to. The only thing we had to do was get the product out before the end of 1996. We threw EVERYTHING out the window that got in the way of that goal - process, good manners, rigor, testing, etc.” Karen added “I think this project was a terrific success story but it was IN SPITE OF the management and processes (emphasis Karen’s).”

Instead of an interaction design process, they managed the design through continuous prototypes, using demonstrations and functional prototypes. No product management process was used either, although the project assigned a product managers for each of the seven projects under the program. According to Karen, “That was part of the confusion. Everyone fought to get "their" functionality taken care of first.”

3. Paul, Product Manager - SearchBuddy project

As senior product manager for the SearchBuddy project, Paul managed a smaller project team of about five for a product defined for a specific customer market segment. The team included two product managers, a software developer, a technical manager, with Paul as "sponsor." The SearchBuddy product was designed to be a simple news search interface, for use by managers and other non-professional users that were not comfortable with computers. This project was initiated in 1995, before Web-based products became DOC’s mainstay.

The project team used ad hoc processes for the project, according to Paul.

“Presumably, the organization had in place processes like Stage Gate for project management, prototyping for UI, JAD for at least the requirements-producing piece of product management, and some sort of software-development process. I say "presumably" because none of these, save perhaps for iterations of the UI, was very much in evidence during my particular project. I don't think there was ever even a "formal" requirements document produced.”

Paul identified a problem typical with smaller projects, wherein official processes are not used and no informal product development processes were adopted in their place.

“No one on the team was experienced in project management--I think it fair to say that, in process terms, the team was more or less "clueless." The organizational structure and its culture at that time (including politics) was not geared to provide effective support to this project. We had some smart, hard-working people trying as best they could to cope without the necessary training in project work--kind of like trying to steer a ship without a rudder.”

Although informal processes have been reported effective in other projects (see Karen’s Web News project summary), in Paul’s case the lack of structure impaired the project.

“Not even the ad hoc methods were followed faithfully--the rules kept changing without notice as we moved "forward.” I proposed an approach to the product managers responsible for the end product. The developer reported to 2 or 3 different managers over the course of the project, each of which had a different approach, or set of rules, for development; and technical standards for this sort of product were also changed several times (sans "official" notification) while we were in development.”

Furthermore, as this project progressed, a change in senior management replaced Paul’s sponsor. Even as Paul had attempted to gain customer insight into product acceptability through trade shows and sales representatives as customer proxies, the new sponsor disregarded these efforts and insisted on late-notice personal changes to the product. This was acknowledged as a common practice, that of new executives wishing to impose their “stamp” on products by adding new requirements or changing existing features.

4. Don, Designer/Developer - Infrastructure project

Don was a senior engineer/software designer assigned to company-wide information infrastructure. The specific project reported was to integrate UNIX servers with an NT network. Staffed by 5-6 team members, this project was smaller than most, but led by a very senior technical officer, reporting to the vice-president.

Reportedly, no explicit direction was provided to software engineers, only a list of technical goals. As Don expressed it, “the task was, simply put, “go make this work.” One of the central conflicts reported in this project was that other, organizational goals were hidden from the developers. As this internal R&D project continued, the senior technical officer reportedly micromanaged the team’s efforts on a continual basis, causing Don concern for the reception of his contribution to the project. Don indicated he made a specific, tangible contribution to the project, making sure his work was judged acceptable before choosing to transition from the project to another. He reported the project continued to demonstrate technical and project difficulties even after he left.

5. Lloyd, Product Manager - Online Science

One of the senior staff managing the large Online Science project, Lloyd represented the publishers responsible for content on this product. With a core team of about 20 staff on the project for the first year, Lloyd managed product direction and requirements with a cross-functional team. The project included a senior project manager, senior product manager, very senior core development and data engineering staff, and publishing representatives. The project supported two design and usability staff, and contracted with an external design firm in advance of product release. With over a year in design and development, the Online Science product underwent a phase release to customers, and now stands as a market leader in Web-based scientific journal publishing.

All formal DOC processes were followed due to the high risks and expense inherent in the product. The official Stage-Gate process and project lifecycle templates were integrated into a schedule and work breakdown structure. These project tools were tailored extensively by process consultants, who adapted the standards to fit the unique requirements of this project.

Lloyd characterized the design process as “the mock-up approach. HF would mock up screens and I would get sign off from publishers that this was what they wanted. The mock screens then became the driving force for the engineers, and as we learned more, the mock up screens changed - which caused confusion and frustration for the developers. However, from a product and business standpoint it was seen as a most successful venture ... personally, we over engineered the product.”

As the project progressed, Lloyd indicated the processes evolved with the project. The formal approaches were left behind once the requirements and prototypes were stable and considered representative of the first release. A change control process was instituted once the prototype was considered stable, but requirements continued to fluctuate for several months even into development. Once the change control process was established, only the core staff members participating on the board made design decisions.
6. Jack, Product Manager - Reference Collection project

Jack was product manager for several products in DOC. For the case selected, the project lasted no more than 3-4 months before cancellation. During the project, Jack established requirements, maintained a small project team (product manager, publishing sponsor, software developer) and represented the product to senior management. Although Jack represents his requirements work as unfairly criticized by a manager, the project was cancelled for corporate reasons before the project started any development.

7. Mike, Designer – CAD, Inc. – Wor