Wednesday, November 5, 2008

Week 12: Design expertise

We'll be revisiting the "design knowledge" space with a focus on expertise. We're also transitioning to a instructional approach where the focus is on creating personas (short example, long example, wikipedia example) that help link past discussions and readings to a goal of understanding design learning (and of course, supporting design learning).

A persona is an "archetypal" person that represents the needs of larger group of people in terms of their goals and personal characteristics. Although they are fictitious they are based on knowledge of real people. Personas are a technique used in design (particularly user-centered design) to explore user motivations, expectations, and goals. We are using them in this class as a way to relate research on design knowing and learning to the practice of educating design learners. By seeking to understand design learners we can better understand needs for design education. In other words, it's a way to identify learning goals and learning challenges that might be anticipated.

The readings are:

  • An overview: Cross, N. (2001). “Design cognition: Results from protocol and other empirical studies of design activity.” In C.M. Eastman, W.M. McCracken & W. Newstetter (eds.), Design Learning and Knowing: Cognition in Design Education. New York: Elsevier Press. How do novices and expert designers differ? Going back to the first weeks of school, what does this say about the nature of design knowledge?

  • An example from the design data we've been using all term long: Atman, C.J., Adams, R.S., Mosborg, S., Cardella, M. E., Turns, J. and J. Saleem (2008). “Engineering Design Processes: A Comparison of Students and Expert Practitioners.” Journal of Engineering Education.

Drawing on either of these: imagine a particular design learner (e.g., freshman, senior, graduate in industry for a couple of years, etc.):

  • Who is this learner? What's their story (age, background, kinds of experiences related to design, likes and dislikes about design, motivations, beliefs about design)? Feel free to be creative here - provided you can answer the next question!
  • What do the readings suggest about what this learner might know about (or need to know about) design? Why (draw on past readings and discussion)?


§adieLovingtonNibblesworth said...

So I will speak mostly about the Cross paper even though I read both, mostly because I liked its conclusions and it really ties together a number of readings in the course so far. I am NOT going to focus on page 94 of the Cross which is explicitly about novices and experts, but will focus on other parts of the paper I found enlightening.

What I really love about the Cross reading is the challenge of what exactly people *do* with "ill defined" problems. From page 81 underneath the "Problem Formulation" title it says... "It is widely accepted that design 'problems' can only be regarded as a version of ill-defined problems." Cross then says on page 82 in the same paragraph that "One of the concerns in some other areas of design research has been to formulate design problems in well-defined ways."

Cross strikes at the heart of a fundamental problem in design that we have been thinking about all semester; that there is a natural propensity for engineers (and design researchers) to overextend themselves into the space of generalizing solutions to ill-defined problem spaces in the process of describing said space.
In the same paragraph, Cross goes onto explain that some designers "behavior was characterised by their treating the given problems as thought they were ill-defined problems." I think this is another flaw of engineers (and, subsequently, design researchers). In this world where "ill defined" problems are the accepted as the norm in design, we risk the problem of having designers who treat systems as ill-defined from the start.

Concerning novices and experts. Cross makes some interesting conclusions, (although not as explicitly noted as "novice vs expert" as in the Atman paper). For starters, Cross notes that while spending a lot of time on problem (re)defining isn't indicative of stellar results, time spent on problem *scoping* is very telling. The concept of scoping is related to a designer knowing when to reframe the process he/she is involved in; which is in essence a boundary decision.
The conclusions surrounding the nature of "fixation" are somewhat nebulous in Cross. Although Cross outlines what fixation is, in the conclusions section he leaves the penultimate defining of fixation open by stating fixation can lead to "perhaps only when exercised by outstanding designers... to creative, innovative design." This relates directly to the work we read two weeks ago where the premise of the paper was that although experts could not ultimately define when a frame shift occurs, they know when "when they see it."

celia said...

Mmmm, so I will say sth about Atman et al's paper. Basically, their research is intended to test 12 hypotheses which based on findings from their previous research and findings from literature. 10 hypotheses are confirmed and 2 are not. They discussed the result,especially why two hypotheses are not comfirmed.
This study, together with Cross', provides us some implications for creating persona. We could infer some characteristics of potential design learners,e.g. freshmen, senior,novice, expert.
However, it should be noticed that there are learners from "middle class", who are guys standing between novices and experts. Creating personas for these "middle- class-learners" is also important because these particular learners represent a transitional process in design education.

CJ said...

I chose to try to relate the concepts presented in the Cross work to the findings from Atman, et al., so I expand on some things that Sadie and Celia said and also try to tie them together.

While each of these papers presents a different viewpoint for discussing novice and expert design behaviors, several similarities may be drawn between the findings of each. In the discussion of problem formulation, Cross claims that novice designers may become “stuck on information gathering, rather than progressing to solution generation,” (82) which may also lead to lower quality designs. At first this may seem to be a bit of a contrast compared to the findings presented by Atman, et al. because it was shown that experts spent significantly more time than seniors in problem definition, information gathering, and project realization stages. (Being ‘stuck’ in a phase would imply spending more time there.) However, one point to note is that spending more time in a particular stage (especially problem scoping) does not necessarily lead to better performance or better design outcomes. So, while some novices may seem to spend more time in the problem scoping stage, it is really the information gathered and the conclusions that are drawn which likely lead to “better” solutions. Each of the papers recognizes though that attention to ‘problem scoping’ does seem to be an important aspect of design for both novice and expert designers.

In terms of solution focusing, Cross states that Lloyd and Scott “found that more experienced designers used more ‘generative’ reasoning, in contrast to the deductive reasoning employed more by less experienced designers” (Cross, 83). Evidence is provided that both novice and expert designers use (to a certain extent) a ‘co-evolution of problem-solution’ where iteration occurs between the problem formulation and solution generation. This finding seems to be especially coordinated with Atman’s findings that expert designers, and the seniors with higher quality scores, exhibited a ‘cascade through design activities’ as shown on the timelines, and a transitioning back to problem scoping activities throughout the process.

For solution generation, Cross discusses five areas: fixation, attachment to concepts, generation of alternatives, creativity, and the role of sketching. Fixation could be a negative thing in design if it prevents generation of new or original concepts. However, it may not necessarily be a bad thing when the designer is able to focus (a form of ‘fixation’) on their problem frame, “or on a guiding theme or principle” (86). Findings from various researchers showed that both experienced and student designers sometimes had a tendency to pursue only a single design proposal, or with the specific case of senior students, continue to adhere to a solution even when it was proven ‘less than satisfactory’ (87). Akin and Akin found that experienced architects were more able to perceive their own fixation within a standard frame of reference which then may lead to a “‘sudden mental insight’ (SMI) that is commonly reported in cases of creative problem solving” (89). Novice designers did not possess this type of ‘procedural knowledge’ and therefore were “not able to generate anything other than a very conventional solution” (89).

The aforementioned areas are especially related to the results from Atman, et at. in the problem scoping and information gathering, as well as considering alternative solutions. Atman’s findings showed that experts had significantly higher numbers of coded objects than seniors. However, this is not necessarily an indication of a higher number of alternative solutions. These ideas are also related to analysis of the problem because as Atman states, “a particular challenge is that novices often do not realize that they do not understand the problem or feel that they understand the problem well enough” (Atman, 361). A lack of understanding in the problem definition stage may also lead to fixation or fewer alternative solutions because the scope of the problem has not been fully realized.

The last major area discussed is that of process strategy. Fricke found that, for mechanical engineers with varying degrees of experience, those “following a ‘flexible-methodical procedure’ tended to produce good solutions” (Cross, 91). This statement seems a bit ‘soft’ to me in that it is quite difficult to qualify and probably to explain (and to some people, even difficult to understand). I am especially thinking about trying to teach students a design process to use as a guideline, but asking them to use it ‘flexibly’. I think that inexperienced designers (students learning design) would not really know how to interpret that and even more, how to implement the concept. When should they follow the given process, when should they stray? This brings a little bit of the novice/freshman persona into the picture. Nevertheless, it seems that the idea can provide some insight into the process of design.

Finally, for varying degrees of experience, McNeill (presented by Cross) confirmed that over the whole design episode there are both short-term cycles of analysis-synthesis-evaluation and also one large pattern of beginning by focusing on analysis of the problem, then moving to synthesizing the solution, and finally spending most of the time evaluating the solution. Overall quality of design concepts may be related to rapid alternation of activities, which was found for both expert designers and students. This has a nearly direct correlation to Atman’s findings that experts and seniors had approximately the same number of activity transitions, and approximately the same transition rate among design activities. It also relates to the process timelines presented by Atman, et al. and again the idea of both ‘cascading through design activities’ and transitioning back to problem scoping activities.

Aidsa said...

There isn’t much to add to previous comments, but I will provide my “connection” to the findings of both papers which might be similar to what my peers have already mentioned (sorry for the duplicity of thoughts). I identified the following “key points” from Cross’ work: (1) design cognition is domain-independent, (2) designers are ill-behaved problem solvers, (3) over-concentration on problem definition does not lead to successful problem outcomes, (4) successful design behavior is based on adequate problem scoping and on a focused approach to gather information, (5) setting and changing goals are inherent elements of design activities, (6) experience in a specific problem domain allows designers to move quickly from the problem identification to solution generation phase, (7) higher structural process leads to greater design success. In the case of Atman, the most influential findings were: (1) problem scoping and information gathering are the major differences between novices and experts, (2) it is important to progress among all stages, (3) quality of designs increased in expertise because experts spent more time: on tasks in each stage of design, on problem scoping, and gathering information.

little-T truth said...

The readings Cross and Atman et al. both make compelling cases for attributes that designers may encompass. One major concession to be wrestled with in tring to identify the "persona" of designers as a group is the variety present within individual designers' approaches, yet the appearance of a larger guiding framework of process, cognition, and socialcognition.
In conceptualizing the emerging designer, it is easy to envision the proscribed process appraoch being provided as a framework to them as students. The challenge is accounting for outside forces, particularly experiences -- instituion-oriented or individually obtained -- that shapes the designers' view and problem solving approach. An exaple of this comes from the role experiential learning, particularly internships, alters the way students engage and think about their coursework and approach the problem-solving/problem defining process. Why? Because they have, most likely, had contact with the advanced approaches of "expert" engineers in their domain who challenged, or at least demonstrated, a different way of thinking through the somewhat ill-defined problem interns are provided for their experience. Granted, I must acknowleddge that every internship expeience is different -- based on authenticity of activity and mentoring/access to "expert" engineer -- yet anecdotally almost everyone I know has come back to school with some new understanding about the domain and view of a different approach to the domain than they had before they left. Setting aside experience, what really happens over time to make designers obtain a unique, yet somewhat consistent and identifiable construct for engaging in designing?

I think the papers support the idea that designers are a combination of identifiable practices, which emerge over time through personal experience, for designing AND individual traits that make each designer unique in their approach to design and level of engagement of recognized design attributes. The differences may break down along the lines of how committed a designer is to the process, cognition, or social cognition aspects of design. I am not sure how to break this out yet, but I am pondering what impact a designer's committment and comfort level with engaging problems from these approaches may influence the final design space from which they operate.

I think it is clear from the articles that designers are willing to engage problems, ill-defined or well-defined, as open to interpretation where constraints and general known problem information should be evaluated and considered to clearly "scope" the proble to be solved. I also believe that designers begin to recognize, as a problme comes into shape, potential lines of solutions associated with the identified problem that allows them to shape their situating of the problem, as well as starting the process of moving from problme to solution stage.

I am still reconciling other characteristics between the papers and my understanding, yet I believe the concepts of "fixation" sketching, generating alternatives, "process structuring" and opportunism all have some role in the final designers makeup.