Gray, C. and Malins, J. (2004) Visualising Research: A Guide to the Research Process in Art and Design. Oxon: Routledge

‘Socrates asked questions, Aesop told stories. In learning contexts, the use of Socratic dialogue involves the teacher asking questions that the student tries to answer. On Aesopic dialogue (Ferguson et al., 1992) the student asks questions and the teacher answers with stories. Stories are powerful and memorable means of making sense of the world and engaging imaginatively in learning.’

How have you come to the decision to do research? What are your motives? Write a short story about it.

My teaching journey has grown out of my experience of undergraduate study. A first generation disabled student from a low-income household struggling through an elitist Russel Group university on an access scholarship. I felt less like a token mascot – perhaps that would have been better in some ways? – more like a poor relation. Every tutorial, class and dissertation meeting was a burning reminder of inferiority. It wasn’t just the staff, but my peers. The faculty dogmatically cultivated an ethos of superiority, and for a time I tried to deconstruct my identity to fit in. I suspect that’s why when, a few months ago, a peer from my course commented ‘you say you didn’t have the cultural capital, but you seem to talk the talk!’
My immediate response to this comment is what has lit a fire under my desire to research; there was a time where that would have been the greatest compliment. To hear that I had dissected away my true self to reveal an acceptable academic would be an honour and a pass into belonging. However, as those words left his mouth I felt an intense anger. A tightening of my stomach and chest as I recalled three years of trauma, making myself smaller, steeling myself to walk into the studios each morning. One can’t just change oneself without losing oneself, and this process often involves self flagellation which can be both metaphorical and literal. For me, the metaphorical self punishment was accompanied by anorexia and addiction which continued well after I received my degree. I don’t remember what I said in response to my peer that day, I was lost for words. The trigger didn’t lie so much in what he said, but in the subtext: an assertion that my identity again was fair play for discussion, and not on my terms.
I am privileged in many ways. My positionality is host to many of society’s ‘gold stars’: I’m white, now middle class, and heterosexual. I have a psychological disability, but it’s often easy to conceal so allows me to pass as able if I choose to. At UAL we have students who are not just first generation, but are also people of colour, and/or identify as LGBTQIA+, and/or have a visible disability. We know that a sense of belonging-ness is integral to student attainment, continuation and retention, but we do not have a metric for that. Furthermore, we know it isn’t sufficient to rely on KIS data to ensure the safeguarding of our learners. It doesn’t tell us the underlying story, and it can encourage a false sense of self-satisfaction, a dangerous fairy tale that we are ‘doing enough’. My research aims to unpack how we can tangibly enhance and measure belonging within our pedagogy. Supported by an extensive literature review, first hand research via listening rooms, and collaborative analysis and interpretation through members of our learning community, I hope this work will be the start of a longterm dialogue where our institution becomes a meaningful ally to our students.

What might you research?

• ‘Whatever the initial impetus, the ‘what’ should come from a genuine desire to find something out, or else it is unlikely that the study or the enthusiasm for it will be sustained.’ (p. 12)

Why is your research needed?

• ‘…is there a wider need and can this be confirmed?’ (p. 13)
• Get feedback from peers and others in professional and research contexts
• Gather background information on your research proposition and its ethical implications

Contextual/literature reviews i.e. ‘mapping the terrain’:

• Essential to have some idea of who else has contributed to the ‘map’ and what ‘projections’ (perspectives and methods) they have used. The review allows you to acknowledge their different contributions, but also encourages you to state your views critically (…) by the end of this process you should be in a good position to ‘locate your position’ within the professional context and formulate a viable research question in ‘uncharered terrain’ and an appropriate research strategy’. (p. 14)

So what?

• ‘By this stage you should be in a position to make a conclusion and what you have discovered and its value and significance to the wider research context. At PhD level this should be a new contribution to knowledge. At Masters level you should be able to demonstrate a critical evaluation of your research context and show that you have an understanding of methodological issues. Your research findings need to be made ‘accessible’ and presented in a variety of imaginative ways. The thesis – your argument – may comprise several complementary but coherent elements – a body of work, a written text, other supporting material in various formats. It will be necessary to ‘defend’ your argument – especially for a research degree – in an examination viva. An important part of any thesis is the identification of future research leading on from your work. This brings the research process full cycle – the identification of new research questions and new territory to be explored.’ (p. 16)

Back to basics:

• ‘What could research in Art and Design be? Why might artists and designers do research? BEFORE WE ASK: How might artists and designers do research? (p. 18)
• Ontological and epistemological questions.

Paradigms of inquiry:

‘According to Guba (1990), the choice of methodology should be a consequence of ontology and epistemology – that is, methodology is evolved in awareness of what the researcher considers ‘knowable’ (what can be researched, what is an appropriate research question), and in an awareness of the nature of knowledge and the relationship between the researcher and the ‘knowable’. For instance, the positivist paradigm of inquiry is characterised by a ‘realist’ ontology (reality exists ‘out there’), and an objectivist epistemology (the researcher is detached); methodology is therefore experimental and manipulative. in contrast, the constructivist paradigm is characterised by a ‘relativist’ ontology (multiple realities exist as personal and social constructions) and the epistemology is subjectivist (the research is involved); as a consequence, methodologies are hermeneutic (interpretative) and dialectic (discursive).
What might characterise an ‘artistic’ or ‘designerly’ paradigm of inquiry? It is our collective task to develop this, and will require contributions from many practitioner-researchers over the coming years. Guba’s analysis of both positivist and post-positivist provides us with a framework to help describe and contextualise in philosophical terms, the research we do, and reveal our belief and motives for this research.

Reflective practitioner:

• ‘Reflection-in-action’ is a particular activity of professional practitioners and involves thinking about what we are doing and reshaping action while we are doing it. In this sense it is improvsational and relies of feeling, response and adjustment.’ (p. 23)

The practitioner-researcher:

• ‘Robson points out a major disadvantage as that of ‘insider’ problems – the difficulty in adopting an open-minded approach and now allowing preconceptions to cloud the issues. Given that absolute objectivity is impossible, this is a challenge for all researchers – positivists and post-positivists! It can be addressed to some extent by always exposing ideas and practices to other professionals for feedback, support and advice. In seeking the views of others, which will inevitably be subjective, we can develop inter-subjective views, which are less likely to be one-sided. Of course, keeping a critical view of your research at all times is essential. However, the advantages of the practitioner-researcher role are compelling: your ‘insider’ knowledge, experience and status usually lends your research credibility and trustworthiness in the eyes of your peers, that is, you are not an ‘external’ researcher. Most important, you are inquiring as a reflective practitioner, acknowledging the complexity, dynamism and unpredictability of the real world.’ (p. 23)

Actions for reflection:

1. Consider what characterises your professional context? How do the best ‘professionals’ operate in that context?
2. In what ways are you already, or could be in the future, a reflective practitioner?
3. What problems do you think you might encounter being a reflective practitioner-researcher? (p. 24) ACTION: RESPOND TO ABOVE.

Visualising research:

• Andrew Stonyer’s PhD ‘The development of kinetic sculpture by the utilisation of solar energy’ ‘demonstrates the beginnings of inquiry through practice.’ (p. 26)
• From these examples (the book includes examples of MPhil and PhD students A&D work from pp. 27-29) ‘the starting points for research are issues arising from practice, usually the researcher’s own practice (providing a personal rationale), but also issues that can be recognised as valid in the wider professional context (providing an external rationale). This practice-based approach to research naturally prompts us critically to consider and evaluate methods used in practice as to their appropriateness as robust and rigorous methods for accessible and disciplined inquiry, for:
  • Experiencing/exploring, gathering, documenting information and generating data/evidence,
  • Reflecting on and evaluating information, selecting the most relevant information,
  • Analysing, interpreting and making sense of information,
  • synthesising and communicating research findings, planning new research.’ (p. 30)
• Using more than one method to gather information on an issue: triangulation.

Purpose of a contextual review:

Rationale:

Why is your research needed and what evidence is there to support this? (Rationale)

Who else in the field has addressed signifiant aspects of your research question? (Competitors, contributors, co-operators)

When (and possibly where) was the research carried out? (Currency, cultural context)

How has the research been carried out, and what are the implications of this for your methodology and specific methods?

What aspects remain unexplored or require further work? (‘Gaps’ in knowledge, new ground)

There are two phases for a contextual/literature review:

1. Initial surveys to establish the proposal’s rationale and viability, to provide some background information, and to help focus the proposal.
2. The use of these sources to develop a critical review of your research context, leading to the development of your question and shaping a convincing argument. (p.36)

Don’t rush the mapping stage: it is important that the scope of the inquiry is feasible, and therefore a balance has to be maintained between breadth and depth. Initially it is important to cast the net very wide and develop an overview and understanding of the field. This is the mapping stage and can help in deciding what comes within the scope of the research and, equally important, what lies outside. (p. 37)

What is critical thinking?

it evolves ‘meta-thinking’ […] and self-evaluation. it involves not jumping to conclusions too quickly and maintaining an open mind, considering all aspects of an issue before making up your mind. It involves maintaining some degree of distance in order to prevent personal bias or prejudice interfering with your reasoning.

• It encourages questions (‘why’s that…’)
• Imagining (‘what is, how about…’)
• Connecting (‘try linking this to that…’)
• Interpreting (‘could this mean…’)
• Applying (‘I’ll try this out…’) p. 38

Developing convincing arguments:

Toulmin (1958) developed a model of a structure of an argument:

• Claim: an arguable statement, for example formal research in Art and Design is an important activity.
• Evidence: data used to support the claim, for example an analysis of the Higher Education Statistics Agency data reveals a rapid increase in completed research for higher degrees in the creative arts and design between 1994 – 2002.
• Warrant: An expectation that provides a link between the evidence and claim, for example formal research in Art and design should be encouraged.
• Backing: Context and assumptions used to support the validity of the warrant and evidence, for example formal research should be encouraged because it contributes to the rigorous investigation of practice encouraging new developments and new roles for practitioners.

Intellectual standards:

There are ‘intellectual standards’ that you can apply to check both your own use of critical skills and those of others:

• Clarity: is a statement expressed in the best way? How else could it be expressed? Is it sufficiently elaborated? Is there too much jargon/over-specialised language? Are there relevant examples or illustrations?
• Accuracy: is this true? Can its accuracy be checked? Is it appropriately attributed? A statement can be clear but inaccurate, for example ‘all research in art and design is practice based’.
• Precision: is there enough detail to explain the meaning? Could it be more specific or more clearly defined?
• Relevance: how is this related to the topic? Is it truly relevant? Is it out of context?
• Depth: are the complexities of the question addressed? Is the statement qualified by reason and evidence? Is it superficial treatment?
• Breadth: are there issues that have been omitted? Is there another way to look at this? are there other acknowledged perspectives on this? Is a balance provided? E.g. an argument can be accurate, precise, relevant, and in-depth, but it might ignore other views. For example, a strong argument for the effectiveness of distance learning would lack breadth if it ignored a comparison with other modes of learning or failed to consider the cost involved in buying equipment and spending time on-line.
• Logic/reason: you should definitely combine ideas to develop an argument, but if the ideas don’t support each other or aren’t sequenced properly/contradict themselves, there’s no logic there. (p. 40)

The importance of playing devil’s advocate:

Deliberately take a conflicting or different (possibly uncomfortable) position in order to see things from another perspective. It can make us aware of the limits of our knowledge and understanding.

• Try to outline the strengths and weaknesses of different positions in order to explain/justify/defend your preferred position. Flexibility of thinking is a creative characteristic. Playing with ideas, adopting an ‘imaginative’ agenda’, extends our capacity for creative response and may even prompt a shift in position and an advancement of understanding. Another useful device for making sense of, and understanding, ideas is to develop a conceptual framework. For example, if we were trying to evaluate and make sense of ‘research’, we could develop a framework for understanding it by using a concept map:
  • Purpose of research
  • Types of research
  • Kinds of research questions
  • Methodological assumptions
  • Related literature/public output
  • Scale and scope of research

• This kind of conceptual framework allows us to develop an overview of the topic/idea and to begin to ask questions of it. The overview could then be used to develop a more subject specific framework. (p. 41)

What to do when you have LOADS of reading to do:

• Obtain the source and don’t forget to capture the referencing details for your bibliography
• Try to get a quick overview of content and structure – look at the abstract/index/chapter section headings
• Keep our keywords and research descriptors in mind – they are like the ‘spectacles and sieves’ to help you focus and select
• Scan and skim read (try scanning down the middle section of the page, the theory is that your peripheral vision picks up the rest
• Read more carefully the various sections that seem significant
• If you photocopy sections, use coloured highighters on the copy to code key content
• Extract key information only – you can always go back to it later. (p. 45)

Example of a contextual review in art and design:

Many artists struggle with writing…

But – if you adopt an honest approach to writing, your enthusiasm and belief in the research will see you through. It’s tempting to want to use ‘research speak’ or ‘critical theory’ mode and become really overly complex and over intellectualised. But the most important thing os that your work is clear, concise, and allows you to demonstrate that you understand your context and that you can locate your research within than context (p. 52)

‘Mapping the terrain’ of your contextual review:

For it to be useful to you, you must organise and classify your research. You may want to do this in chronological order, you may want to group those which look at a similar theme, you may want to group them by methodology. You might even want to group them using a few different ways and then you have several different maps which help you decide which is the most appropriate way of structuring your review.

• You may want to use a mindmap where you use particular key words at the centre

• You may want to use a matrix i.e. two different dimensions e.g. references against concepts. It can be especially useful to make comparisons against authors. It can also be helpful to point out gaps e.g. if there is a lack of references against a concept then there is a gap in that research. (pp. 54-55)

When it comes to writing the review:

• You should only select what you think is the most relevant sources.
• Before you start writing, it’s a good idea to do a mindmap of your key ideas and arguments now that you’ve mapped out the rest of your terrain. Example below:

Final thoughts on mapping the terrain through contextual reviews:

‘How can you know what’s new if you don’t know what already exists? How can you progress on your journey of discovery unless you are aware of the surrounding landscape and the nature of specific features of the terrain? How can you avoid dead ends or going over old ground? Answer – by making a thorough survey of what is out there and developing a critical understanding of what is directly relevant to your own research context […] some travellers keep a ‘journal’ as a way of reflecting on, and making sense of the experience of exploration, though plotting key coordinates on their map. The reflective journal helps you to see where you’ve been, know where you are and, most importantly, imaging where you want to be.’ (p. 63)

Ch. 3: Locating your position – orienting and situating research

• Remember that your contextual review is an ongoing process essential for framing and, sometimes, reframing your project.
• Map analogy: (p. 67)
  • Contextual review should have provided you with an overview picture and an understanding of your particular research context: MAP OF THE WORLD.
  • Contextual review will have enabled you to locate your research proposal in relation to that context: MAP OF THE COUNTRY.
  • Identification and evaluation of the most important research and practice in that context should enable you to focus on a particular area. Acknowledging what has already been achieved and what might still be explored and evaluated: MAP OF THE CITY.
  • The identification of a specific research question: STREET MAP. Your own house can clearly be seen.

A good question:

• A good question is hard to come by, so you shouldn’t rush landing on one. For example, ‘Did God create the university’ is a metaphysical question and can’t be answered by inquiry. Questions like ‘is practice more important than research’ would provoke heated debate but will remain an issue of personal belief, so is no good. Some questions that can be answered definitively have the unfortunate side effect of being uninteresting i.e. ‘how many completed PhDs are there in Art and Design.’ Most research questions in our field don’t have easily quantifiable answers and can’t be ‘proved’ in the scientific sense. So the best you can do is provide a ‘credible argument based on good quality evidence’ i.e.:
  • ‘How can new technology be integrated into ceramic designer-maker practice?’
  • This can become a proposition: ‘the use of new technology can introduce sustainability into ceramic designer-maker practice.’
  • You can then develop an argument which persuades us that this is the case or not.
  • This research proposition helps us to develop the plan of work and guides the methodology and use of specific methods. (p. 67)

Advice on developing a research question:

• Step 1: Know the area
  • Your contextual review will provide you with an understanding of this.
• Step 2: Widen the base of your experience
  • Look at other research questions in this area, did you read any theses look at a similar theme? Similar journal articles? Select them, evaluate them.
• Step 3: Consider using techniques for enhancing creativity
  • Using brainstorming techniques to develop questions, or playing devil’s advocate and deliberately taking a conflicting or different position in order to see things from another perspective and so raise a range of possible research questions. You can then use a mind-map to focus those down.
• Step 4: Ethical considerations
  • Have you acknowledged that you don’t know something, which is why you want to do some research in order to learn and discover new things? Assumed a position of humility? Have a genuine desire to learn something new? Accepted the formal framework of academic research, complete with ethical obligations? (pp. 68-69)

Steps for ethical consideration:

• Step 1: Clarity of research framework and participants’ involvement
  • Make it clear to your participants that you are conducting a research project and the purpose of the project.
  • Explain how you will operate as a researcher within the project e.g. participant observer.
  • Discuss the kind of involvement the project requires of them.
  • Respect the wishes of those who decline taking part. If appropriate, find out why – it could be useful research information.
• Step 2: Gaining permission and access
  • Make sure you have explicit, documented permission to carry out all aspects of research e.g. access to special archives or other bodies of sensitive information.
  • Ensure that your research does not exploit the participants in any way.
  • Agree to keep the research material confidential until it needs to be communicated as part of the research findings.
• Step 3: Use of data/information resulting from participants’ involvement
  • Share your developing understanding of the research with your participants by regularly reporting progress. Get their feedback on this and integrate into the research anything that improves their accurate representation.
  • If you want to include quotes from participants, or visual material of them. They should have the opportunity to check the material before publication.
  • In reporting your research, apply judgement and sensitivity so a snot to misinterpret or misuse information with regard to participants (pp. 70-71)

Managing project information involving practice:

• If your practice is part of the research project:
  • For example, part of your documentation is sketchbooks, portfolios, photographs, digital images, video and audio etc. Organise in a purposeful way in relation to the stages of the research process i.e. are these part of the: background/introductory material; contextual material/evidence; information on the use of various methods and evidence from their use; information/evidence for evaluation and analysis; or material for conclusions and discussion.
• Colour coding project information:
  • Colour coding information is helpful to quickly identify different types/sections if information.
  • For example, you may want to colour-code various thematic areas of your literature review, so new information can be colour coded and more easily related to the appropriate section.
• Use of icons/symbols:
  • Same as colour coding, but uses icons/symbols instead.
• Reflection journal:
  • Having all the material in one place is very helpful when it comes to evaluation and analysis.
  • For example, storing studio/workshop activity information – an activity log; at other times it may function as a diary; it may contain documentation of WIP, help you contextualise your project; it could indicate pace and progress of your project.
• Bibliographic database:
  • Your bibliographic database will need revisiting regularly and may require you to update your lit review.
  • Important to regularly survey your context for new developments which might help to share and reshape your research. Good research is responsive to new developments.
• Project specific glossary:
  • Many research projects need to define the terms used in the research in order to develop clarity and shared understanding.
• Other databases:
  • As your project progresses you will want to make new databases i.e. for contacts, resources, images, other media etc.
• Software for managing research information:
  • If applicable.
• Proformas:
  • Devising simple proformas can be a good way of storing and organising notes on meetings, exhibition visits, other events etc. The use of a consistent structure can serve to remind you of your key objectives, as well as providing a means of comparing information from one similar event to the next (share the example of the proforma for gathering information for lit review).
• Documentation related to other research methods:
  • Photography for research purposes: photography alone is insufficient. It needs annotation – labelling and expanding with key information. Audio and video files need clear labelling and a short summary on a word document of the same name. (pp. 86-89)

Ch. 4: Crossing the terrain – establishing appropriate research methodologies

• We need to be open minded and critical about evidence, recognising that whilst there may be corroborating evidence, there may well be conflicting or ill-fitting information. Disappointing and difficult though this may be when it happens to you, it’s essential that you acknowledge and discuss this, and make some sense out of it. There might be an opportunity to try another research method, or to take findings back to participants to discuss. Above everything, it is important to understand and acknowledge the complexity of a research issue than try to uncritically ‘paper over the cracks’ of a less than perfect’ investigation (p. 98)

Primary vs secondary data:

• Secondary data already exist (remember that datum is singular and data is plural). They’ve likely already been processed in some way. Therefore, with secondary data it can be hard to take a ‘fresh view’ of the material because of the convincing arguments that have already been made. But, different analyses and interpretations can be made by others, and it’s entirely possible that an independent and original contribution to knowledge can be made by the reinterpretation of existing data.
• Primary data are those discovered or generates as a result of the application of research methods. Often ‘raw’, could be incomplete and/or only give a partial view. Could be opinion polls, transcripts or design prototypes. Primary data is important but you should acknowledge that it is ‘risky’ i.e. it’s new, hasn’t been subject to scrutiny beyond the particular research project it’s from, and so you really need to be rigorously critical of it.

Choosing a research methods:

• So important to pilot any approach you choose, check the appropriateness of the methods and your methodology. A few hours doing a ‘dry run’ sounds annoying, but it can prevent hours of delays and headaches further down the line.
• You should keep in mind that your research method should be:
  • Responsive to the research context and appropriate for use in it (relevant and ethical)
  • Valid – acceptable to other researchers; useful in reality; meaningful to users within a particular context (internal validity); wherever possible generalisable to wider contexts (external validity); as dependable and trustworthy as possible
  • Used with the utmost rigour (applied consistently, with discipline and care)
  • Documented and described thoroughly and clearly in any report of the research, so that it might be usable by other researchers (accessible, explicit and transparent). (p. 102)

Types of methodology:

We are going to look at some common types of methodology often seen in visual research, as well as those we take from social sciences.

1. Practice-based methodology
2. Observation
3. Visualisation
4. Photography
5. Video
6. Sketchbook
7. 3d models/maquettes
8. Reflective journal/research diary
9. Audio reflection
10. ‘Sweat box’
11. Case study
12. Interview
13. Questionnaire
14. Personal constructs

Practice-based methodology:

Definition/function	Developing and making creative work as an explicit and intentional method for specific research pruposes, for example gathering and/or generating data, evaluation, analysis, synthesis, presentation, communication of research findings
Context for use	Research for higher degrees, distinct from ‘practice as usual’ in its use of practice within an academic research framework, which is accessible, transparent and transferable (in principle if not specifics) the work might embody research concepts, provide visual evidence and/or illustrate research findings in some way.
Tools required	Any of the tools used in the development and making art/design work; essential to document the process. Must have explicit criteria for evaluation and analysis
Advantages	Researching and learning through doing to develop ‘deep’ understanding; the participant researcher has an informed perspective on issues relating to practice
Disadvantages	Open to criticism of over-subjectivity is not within a formal framework and/or lacking methodological transparency
Ethical considerations	Requires a high degree of integrity and honesty in evaluating and communicating the outcomes from practice
Further references	Douglas, A. and Scopa, K. (2000) Research through practice: positioning the practitioner as researcher – http://www.herts.ac.uk/artdes/conex/res2prac
Examples of the role of practice in research for higher degrees	– Investigating the subject/content/context of one’s own activity in order to advance or innovate; understanding one’s own creative process – Discovering something new about ways of approaching/doing your practice by experimentation – Understanding a range of different practices – using your practice to contrast/compare those of the practitioners – Using the skills of the artists/designer to visualise and understand complex processes (perhaps in other field) – making the invisible visible.

Observation:

Definition/function	To watch something closely and accurately record it to capture data relevant to the research project issues (note how you need a firm hypothesis i.e. it’s not possible to use an observation model if you’re expecting yourself to record everything)
Context for use	SO: self-observation. For example a designer recording themselves make work in order to become more aware of their creative process. NPO: non-participant observation. Researcher attempts to remain objective and detached from the thing being observed. PO: Participant observation. The researcher engages in the activities they set out to observe, becomes involved, empathises, observes through the experience of engagement.
Tools required	A range of qualitative and quantitative techniques.
Advantages	Relatively easy.
Disadvantages	All kinds of observation can be obtrusive to natural activities, perhaps causing changes in behaviour (the Hawthorne effect) PO requires trust to be gained from other participants SO can be too self-conscious unless it is carried out regularly
Ethical considerations	Possible invasion of privacy
Further references	Robson, C. (1993) Real World Research, Chapter 8, Observational Methods; Chapter 3, Ethical Considerations, pp. 29-35 (Sage)

Visualisation:

Definition/function	Visual thinking, making visible ideas through a range of techniques in order to explore research project issues and/or present research findings
Context for use	In practice-based research: making use of visual expertise to externalise and communicate concepts, interpretations, evaluations, and so on In collaborative research: a means of sharing information, gaining feedback and generating new ideas
Tools required	Drawing, diagrams, concept maps, mind maps, flow charts, storyboards, matrices, network displays etc.
Advantages	Visual overview of complex material improves its digestibility Increase importance of the visual in the 21st century, so, makes sense
Disadvantages	Possible or deliberate ambiguous nature of some visual material
Ethical considerations	All images are socially and technically constructed – visualisations should not deliberately mislead, should be used with integrity and care.
Further references	Tufte, E. (1997) Visual Explanations: Images and Quantities, Evidence and Narrative (Graphics Press).

Photography:

Definition/function	Providing evidence of particular significant features related to the research project issues, or part of the practice
Context for use	Analysis of visual data, presentation of research findings. For research purposes, photos need to be annotated, identifying the kinds of data presented in order to be useful evidence.
Tools required	Appropriate hardware. Suggest standard proforma for consistent annotations.
Advantages	Ubiquitous technology and presents complex detail
Disadvantages	Annotation is essential yet often overlooked
Ethical considerations	Possible invasion of practice. Edited works should not deliberately mislead. Should be used with integrity and care.
Further references	Banks, M. (2001) Visual Methods in Social Research (Sage)

Video data: as above

Sketchbook:

Definition/function	A portable book or bound collection of sheets that contain the development of ideas over time, and subsequent reflection and analysis. Can be analogue or digital.
Context for use	In practice-based research, to capture and store a range of visual ideas, data/information, to encourage visual thinking. Could be seen as a log book equivalent.
Tools required	Relevant analogue or digital tools
Advantages	Availability and accessibility in most situations, non-preciousness, a space for experimental thought, capacity for tracking developments over time, reflective and analytical space
Disadvantages	Some practitioners may prefer a more haptic/hands-on approach – there are limits to 2D and sequential structures
Ethical considerations	Possible reluctance to make accessible such a personal document, or to be meaningful honest with it knowing it will eventually be public. Should be used with integrity and care.
Further references	Klee, P. (1968) Pedagogical Sketchbook (London: Faber and Faber)

3D models/maquettes

Definition/function	Physical models or computer model
Context for use	In practice-based research, where two-dimensional information in insufficient for understanding and communicating ideas/structure/form/texture etc.
Tools required	Any relevant material and process, or computer-aided design
Advantages	Allows exploration and presentation of spatial and tactile data
Disadvantages	Models/maquettes take up space. Often required to be documented via photo/video.
Ethical considerations	Should not deliberately misrepresent e.g. misleading scale.

Reflective journal/research diary

Definition/function	Reflective journaling is a purposeful process and framework to help you explose and explore various models of practice. The journal is a repository for a range of information which is added to and consulted on a regular basis. Probably contains lots of different information.
Context for use	In practice-based nature, to help capture the dynamic and reflexive nature of practice. Invaluable for the regular documentation of the application of methods and evaluation of their outcomes.
Tools required	Off-the-shelf large format diary or a digital version. You may want to prep space for a development log, documentation of work in progress and completed, contextual references etc.
Advantages	A comprehensive store of practice-based thought and action, with evidence and example. Could form an important part of a ‘portfolio’ of research evidence/learning
Disadvantages	An idiosyncratic set of information, possibly little use to others but value to you as a reflective tool.
Ethical considerations	Should provide an honest picture of development and progress.
Further references	Newbury, D. (2001) ‘Diaries and field notes in the research process’, Research Issues in Art, Design and Media, Issue No. 1, Autumn.

Audio reflection (as above)

Case study

Definition/function	Case – a single instance or example of something. ‘A formal collection of evidence presented as an interpretative position of a unique case… reports on a project or innovation or event over a prolonged period of time or by telling a […] story as it has evolved.’ (McKernan, 1998, p. 74)
Context for use	Where a complex something/someone/situation needs to be studied qualitatively, intensively, in-depth and comprehensibly. Similarly, in Art and Design research where the case may be a practitioner, an environment, for example a studio/workshop, a project, a commission, a consultancy, a learning setting and so on.
Tools required	Can be all sorts of data, any imaginable.
Advantages	Lets you study something in depth and breadth, reality, use of multi-methods aids corroboration and validation, produces credibly and accurate account of setting and action.
Disadvantages	Specific and idiosyncratic example/case, so generalisations beyond the specific case can’t be made (unless a load of cases are studied). Super time consuming, critical view sometimes difficult as researchers can be influenced by respondents (if used).
Ethical considerations	Permission/authorisation required, ensure permissions have been given for use of data, used with integrity and care.
Further references	McKernan, J. (1998) Curriculum Action Research: A Handbook of Methods and Resources for the Reflective Practitioner, Chapter 4, Observational and Narrative Research Methods, pp. 74-83 (Kogan Page).

Interview

Definition/function	A key method in seeking the particular opinions of others about an identified research topic. A purposeful conversation initiated by the interview and focused by him/her on content specified by research objectives.
Context for use	In case study research where a particular and in-depth view is required, helpful in the exploratory stages of research to elicit views, identify variables, important factors as a method of structuring further research.
Tools required	Recording equipment, could set up a listening room.
Advantages	A good way of finding out a person’s values, preferences, attitudes, beliefs and feelings; opportunity for direct verbal interaction, encouraging in-depth response – a discursive method, can be done at a distance using telephone or online.
Disadvantages	Prone to subjectivity and bias (leading questions), the setting of the interview can influence the quality of the responses, for example formal or informal, time-consuming, if recording tools and notes are not used the interviewer could have poor recall.
Ethical considerations	Participant(s) must have given authorisation before recording, ensure confidentiality, ensure permissions have been given for public use of data, edited work should not deliberately mislead – editing can present an extremely selective, and possibly distorted view, should be used with integrity and care.
Further references	Keats, D. M. (2000) Interviewing: a Practical guide for Students and Professionals (Open University Press)

Questionnaire

Definition/function	A key method in seeing the general opinions of many others about an identified research topic. A purposeful survey in order to gather data at a particular point in time to describe the nature of existing conditions or identify standards against which existing conditions can be compared, or determine the relationship that exists between specific events.
Context for use	One of the most commonly used descriptive methods; useful for identifying trends in large populations, can yield good generalisations.
Tools required	Paper or preferably web-based proforma. Design should be simple but engaging; sampling and size are important – who are involved and why?
Advantages	Capacity for a large and widely distributed sample; more economical in time and money than interviewing; anonymity of respondents may encourage greater honesty; allows for quantitative data to be collected.
Disadvantages	the larger the sample the more generalised the response; the larger the sample the more time it will take to analyse; low return rates (20% is considered good!), no clarification of responses possible (if you’ve ensured it is anonymous), responses may be questionable either due to poor design or as a result of ‘questionnaire fatigue’ on the part of the respondent; if quantitative data are selected then statistical analysis is often required in order to establish whether the results are statistically significant.
Ethical considerations	Questionnaire MUST include information about the purpose of the research and how the resulting data will be used; ensure confidentiality; should be used with integrity and care.
Further references	McKernan, J. (1998) Curriculum handbook research: a handbook of methods and resources for the reflective practitioner, Chapter 4, non-observational, survey and self report techniques (Kogan page).

The benefit of choosing multiple methods:

Using different methods allows you to get a fox on something in order to understand more fully the complexity of issues by examining them from different perspectives, and generating data in different ways by using different methods. The more information we have from varying perspectives, the more able we are to test our ideas and eliminate bias that might arise from each method. So, choose your convoy!

• As always, remember to state your rationale for using several methods, and to describe clearly how they interrelate.
• You may want to map out your different methods visually to help you to rationalise and explain your decisions when it comes to communicating your study.

4.4 Considering preliminary evaluation and analysis

It’s important to be able to play with your data. No, really.

Sort the data in as many different ways as possible (do not discount anything initially)

Making data as visual as visual as possible (using colour coding, symbols, mapping, and so on) so that patterns can be seen, for example recurring themes.

Discus the data with participants/collaborators for feedback

Fundamentally, remember to make the invisible, visible. The implicit, explicit.

It’s an iterative process:
– As you generate and gather evidence, you must be constantly weighing up its value, quality, and potential significance.
– You should go over old ground and ensure you haven’t missed anything that could be valuable. Keep an open mind, and be critical in order to avoid only seeing those things we want to see – a ‘mirage of an oasis in the desert’. (p. 125)

Ch. 5: Interpreting the map: methods of evaluation and analysis

Evaluate your methods and methodology:

If you find that your methodology is inappropriate, this throws the validity of the research into question.

An example of this is how early research into creativity adopted a purely scientific approach, trying to ‘measure’ and ‘explain’ the ‘results’ rather than understand the person and the process in relation to the outcomes.

How do you do this? Speak with dissertation supervisors and peers. Do they feel that your research is generating/yielding good quality data? Are there flaws in the research methods e.g. inaudible interview recordings, incorrect transcription etc.

Ideally, you should revise the study and reapply a more appropriate methodology/use your methods more effectively. If that isn’t a possibility, then discuss the limits of your research in your dissertation. Critically evaluate them as part of the analysis of your work. (p. 129)

Validity and reliability: towards research quality

In scientific methodology, validity and reliability are the corner stones of research. They are also important in our fields, but perhaps we should think of it more as ‘trustworthiness’. To establish trustworthiness – ask yourselves these questions:

• Have you don a good, thorough and honest job?
• Have you tried to explore, describe, explain in an open and unbiased way?
• Or are you more concerned with delivering the required answer or selective evidence to support a case?

If you can’t answer these questions as yes, yes and no respectively, then your findings are worthless (Robson, 1993, p. 66)

Spectacles and sieves: criteria

We know that evaluation and analysis is important in research, but nothing can be evaluated or analysed without criteria with which to make judgements/assessments.

For example, you cannot analyse or evaluate what is ‘good design’ without criteria for doing so. For example, you might pick three criteria: effectiveness, efficiency, economy, Then these are also unpacked to provide more focused criteria (which will make your job easier) i.e. ‘effectiveness in relation to context and aesthetics; efficiency in terms of function and use; economy in terms of cost and use of materials.

Criteria allow us to focus, capture and distil value and meaning.

Never assume that your initial set of criteria is perfect: the only way that you’ll find out if they work or not is to play around and try them out. As soon as you apply them, you might realise they are unsuitable. Like always, it’s a good idea to talk through your criteria with a peer or supervisor. That gives the other person the opportunity to reveal strengths and weaknesses you might not yet have considered. (p. 131)

Final considerations for analysis:

• Analysis shouldn’t be the last phase of your research process. It should happen throughout and is cyclical and iterative, serving to inform and drive your work.
• The broad aim of analysis is to look for meanings and understanding.
• Analysis should achieve a sense of the whole
• Analysis is a reflective activity, moving from the data to a conceptual level. It is helpful to track this process of reflection using a reflective journal – it helps you shift from the details to the bigger picture, and also provides accountability of the analytical process.
• A key tool of analysis is comparison – exploring similarities and differences will help you to form categories and boundaries for your data.
• Be alert to the limits of evidence – if it’s inadequate then you need to acknowledge it.
• Important to recognise that analysis is never exhaustive and never finished. It is ‘kind-of’ completed when the research question has been addressed, so it’s important to state to the reader the scope and confines of your analysis. Analysis only ‘ends’ after new data no longer generates new insights, thus exhausting the data.

Framework for analysis:

The use of existing frameworks for analysis helps to link your research outcomes to established research.

Share an example of an analysis framework in the presentation i.e. hyperrational reading of an object etc.

Making sense

The outcome of analysis should be to create a ‘big picture’. Ideally, ideas that are valuable and applicable in broader contexts.

Your interpretation must be based on the available evidence, so all research evidence must be accessible to your reader so that they could make their own interpretations if they wanted to. Include primary data in appendices, and make sure it’s accessible.

An example of an experimental framework for analysis:

A good examples of visual analysis to draw on:

• Visual Explanations: images and quantities, evidence and narrative (1997) Edward R Tufte (requested e-book from library, but lots of copies available in main collection at 741.622 TUF)

Three major activities in analysis:

• Data reduction
• Data display
• Drawing conclusions from the above two

This is Miles and Huberman’s ‘three concurrent flows of activity in analysis (p. 144). We can break it down further:

• Data reduction: any structure or tools that encourage you to sort, select, focus, order, simplify data; for example, applying criteria, coding data by colour of symbols, condensing, grouping, clustering etc.
• Data display: any structure/tools that present data in an organised and usually compressed visual format so that the reader can gain an overview and understanding of the whole (literally ‘see’ what you mean
• Drawing conclusions: interrogate i.e. how many times, are themes apparent, what relationships exist, how does this relate to existing concepts etc. (p. 145)