mapping, in-depth interviews (life history and participation narratives), ethnography and social network analysis in six
“I think the boxes will have to get larger” Equity and polity in measuring quality & articulating values Dr. Abigail Gilmore, University of Manchester www.everydayparticipation.org @Abi_Gilmore
Excellence Quality Value ‘Art’ Triangles Circles (Square) boxes NESTA R&D Culture Counts/Quality Metrics AHRC Connected Communities Understanding Everyday Participation – Articulating Cultural Values
Performance assessment mechanisms for managing accountability relationships (Chiaravalloti, 2016) between public, policy communities & arts organisations – Holden’s ‘golden triangle’ of public value (2006) Technocratic procedures characteristic of systemic New Public Management (Zan, 2000; Belfiore, 2004; Gray, 2002) but growing interest in ‘data-driven decisionmaking’, big data and machine learning (e.g. Lilley & Moore, 2014) The articulation of artistic value, quality and excellence - highly contingent on cultural capital and power relations between these stakeholders.
Public/community
Participation and co-curation as a means to increase public value and cultural democracy (Jancovich, 2015) What are the ‘ways in’ to judgement, evaluation and decision-making? How are publics represented within these processes of measurement? Policy-oriented/ customers
Institutions/ professionals
Funders prefer quantifiable indicators of artistic quality; require transparency as well as efficiency (Carnwaith & Brown, 2014, Mowlah et al, 2014) Arts organisations concerned with the quality of production processes; meeting original objectives Artists/producers technical execution; do artworks contribute to art form, larger body of work (Bunting, 2007) ‘Public’ audiences more concerned with their emotional response, but discomfort with the term ‘quality’, believing that their own subjective definition of the term would not be as valuable as that of an artistic expert (Florack, 2015)
NESTA/AHRC/ACE Digital R&D Project exploring benefits of digital platform providing shared measures for evaluating arts & cultural experiences App-based pre- & post-event questionnaire asking about ‘quality dimensions’ for public, self & peers Academic research to explore value of approach within wider context consortium of cross-art form partners, with Australian tech partners, Culture Counts https://culturecounts.cc/ & University of Manchester www.culturemetricsresearch.com
Sel f thea tre 1.00
1.00
Publ i c thea tre 1.00
0.98
1.00 0.92
Peer thea tre
0.93
0.92
1.00 0.97
0.91
0.90
0.87
0.90
1.00 0.94
0.90
0.90
0.84
0.81
0.79
0.80
0.71 0.70
0.70
0.72 0.68
0.66
0.60
0.65
0.54 0.50
0.57 0.50
0.48
0.50 0.40 0.30 0.20 0.10
Im pa ct Lo ca l
sm us ia En th
ni ng M ea
at io n Ca pt iv
Ch all en ge
an ce lev Re
Ri go ur
in ct ive ne ss Di st
tio n en ta Pr es
Co nc ep t
0.00
Post-event surveying limited to conscious impressions/short term impacts (Carnwath & Brown, 2014). Audience development approaches tend to privilege institutional perspectives (Lindelof), and neglects socalled non-users & everyday forms of cultural value (Miles & Gibson, 2016) Qualitative research can have multiple advantages, including enriched arts experience & longer engagement (Walmsley, 2012; Radbourne et al, 2009) Audience anxiety about lack of expertise; positive responses to evaluation - leading to a virtuous circle (Johanson & Glow, 2015)
AHRC Connected Communities research programme 2012 - 2017 Core research questions Decontextualised notion of“value” in cultural policy which bolsters ‘deficit model’ of participation (Miles & Sullivan 2012) What is ‘cultural participation’? What does it ‘do’ and and why does it matter? [value] What is the relationship between participation and place? Policy-facing aims: Help organisations to promote participation by reconnecting policy and practice with the everyday realm Address issues of democratic failure in cultural policy Understand how communities are negotiated through culture
Histories – discourses of cultural participation and value; cultural technologies; cultural policy, place and economy; representations of everyday life; community cultural practices Reanalysis of survey data – time-use, patterns of participation by place and through the life course Cultural ecosystems research – local histories, assets mapping, in-depth interviews (life history and participation narratives), ethnography and social network analysis in six contrasting locations Application projects – working with communities and partners to develop projects in response to findings Research-policy-practice nexus – reflecting on partnership and developing new models of collaboration
Western Isles
Aberdeen Gateshead
Manchester/Salford Peterborough
Dartmoor
•
Participation in established cultural forms a minority affair (8.7%...), often pursued with ambivalence, still implicated in ‘distinction’
•
No such thing as a ‘non-user’. Participation in the everyday realm is prolific, usually highly committed, often very creative
•
Most participation is not about the individual , but rooted in tradition, habit, community with the emphasis on family, friendships & sociality
•
It is participation per se rather than in any particular form that matters for ‘well-being’
•
Distinctive historical shift from engagement with ‘high cultural forms and ‘traditional working class culture’ towards practices associated with ‘emerging cultural capital’ (following Williams)
•
People don’t participate nationally
•
The spaces and places where people participate affect the dynamics, meanings and relations which in turn impacts on understandings of /identification with ‘place’
•
Sense of place and belonging mediated through local participation, understood through the historical configuration of places – their economies, infrastructures, cultures of governance and the biographies and mobilities of their residents
•
London/provincial (rather than a North-South) divide in regional patterns of participation, with considerable provincial variation which seems to be driven by issues of ethnicity and gender
1. How do we understand quality and value? What kinds of methods and epistemologies? Social life of methods The ‘standardisation’ issue The quantification issue 2. How to let the public in (for public value)? Mixed methods for community involvement participatory programming deliberative decision-making genuine co-produced evaluation greater cultural democracy
Bailey, J. and Richardson, L. (2010) Meaningful measurement: a literature review and Australian and British case studies of arts organizations conducting “artistic self-assessment”, Cultural Tends, Vol. 19, No. 4, pp291-306 Boorsma, M. and Chiaravalloti, F (2010) ‘Arts Marketing Performance: An artistic-mission-led approach to evaluation’. Journal of Arts Management, Law and Society 40 (4) pp.297-317 Bunting, C. (2007, November). Public value and the arts in England: Discussion and conclusions of the arts debate London: Arts Council England Carnwath, J. D., & Brown, A. S. (2014, July 14). Understanding the value and impacts of cultural experiences. London: Arts Council England Chiaravalloti, F. (2016) Performance Evaluation in The Arts: from the Margins of Accounting to The Core of Accountability, PhD Thesis, Groningen: University of Groningen Florack, F. (2015) Literature Review on Co-Production of Cultural Value, unpublished report for Culture Metrics. Holden, J. (2006) Cultural Value and the Crisis of Legitimacy. London: Demos. Jancovich, J. (2015) Breaking Down the Fourth Wall in Arts Management: The Implications of Engaging Users in Decision-Making International Journal of Arts Management, Vol 18/1 pp 14-28 Johanson, K., & Glow, H. (2015) A virtuous circle: The positive evaluation phenomenon in arts audience research. Participations, 12 (1): 254–270. Lilley, A., & Moore, P. (2013). Counting what counts. London: NESTA/Arts Council England. Lindelof, A. (2015) Audience development and its blind spot: a quest for pleasure and play in the discussion of performing arts institutions, International Journal of Cultural Policy, Vol 21, No. 2, pp.200-218 Radbourne, J., Glow, H., & Johanson, K. (2010). Hidden stories : listening to the audience at the live performance. Double Dialogues, (13), 1–14.; Radbourne, J., Johanson, K., & Glow, H. (2009). Empowering audiences to measure quality. AIMAC 2009 : Proceedings : 10th International Conference on Arts & Cultural Management, 1–12. Walmsley, B. (2012) Towards a balanced scorecard: A critical analysis of the Culture and Sport Evidence (CASE) programme, Cultural Trends, 21:4, 325-334, .
