Social media as semiotic technology and social practice: the case of ResearchGate’s design and its potential to transform social practice

ABSTRACT

This article addresses the need for critical approaches to social media by bridging the focus on language and other semiotic resources that characterises discourse studies with the broader perspective on social media as social, cultural, economic and technological constructs that dominates media and cultural studies. Specifically, we propose a model for analysing how social media as semiotic technologies, that is, technologies designed to enable and constrain meaning-making, may transform social practices. By incorporating Van Leeuwen’s [2008. Discourse and Practice: New Tools for Critical Analysis. London: Oxford University Press] framework for the critical analysis of discourse and social practice, the model extends the social semiotic approach developed in recent critical multimodal studies of software such as PowerPoint to social media, which function primarily to provide platforms for and commodify social practices, rather than to offer rich arrays of semiotic resources for creating multimodal texts and artefacts. Using the academic social network site ResearchGate and the practice of research peer review, we illustrate the model’s capacity to account for the ways the design of social media platforms – through the semiotic resources they make available and the ways these are presented – enables and constrains their users’ ability to perform key social practices and has the potential to transform these practices.     

Semiotic technologies: a case study of discipline-based practices and pedagogy

This paper examines semiotic technologies, both in terms of the resources they harness and the practices developed around their use. It draws on data collected as part of an ethnographic investigation into the meaning-making practices deployed within civil engineering study. The data is used as a case study for examining semiotic technologies as socially situated resources for disciplinary practices. Using a multimodal social semiotic approach, we argue that technologies are not self-evident, and that their use constitutes specific social practices that require development in the classroom. In order to deploy technologies in pedagogically effective ways, we need to understand the semiotic resources they draw on (including embodied resources). Awareness that technologies are not neutral or value-free, but are socially situated and ideologically laden, may enable meta-level understanding of the discipline, thus creating the possibility for improved pedagogical practices.     

Combining autoethnography and multimodal social semiotics: potentials for theory and method

Different methodological approaches allow varying access to the objects of inquiry and enable one to ask different kinds of questions. This paper explores the possibilities that emerge through the combination of multimodal social semiotics and autoethnography. We discuss the epistemological and methodological bases for each approach and show the potentials in combining them. Drawing on an autoethnographic study in civil engineering, we argue that three aspects of research inquiry are enhanced through the combination of these approaches. These three aspects include a greater understanding of social context and its impact on meaning-making; an awareness of the multimodal nature of meaning-making; and an understanding of the ways in which the “interest” of the researcher (as participant) manifests in texts. Overall, we argue that these approaches augment each other in generative ways and allow for an in-depth look at how texts are embedded in particular practices and communities.     

Analysing the teachers’ use of gestures in the classroom: A Systemic Functional Multimodal Discourse Analysis approach

Classroom research in the past has mostly been based on the study of the language used by the teacher and students. With the availability of video recording technologies, it is now possible to collect multimodal classroom data, such as the teachers’ use of gestures, positioning, and classroom space to enact specific pedagogy. The challenge for the researchers is how to annotate and analyse such multimodal data. This paper proposes an approach for the annotation and analysis of the teachers’ use of gestures in the classroom for meaning-making. Situated within Systemic Functional Theory, the Systemic Functional Multimodal Discourse Analysis approach to the formal and functional classification of teachers’ use of gestures in the classroom is introduced. The approach builds on present research and theoretical conceptions on gestures and extends the classifications with examples from a selection of instances from an authentic classroom corpus. The aim is to develop the theoretical apparatus for the annotation and analysis of the teachers’ use of gestures in the classroom.     

The expanding galaxy of performing arts: extending theories and questioning practices

This paper introduces the Special Issue on the languages of performing arts and is therefore aimed at designing how the context of the latter can be illuminated by socio-semiotic and multimodal approaches to communication. In this Special Issue, performances and performing arts are described as multimodal semiotic acts that co-deploy a range of semiotic resources to produce and construct meanings across different cultures and ages. Seen as dynamic and interactive processes of meaning-making, their analysis calls for new and multidisciplinary frameworks which are collected in this Special Issue. The introduction gives an overview of these papers and discusses their range of diverse phenomena, both live and recorded, including theatre performances and films, art installations, opera, as well as reading out aloud. By outlining the significance and contribution of different disciplines and fields of studies to the broad area of performance studies, the chapter argues the case for innovative approaches that can extend theories and analyse aesthetic and performative practices in context. With the help of some case studies, it provides guidelines for the reading and interpretation of the several theoretical discussions and practical case studies presented to encourage further multidisciplinary research on these domains.     

Studying social media as semiotic technology: a social semiotic multimodal framework

ABSTRACT

How do we study social media technology? While social semiotics provides an extensive toolkit for analysing multimodal texts and semiotic practices, the study of social media as semiotic technology poses a significant challenge to existing research methodologies. In this article, we present a social semiotic framework that allows us to describe in analytical details the multimodal meaning potentials offered by digital social media technology and connect these to multimodal text-making and semiotic practices while underscoring the role of technology. Our framework is organized around seven interrelated and inherently informed dimensions: (1) multimodality, (2) practice, (3) the social, (4) medium, (5) the material, (6) the historical, and (7) the critical. This framework could pertain to most types of semiotic technologies, but will here be developed for accounting for social media technologies, and its viability will be illustrated with examples from Instagram. By developing this framework, we aim at elaborating the theoretical basis and analytical tools of social semiotics, and thereby contributing to bringing forward increased understanding of how social media technology enables making, enacting and managing meaning.     

Innovation and Alliances

Explanations for national success in science and technology typically focus on domestic institutions and policies. However, that line of research has yet to identify any particular set of institutions or policies that explain variation in national innovation rates across cases or over time. This article offers new evidence that the problem with domestic institutions approaches stems from their failure to consider international security factors. Specifically, this article finds a positive effect for U.S. security alliances on innovation. This finding is robust across different specifications and periods of analysis. While countries that ally militarily with the United States are found to realize benefits in economy‐wide, indigenous innovation, such an effect is not observed in military technologies. This suggests that alliances may substitute for being on the frontier in military technologies. Therefore, this article contributes not just to debates over S&T competitiveness, but also to alliance formation.     

‘Lending by numbers’: credit scoring and the constitution of risk within American consumer credit

Abstract

This paper examines how statistical credit-scoring technologies, sanctioned by the state in the interests of promoting equality, became applied by lenders to the problem of controlling levels of default within American consumer credit. However, these technologies, constituting consumers as ‘risks’, are themselves seen to be problematic, subject to their own conceived sets of methodological, procedural and temporal risks. Nevertheless, as this article will show, such technologies have increasingly been applied to other areas of consumer lending, thus interpreting a wider array of operational contingencies in terms of risk. Finally, it is argued that, since the 1980s, the constitution of credit consumers as risks has been deployed to new ends through technologies of ‘profit scoring’ and new practices of ‘risk pricing’.     

A framework for the multimodal analysis of online news galleries: What makes a “good” picture gallery?

The development of digital technology in recent years has led to a revolution in news production and dissemination. In terms of production, we have witnessed a fundamental shift towards visual story-telling. Images dominate the verbal story space and have the potential to become the story themselves. Beyond this, they are also creating unique spaces for themselves (e.g. the online news gallery), with new multimodal genres posing challenges for practitioners and analysts alike. The potential effects of such fundamental shifts on the professional news story-telling practices of the legacy news media provide a rich research opportunity for understanding both how and whether news organisations fulfil their mandate of making sense of the plethora of information that is now available. In this paper, we concern ourselves with one particular innovation in visual news reporting – the online news gallery, or picture gallery in journalism terms. We report on a qualitative analysis of 35 galleries from 12 English-language newspapers with online presence from Europe, North America, Australia and Asia, and interrogate the choices made by institutions in composing this particular type of multimodal “text”, as it is situated in a professional, news story-telling context. In doing so, we explore how a systemic-functional semiotic approach to multimodal news discourse may help us to access the meaning potential of this emerging genre as a vehicle for multimodal digital news reporting, present a framework for the multimodal analysis of online news galleries and consider its implications for the education of media practitioners.     

Multimodal turn allocation in ESL peer group discussions

This article reports on multimodal practices used by English as a Second Language students as they work to distribute primary speakership within their peer group discussions. Following Goodwin’s participation framework, the focus is on the embodied conducts of the non-talking recipients and their nonvocal displays of emerging speakership in peer discussions. Analyses of the non-primary speakers’ gaze, gestures, touch, and bodily conduct show that the students’ turn allocation practices embody their sensitivity to the spatial and visual field of co-participants, project changing participation frameworks, and achieve incremental coordination of speaker nomination. Explorations of such nonvocal behaviors lead to a detailed understanding of the students’ embodied participation frameworks and the visible processes through which the students claim or avoid speakership during peer group discussions.     

From book to stage to screen: semiotic transformations of Gothic horror genre conventions

ABSTRACT

This paper adopts a multimodal social semiotic approach for exploring the semiotic changes involved in the transformation of a novel into stage and screen productions. It examines how semiotic resources are deployed in each medium through elements of mise-en-scène, such as speech, music, sound, lighting, props, staging, and cinematographic techniques, and the viewing perspectives that are thus established for audiences. The genre of Gothic horror is selected for this purpose, given how this form of performance has transfixed audiences for centuries and has been adapted for both the stage and the screen. In order to demonstrate how each performance medium has produced its own unique set of foregrounding devices to enthral and captivate audiences, a comparative analysis of excerpts from the novel The Woman in Black by Susan Hill, a videotaped theatrical performance, and the 1989 British television film of the same name is undertaken. The paper discusses the implications of the multimodal semiotic approach for developing a better understanding of the semiotic transformations that horror genre conventions undergo in different media and the viewership positions that are thus re-drawn for audiences. The paper concludes with a view of multimodal recontextualisation processes which form the underlying basis of human sociocultural life.     

Diffusion milieus as a focus of research on innovation in the public sector

Public policies to increase the efficiency of state and local governments through fostering the use of new technologies are based, implicitly at least, on theories of diffusion. Existing theories of diffusion have been found to be deficient. This paper presents an approach to the diffusion of technological innovations in state and local governments organized around the concept of diffusion milieus.A diffusion milieu contains those elements external to the organization which bring pressure to bear on the organization to alter its existing practices, which define the range of feasible alternatives to these practices, and which condition the information flow on the performance characteristics of the proposed innovation(s). These milieu components are then filtered through the internal decision-making process(es) of the organization to produce a decision to accept or reject the innovation(s). More specifically, the milieu contains (1) the specific nature of the demands upon the agency to change behavior (performance gaps), (2) the supply of alternatives to existing practices (technological change), (3) agency-executive relationships, (4) the activities of suppliers in marketing innovations, (5) the resources available to the agency to acquire new techniques, (6) the characteristics of the knowledge infrastructure which affect the search behavior and the degree of uncertainty associated with adopting new techniques, (7) the impact of intergovernmental relationships, and (8) citizen demands. The components of the diffusion milieu are then (9) related to a set of agency decision-rules concerning the adoption of new techniques.The rate of diffusion in each field will depend upon themagnitude of the components and on thelinkages between them.Functional fields differ in both respects.Research for this paper was supported by a grant from the National Science Foundation, Office of R & D Assessment. We have benefited from the helpful comments of Professor Marvin Rozen, an anonymous referee and the research assistance of Ms. Lee Ann Kozak. The views expressed in this paper are the sole responsibility of the authors. An earlier version of this paper was presented at the 1975 American Political Science Association meetings.     

Communicating sociocultural norms and social and religious practices in Iranian death notices

Drawing on a corpus of 100 randomly collected death notices published in an Iranian national newspaper, this study examined generic structures as well as their lexico-grammatical patterning to reveal what communicative functions are articulated by these generic components. Basing our identification of moves on Swales [1990. Genre Analysis: English in Academic and Research Settings. Cambridge: Cambridge University Press], we identified 12 rhetorical moves in the corpus. We illustrated that these recurrent rhetorical moves reflect the Iranian sociocultural norms, as well as social and religious beliefs. Also, the results substantiated [Kress, Gunther, and Theo van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. London: Routledge] multimodality perspective on textual analysis in that they rejected linguistic items as the only meaning-making device and identified colour, position, photo and size, among others, as worthwhile veritable semiotic devices.     

Defensive semiotic strategies in government: a multimodal study of blame avoidance

In this article, I argue that when officeholders try to avoid blame, they often engage in defensive performances which can be analysed in terms of how they exploit various multimodal semiotic resources to ward off (potential) criticism: the setting, appearance, and manner chosen for a particular interaction with an audience; and both verbal and non-verbal ways of arguing, legitimising, framing, and representing social actors and actions. I apply these analytic categories to interpret the data gathered during fieldwork at a major training event of British government communicators. By combining insights from multimodal discourse analysis, discourse-historical studies of organisational behaviour, and recent research into blame avoidance in public administration, I demonstrate how certain semiotic strategies used by officeholders have an effect of backgrounding the ideas about any possible norm violations that government communicators may have been associated with in the eyes of critical audiences, such as lying, spin doctoring, and using tax money for propaganda campaigns that may not actually serve the interests of the public. I suggest that analysts of government communication should pay more attention to the defensive practices of government insiders, and study in great detail how these practices are incorporated into everyday behaviour through professional training.     

New Region,New Story: Imagining Mobile Subjects in Transnational Space

This paper investigates the changing relations between citizens and Europe's internal borders by seeing these relations as objects of governmentality of mobilities. It focuses on the particular discursive space of one transnational ‘corridor in the making’, studying how, through practices of visioning and strategy-making, governmental actors make new transnational governmental territories by breathing life into the imagined mobile subjects of these future territories. These mobile subjects play a central part in the politics of emergent transnational governmentalities and in legitimising potential border-crossing infrastructures. In a case study of the COINCO project (Corridor of Innovation and Co-operation), linking Oslo, Gothenburg, Copenhagen, Malmö and Berlin, it is shown how deconstructing imagined mobile subjects reveals the will to power over mobility and sheds light on how governmental practices are remaking European borders.     

Scopic valuations: how digital tracking technologies shape economic value

How do market actors in the contemporary economy make decisions when they are uncertain of the value of what is being traded? To answer this question, this paper analyses the changing temporal and spatial orientation of economic valuations in increasingly digitalized economies, in which markets can no longer be understood as locally bound and closed sites of valuation and exchange. Drawing on ethnographic material from the fishing industry, the paper shows how digital tracking technologies link and synchronize auction markets with real-time information on fishing activities. Hence, ‘scopic media’ such as chart plotters and computer screens are not only deployed by skippers to monitor one another, but are also used as valuation devices that recontextualize local knowledge for the purpose of coping with economic uncertainty. As a consequence of this panoptic valuation regime, fishermen control their virtual identities by disciplining harvesting practices according to their buyers’ expectations.     

Border work: surveillant assemblages,virtual fences,and tactical counter-media

The new technologies of bio-informatic border security and remote surveillance that have emerged as key infrastructures of reconfigured mobility regimes depend on various kinds of labor to produce the effect of bordering. The current retrofitting and technological remediation of borders suggests their transformation away from static demarcators of hard territorial boundaries toward much more sophisticated, flexible, and mobile devices of tracking, filtration, and exclusion. Borders require the labor of software developers, designers, engineers, infrastructure builders, border guards, systems experts, and many others who produce the “smart border”; but they also depend on the labor of “data-ready” travelers who produce themselves at the border, as well as the underground labor of those who traffic in informal and illegalized economies across such borders. Bordering increasingly relies on technological forms of mediation that are embedded within hi-tech, military and private corporate logics, but are also resisted by electronic and physical “hacks” or bypassing of informational and infrastructural architectures. In this paper we consider three socio-technological assemblages of the border, and the labor which makes and unmakes them: (1) the interlocking “cyber-mobilities” of contemporary airports including visual technologies for baggage, cargo, and passenger inspection, as well as information technologies for passenger dataveillance, air traffic control, and human resource systems; (2) the development of the Schengen Information System database of the EU, and its implications for wider migrant rights and internal mobility within the EU, as well as radical border media that have attempted to intervene in that border space; and (3) elements of the US–Mexico “smart border” regime known as the Secure Border Initiative Network (2006–2011), and those who have tried to tactically evade, disrupt, or undermine the working of this border.     

U.S. Congressional Campaign Communications in an Internet Age

New technologies – with perhaps the most notable being radio and television – often change the face of political campaigns. The Internet, and particularly campaign websites with their concomitant technologies (e.g. interactive and multimedia features), has evolved at a faster rate than any other prior innovation. This raises a critical question: have website technologies altered how congressional candidates campaign? We address this question with a novel dataset from 2008. Not only do we chart technological change on sites over the course of the campaign but we also explore how and when candidates use certain technologies. We discover two critical and, to our knowledge, novel points. First, congressional candidates use these technologies to a much lesser extent than one may suspect. Second, their scant usage is driven by how certain technologies limit control of the candidate's message, the candidate's status in the race and other key variables such as the employment of campaign consultants. In sum, the Web 2.0 era (which began around 2008) does not appear to have dramatically altered congressional campaigns.     

An exploratory comparative analysis of the use of metaphors in writing on the Internet and mobile phones

Scant attention has been paid to the use of figures of speech in describing mobile telephony and how it is used. While there are isolated cases, as yet there is no study of metaphor use across a larger corpus of mobile phone research. This article addresses this gap by developing a preliminary survey, or mapping, of the use of metaphors and other discursive figures and tropes in the available literature on mobile phones. It does this by using metaphors in Internet literature as a springboard for comparative analysis, as a comparable communications medium. Both for their general role as communications technologies and their specific historicized position as “new media”, and with their capacities increasingly overlapping, mobile phones and the Internet form a useful counterpoint. The article summarizes the key findings from earlier work mapping and critiquing the use of metaphor in the critical and popular writing on the Internet. It then compares and contrasts these Internet-based metaphors against those in the mobile literature. The paper concludes with a discussion of the perils and the promise of metaphor use in writing on communications technologies.