Studying social media as semiotic technology: a social semiotic multimodal framework

ABSTRACT

How do we study social media technology? While social semiotics provides an extensive toolkit for analysing multimodal texts and semiotic practices, the study of social media as semiotic technology poses a significant challenge to existing research methodologies. In this article, we present a social semiotic framework that allows us to describe in analytical details the multimodal meaning potentials offered by digital social media technology and connect these to multimodal text-making and semiotic practices while underscoring the role of technology. Our framework is organized around seven interrelated and inherently informed dimensions: (1) multimodality, (2) practice, (3) the social, (4) medium, (5) the material, (6) the historical, and (7) the critical. This framework could pertain to most types of semiotic technologies, but will here be developed for accounting for social media technologies, and its viability will be illustrated with examples from Instagram. By developing this framework, we aim at elaborating the theoretical basis and analytical tools of social semiotics, and thereby contributing to bringing forward increased understanding of how social media technology enables making, enacting and managing meaning.     

Exploring construction project design as multimodal social semiotic practice

ABSTRACT

The semiotic nature of design and engineering communications has often evaded direct interrogation; specifically the representation of requirements using multiple semiotic modes. This paper explores the semiotic character of requirement representation in the construction project design and engineering process by mobilising a social semiotic analytic framework to reveal how project artefacts (e.g. drawings; digital imagery; physical objects) are multimodal resources that critically affect stakeholder engagement and interpretation. The focused empirical study from a hospital construction project reveals the motivations and effects of multimodal and semiotic communicative choices of different parties (e.g. client; architect; engineer; design consultant) engaged on the same project; the representation (or not) of requirements via various semiotic modes being a social and political process. The contribution of the paper is in revealing the semiotic character of requirement representation and the relevance of semiotic-informed enquiry for an industry continuing to develop new technologies (e.g. virtual reality; BIM – building information modelling) for design and construction work.     

Social media as semiotic technology and social practice: the case of ResearchGate’s design and its potential to transform social practice

ABSTRACT

This article addresses the need for critical approaches to social media by bridging the focus on language and other semiotic resources that characterises discourse studies with the broader perspective on social media as social, cultural, economic and technological constructs that dominates media and cultural studies. Specifically, we propose a model for analysing how social media as semiotic technologies, that is, technologies designed to enable and constrain meaning-making, may transform social practices. By incorporating Van Leeuwen’s [2008. Discourse and Practice: New Tools for Critical Analysis. London: Oxford University Press] framework for the critical analysis of discourse and social practice, the model extends the social semiotic approach developed in recent critical multimodal studies of software such as PowerPoint to social media, which function primarily to provide platforms for and commodify social practices, rather than to offer rich arrays of semiotic resources for creating multimodal texts and artefacts. Using the academic social network site ResearchGate and the practice of research peer review, we illustrate the model’s capacity to account for the ways the design of social media platforms – through the semiotic resources they make available and the ways these are presented – enables and constrains their users’ ability to perform key social practices and has the potential to transform these practices.     

The civilian's visual security paradox: how open source intelligence practices create insecurity for civilians in warzones

ABSTRACT

Images taken by civilians and shared online have become an important source of conflict intelligence. This article explores issues around how states and non-state actors appropriate civilians’ images to produce intelligence about conflict, critically scrutinizing a practice often called open source or social media intelligence. It argues that image appropriation for open-source intelligence production creates a new kind of visual security paradox in which civilians can be endangered by their everyday visual practices because their digital images can be appropriated by outside actors as conflict intelligence. The transformation of everyday images into conflict evidence relies on what Barthes termed the photographic paradox, the paradox that while a photograph is clearly not the reality it depicts, the photograph is casually interpreted as a copy of that reality. When images are appropriated as conflict intelligence this photographic paradox translates into a security paradox. A visual security argument can be made without the intention or knowledge of the image producer, who then comes to perform the role on an intelligence agent. Yet civilians in warzones can hardly refrain from producing any images when they need to call attention to their plight, and to stay in contact with friends and relatives. The paradox, then, is that such vital visual signs of life can rapidly become sources of danger for the civilian. This civilian visual security paradox, it is argued, demands that intelligence actors respect the protected status of civilians in their online collection practices. So far, however, there is little sign of such respect.     

Semiotic technologies: a case study of discipline-based practices and pedagogy

This paper examines semiotic technologies, both in terms of the resources they harness and the practices developed around their use. It draws on data collected as part of an ethnographic investigation into the meaning-making practices deployed within civil engineering study. The data is used as a case study for examining semiotic technologies as socially situated resources for disciplinary practices. Using a multimodal social semiotic approach, we argue that technologies are not self-evident, and that their use constitutes specific social practices that require development in the classroom. In order to deploy technologies in pedagogically effective ways, we need to understand the semiotic resources they draw on (including embodied resources). Awareness that technologies are not neutral or value-free, but are socially situated and ideologically laden, may enable meta-level understanding of the discipline, thus creating the possibility for improved pedagogical practices.     

Social capital,voluntary associations and collective action: Which aspects of social capital have the greatest ‘civic’ payoff?

Abstract

Despite a great variety of theoretical approaches, empirical analyses of social capital are surprisingly similar. Virtually all of them treat membership in voluntary associations as the chief indicator of community involvement while neglecting another form of community involvement: participation in elite-challenging actions. Likewise, authors readily attribute manifold civic benefits to associational life, while hesitating to attribute such benefits to elite-challenging activity. We question these views on two grounds. Firstly, we argue that elite-challenging action reflects social capital, even though this is a specific form of it: an emancipative form typical of self-assertive publics. Secondly, we use data from the Value Surveys to demonstrate that elite-challenging action is linked with greater civic benefits, at both the individual and societal level, than is membership in voluntary associations. This finding confirms the concept of human development, which suggests that emancipative forms of social capital are more civic in their consequences than others. Following this concept, we show that mass self-expression values nurture emancipative social capital, in motivating elite-challenging action. Finally, we locate self-expression values and elite-challenging actions in a theory of emancipative social capital.     

French engagement versus British commitment. Was it interlingual miscommunication or political manipulation?

ABSTRACT

The efforts undertaken to establish the contemporary European Union started many years earlier. The history of establishing a pan-European organization is interesting not only for historians and economists but also translators and linguists as it is the history of interlingual communication (negotiations and agreements). One such negotiation was allegedly ineffective due to the difference in understanding the French term engagement and its English equivalent employed by translators and interpreters, that is to say the English term commitment. The authors aim at presenting the political background of negotiations and the social semiotic analysis of the terms in question in order to provoke the reader to find the answer to the question whether the negotiation was broken due to interlingual miscommunication resulting from erroneously chosen equivalent or it was destined to failure from the very beginning due to socio-political and economic interests of negotiating parties. In order to achieve that goal, the authors have analysed notes exchanged between the governments of France and Great Britain in course of negotiation. The history of negotiation and the linguistic analysis are combined to illustrate the complexity of meaning construction and the semiotic implications of the contextual dependence of meanings of terms and their dynamic evolution in time and space.     

Lamppost networks*: stickers as a genre in urban semiotic landscapes

ABSTRACT

This paper examines the pragmatics of stickers as a genre prominent in communication in urban public space. Although normally small in size, stickers may quantitatively dominate signage in certain areas of cities. Stickers are examined here as localized communicative events that mediatize social practices through a range of complex multimodal and linguistic processes, based on data from the Digbeth area in central Birmingham, UK. An analysis of the distribution of stickers, their agency, audience, and the multimodal practices involved in their creation reveals that they bring together transgressive, artistic and commercial discourses and form a specific layer of urban communication, especially in areas of pedestrian transit within a city. A large number of stickers also initiate communications that can potentially be continued online.     

From book to stage to screen: semiotic transformations of Gothic horror genre conventions

ABSTRACT

This paper adopts a multimodal social semiotic approach for exploring the semiotic changes involved in the transformation of a novel into stage and screen productions. It examines how semiotic resources are deployed in each medium through elements of mise-en-scène, such as speech, music, sound, lighting, props, staging, and cinematographic techniques, and the viewing perspectives that are thus established for audiences. The genre of Gothic horror is selected for this purpose, given how this form of performance has transfixed audiences for centuries and has been adapted for both the stage and the screen. In order to demonstrate how each performance medium has produced its own unique set of foregrounding devices to enthral and captivate audiences, a comparative analysis of excerpts from the novel The Woman in Black by Susan Hill, a videotaped theatrical performance, and the 1989 British television film of the same name is undertaken. The paper discusses the implications of the multimodal semiotic approach for developing a better understanding of the semiotic transformations that horror genre conventions undergo in different media and the viewership positions that are thus re-drawn for audiences. The paper concludes with a view of multimodal recontextualisation processes which form the underlying basis of human sociocultural life.     

Essence,Identity, Signature: Tattoos and Cultural Property

This paper examines a range of problems centring on the theorization of cultural identity and cultural property by reference to debates about the appropriation of the Maori tattoo, or ta moko , and the authenticity of contemporary Maori tattooing practices. Through a consideration of the relationship between cultural identity and tattooing, and the question of whether tattooing is an effect of a specific identity or constitutive of that identity, it addresses the paradox inherent in attempts to protect indigenous, cultural and artefacts from (mis)appropriation: that is, that to re-articulate such non-Western cultural phenomena in terms amenable to their repositioning as property is precisely to render them meaningless or useless, in terms of their contextually specific uses and significance. Against the background of these issues, the use of ta moko as a form of signature or authorizing mark of identification is taken to highlight issues concerning the complex relationship between the attribution of certain cultural practices, characteristics or properties, to a certain group, and the notions of authorship and authority that underwrite such designations.     

Interdependence,interaction and metaphor in multisemiotic texts

Systemic frameworks for discourse analysis of visual/verbal and mathematics texts are used to examine the functionality of language? visual display and mathematical symbolism, and the meaning arising from interaction and interdependence between these semiotic codes in joint constructions. The nature of the interaction between options, classified as unmarked or marked, determines the meaning made in that instance and, more generally, provides an impetus for the expansion of the meaning potential of each semiotic. The phenomenon where semantic shifts occur when functional elements are reconstrued in another semiotic is called semiotic metaphor. Rather than the notion of an overlay of meaning typically associated with grammatical metaphor, the potential exists here for more dramatic forms of semantic shift. Examples of semiotic metaphor categorised as ‘parallel semiotic metaphor’ and ‘divergent semiotic metaphor’ are given in the text analysis.     

A framework for the multimodal analysis of online news galleries: What makes a “good” picture gallery?

The development of digital technology in recent years has led to a revolution in news production and dissemination. In terms of production, we have witnessed a fundamental shift towards visual story-telling. Images dominate the verbal story space and have the potential to become the story themselves. Beyond this, they are also creating unique spaces for themselves (e.g. the online news gallery), with new multimodal genres posing challenges for practitioners and analysts alike. The potential effects of such fundamental shifts on the professional news story-telling practices of the legacy news media provide a rich research opportunity for understanding both how and whether news organisations fulfil their mandate of making sense of the plethora of information that is now available. In this paper, we concern ourselves with one particular innovation in visual news reporting – the online news gallery, or picture gallery in journalism terms. We report on a qualitative analysis of 35 galleries from 12 English-language newspapers with online presence from Europe, North America, Australia and Asia, and interrogate the choices made by institutions in composing this particular type of multimodal “text”, as it is situated in a professional, news story-telling context. In doing so, we explore how a systemic-functional semiotic approach to multimodal news discourse may help us to access the meaning potential of this emerging genre as a vehicle for multimodal digital news reporting, present a framework for the multimodal analysis of online news galleries and consider its implications for the education of media practitioners.     

Re-Semiotizing the South African Democratic Project: The African Renaissance

This article reviews the relationship between the semiotic implications of the emerging concept of the African Renaissance, and the historical trajectory of semiotic studies in South Africa. Linked with the residual ideological territories occupied under the rubrics of ‘semiotics’ and/or ‘semiology’ are the ways these discourses have been re-articulated or re-presented within the politics of the post-apartheid era. As a concept of the ‘post’, therefore, the semiotic implications of a projective strategy like the African Renaissance require more than the mindless dyadism of linguistic semiology. The article sketches the outline of the case for a radical appropriation of the semiotics embedded in C.S. Peirce's pragmatism as a way of constituting the reality of ‘Renaissance’ concepts in terms of a Task. The place of semiotics within the broader philosophical and analytical programme is illustrated by means of a table that links semiotics, pragmatism, and radical theory. The table indicates how the issues discussed in this article can be seen to fall under a schema that foregrounds Peirce's realization that aesthetic and ethical concepts are prior to those based purely on logic. As such, therefore, the article suggests that topics in social or cultural semiotics are more readily amenable to semiotic interpretation and analysis under a pragmatic rather than a linguistic methodological regime.     

NGOization and de-NGOization of Public Action in Nepal: The Role of Organizational Culture in Civil Society Politicality

Abstract

This article explores the relation between organizational culture and the politicality of civil society organizations, or rather their social construction as political. It is based on a case study of a network (NW) of human rights NGOs in Nepal during the last few years of a Maoist insurgency and the period of autocratic rule by ex-king Gyanendra, and its immediate aftermath. Through detailed ethnographic material, this article highlights the central role of the NW's organizational culture in allowing it to act in ways that were recognized as political. Specifically, it shows how a process of ‘de-NGOization’ of everyday practices and values enabled NW to become a credible actor for political change during a crucial period of Nepal's history. This article contributes to the ethnography of civil society, urges that more attention be paid to the relation of civil society to the political domain, and suggests ways forward in researching this topic.     

Deradicalization as Soft Counter-insurgency: Distorted Interactions Between Somali Traditional Authorities and Intervening Organizations

ABSTRACT

This article focusses on how traditional authorities (TAs) are involved in deradicalization practices as part of the counterinsurgency campaign in Somalia. Through a lens of hybridity, it traces and discusses the adaptive and unpredictable forms of local-global governance that have emerged. TAs fulfil a mediating role between international expectations and local sensibilities. With important variations, they have taken on a deradicalization identity. However, many dilemmas arise from their interactions that will force adjustments and adaptations on all concerned actors.     

The observer actant in the contemporary legal discourse: a semiotic meditation*

ABSTRACT

The sensorial access to the law that a society grants to its members is underpinned by a logic of exhibition and disclosure that, in turn, results from a more abstract cultural propensity to either transparency or opacity. Through investigating the systems of signs by which the functioning of the law is either concealed to external audiences or manifested to them in a more or less spectacular way, one can gain a firmer grasp on the juridical aesthetics of a society. This semiotic approach entails two advantages: on the one hand, the possibility to compare and contrast different regimes of disclosure and secrecy in the synchronic dimension, connecting the discourse of the law with other – apparently distant – types of discourse, adopting the same aesthetics of transparency or opaqueness in other domains of social life and discursive production; on the other hand, the opportunity for a more insightful intelligence of the diachronic development of such aesthetics, so that it may be interpreted as the long-term byproduct of historical watersheds in the history of culture. The article, in particular, proposes to link the socio-pathology of anorexia with several other practices and texts of present-day culture that adopt the same rhetoric of transparency in other discursive arena, including that of law. In all these sign productions, indeed, the trend that predominates is one that, adhering to an ideology of immediacy and transparency, pretends to eliminate all filters, all hindrances, but also all material signifiers that would mar the purity of the content. Although this utopia cannot correspond to any actual semiotic state – for any content needs a material expression to be conveyed – it nevertheless exerts a powerful influence on the present time, until it manifests itself in extreme forms of ‘transparentist’ radicalism.     

Social innovation driven by citizen-led digital innovation: technologies of citizenship and the war on unemployment in Flanders

ABSTRACT

Citizens’ participation in technology design is increasingly deployed as a means to tackle social issues and a technique of empowerment. Techniques of empowerment belong to a set of strategies and methods employed by governmental and nonprofit organizations to constitute active, participatory citizens. This contribution exposes the political rationalities underpinning emerging initiatives that perpetuate citizens’ subjection by deploying participation as their essential governing technology. It analyses an initiative developed by a Belgian nonprofit organization to involve citizens in the design and development of digital tools to tackle unemployment. Findings unveil the potential downsides of participatory practices of technology design for social innovation. Critical reflections invite practitioners to pay greater attention to their design and implementation towards making them truly empowering processes.     

Rescuing constructivism from the constructivists: a critical reading of constructivist interventions in Southeast Asian security

Abstract

Constructivist contributions to the study of Southeast Asian security raise much-needed awareness of identity concerns and introduce conceptual and methodological innovations into the study of identity. However, their shared rationalist proclivity to couple subjectivity with sovereignty revives the enduring problem of treating agency as ultimately pre-given. Contrary to their professed aim to restore to security studies an appreciation for history and practice, the contributions of many Southeast Asia constructivists are quite tellingly essentialist, particularly their concessions to state-centrism and ideational/normative determinism, both due partly to an uncritical emulation of rationalist constructivist perspectives in International Relations (IR) theory. In granting ontological priority to states, Southeast Asia constructivists cannot fully transcend reification because their denaturalizing of international anarchy or regions comes at the expense of a reified state. In reifying either the state or ideas/norms, their claim to privilege practice in their analyses of Southeast Asian security becomes suspect. But if process and practice are to be taken seriously, then Southeast Asia constructivists must avoid presuming a preordained subjectivity that invalidates their claim to study social construction.