Intersemiotic Texture: analyzing cohesive devices between language and images

Recent research on multimodal discourse has explored the nature of semantic relations between different semiotic resources. Drawing on the interpretation of language as a social semiotic resource, this article proposes Intersemiotic Texture as the crucial property of coherent multimodal texts and presents a preliminary framework for cohesive devices between language and images. The framework is illustrated through examination of print media to demonstrate how the image–text relations are meta-functionally orchestrated across experiential, textual and logical meanings at the discourse stratum. A discourse-based model is suggested to analyze image–text logical relations complementary to existing grammar-based approaches. This research also develops a meta-language to describe Intersemiotic Cohesive Devices from two complementary perspectives: Intersemiotic Cohesion not only functions to integrate different modes together when multimodal discourse is conceptualized as a finished product, it also constitutes essential text-forming resources for semantic expansions across language and images during the ongoing contextualization of meanings in real time.     

Interformative meaning of signs: brand naming and globalization in China

This article looks at how global and local values are interformed in the linguistic signs of global brands transposed in China. The data under study here consist of a variety of American global brand transpositions collected from Chinese websites, newspapers, magazines, and television. A close examination of these Chinese transpositions reveals that American brands are transposed in three different methods: phonetic, semantic, and phonosemantic. Drawing on the social semiotic approach, this study attempts to explicate the specific ways in which the Confucian views of the interconnection between sign and reality are signified in the globally stylish signifiers of the US brands. It is argued that global brand transposition is a process of semiotic construction in which global and local cultures converge into a unique system of signification in China.     

The meta-discourse of contemporary brands and the indexing of consumption: a way to build bonds of meaning

There is an important volume of reflections on the theoretical and methodological proximity of semiotics and brand. I emphasize the texts of Lencastre and Corte-Real on brand myopia, Perez on brand expression and the proposition of a brand analysis model based on TGS de Peirce, and Mick with his studies of branding, marketing, and advertising, among others. However, the constitutive tension of the sign-brand, in its complexity as a media phenomenon determined by the sign object, pulsates in harmony with the emotional, associative, and cognitive relations it is able to generate in the interpreting minds. Brand as a complex sign, detached from the shackles of marketing management, grows toward the performing hybridism of digital technologies and arts, finding its interpreters – open-minded, active, and desirous of constant negotiations of meaning – in countless semioses. The purpose of the present article is to show that the sign strength of contemporary brands lies in their ability to index consumption based on advertising metadiscourse that reveals its audiences' social values and the objectual power by which it is determined. To this end, the study integrated knowledge of the semiotic analysis of the expressions of twenty international brands in the fashion, food, and technology industries, and the systematization and analysis of their value offers.     

Exploring the complementary effect of post-structuralism on sociocultural theory of mind and activity

This article explores the methodological value of post-structuralism for sociocultural theory. The Peircean notion of knowledge serves as a rationale for incorporating a post-structuralist perspective of discourse into Vygotsky-inspired sociocultural approach to conceptual development. Central to this incorporation is an analytic frame of reference devised to examine how knowledge construction is mediated through the use of language and other semiotic tools. The article provides an interpretive analysis of mother–child interaction in the reading of dual-language picture books, interwoven with a post-structuralist exposition of knowledge, power and discourse. It is suggestive of what sociocultural theory would need to consider in terms of power relations when theorising the role of semiotic mediation in transforming human mind and activity.     

To be or not to be Charlie Hebdo: ritual patterns of opinion formation in the social networks

The social sciences have mostly focused on the formation of social opinions from a semantic point of view: given a certain semantic field, interviews, statistics, and other analytical instruments are commonly deployed in order to map the distribution of views, their evolution, their conflicts and their agreements. Socio-semiotics, social semiotics, and the other semiotic branches that bear on social inquiry have contributed to the effort by providing semiotic grids of categorization. These grids too, however, have been mostly related to semantic contents circulating through societies and their cultures. The present article pursues a different hypothesis. After briefly recalling the events of 7–9 January 2015 in the Parisian area, the article seeks to survey and map the syntax of progressive differentiation of opinions circulating in the social networks about such events. Some patterns are identified and semiotically described: (1) cleavage; (2) comparative relativizing; (3) blurring sarcasm; (4) anonymity; (5) unfocused responsibility; (6) conspiracy thought. A new semiotic square is created to visually display these patterns, their positions, their relations, and their evolution.     

Social media as semiotic technology and social practice: the case of ResearchGate’s design and its potential to transform social practice

ABSTRACT

This article addresses the need for critical approaches to social media by bridging the focus on language and other semiotic resources that characterises discourse studies with the broader perspective on social media as social, cultural, economic and technological constructs that dominates media and cultural studies. Specifically, we propose a model for analysing how social media as semiotic technologies, that is, technologies designed to enable and constrain meaning-making, may transform social practices. By incorporating Van Leeuwen’s [2008. Discourse and Practice: New Tools for Critical Analysis. London: Oxford University Press] framework for the critical analysis of discourse and social practice, the model extends the social semiotic approach developed in recent critical multimodal studies of software such as PowerPoint to social media, which function primarily to provide platforms for and commodify social practices, rather than to offer rich arrays of semiotic resources for creating multimodal texts and artefacts. Using the academic social network site ResearchGate and the practice of research peer review, we illustrate the model’s capacity to account for the ways the design of social media platforms – through the semiotic resources they make available and the ways these are presented – enables and constrains their users’ ability to perform key social practices and has the potential to transform these practices.     

From book to stage to screen: semiotic transformations of Gothic horror genre conventions

ABSTRACT

This paper adopts a multimodal social semiotic approach for exploring the semiotic changes involved in the transformation of a novel into stage and screen productions. It examines how semiotic resources are deployed in each medium through elements of mise-en-scène, such as speech, music, sound, lighting, props, staging, and cinematographic techniques, and the viewing perspectives that are thus established for audiences. The genre of Gothic horror is selected for this purpose, given how this form of performance has transfixed audiences for centuries and has been adapted for both the stage and the screen. In order to demonstrate how each performance medium has produced its own unique set of foregrounding devices to enthral and captivate audiences, a comparative analysis of excerpts from the novel The Woman in Black by Susan Hill, a videotaped theatrical performance, and the 1989 British television film of the same name is undertaken. The paper discusses the implications of the multimodal semiotic approach for developing a better understanding of the semiotic transformations that horror genre conventions undergo in different media and the viewership positions that are thus re-drawn for audiences. The paper concludes with a view of multimodal recontextualisation processes which form the underlying basis of human sociocultural life.     

Re-Semiotizing the South African Democratic Project: The African Renaissance

This article reviews the relationship between the semiotic implications of the emerging concept of the African Renaissance, and the historical trajectory of semiotic studies in South Africa. Linked with the residual ideological territories occupied under the rubrics of ‘semiotics’ and/or ‘semiology’ are the ways these discourses have been re-articulated or re-presented within the politics of the post-apartheid era. As a concept of the ‘post’, therefore, the semiotic implications of a projective strategy like the African Renaissance require more than the mindless dyadism of linguistic semiology. The article sketches the outline of the case for a radical appropriation of the semiotics embedded in C.S. Peirce's pragmatism as a way of constituting the reality of ‘Renaissance’ concepts in terms of a Task. The place of semiotics within the broader philosophical and analytical programme is illustrated by means of a table that links semiotics, pragmatism, and radical theory. The table indicates how the issues discussed in this article can be seen to fall under a schema that foregrounds Peirce's realization that aesthetic and ethical concepts are prior to those based purely on logic. As such, therefore, the article suggests that topics in social or cultural semiotics are more readily amenable to semiotic interpretation and analysis under a pragmatic rather than a linguistic methodological regime.     

Interdependence,interaction and metaphor in multisemiotic texts

Systemic frameworks for discourse analysis of visual/verbal and mathematics texts are used to examine the functionality of language? visual display and mathematical symbolism, and the meaning arising from interaction and interdependence between these semiotic codes in joint constructions. The nature of the interaction between options, classified as unmarked or marked, determines the meaning made in that instance and, more generally, provides an impetus for the expansion of the meaning potential of each semiotic. The phenomenon where semantic shifts occur when functional elements are reconstrued in another semiotic is called semiotic metaphor. Rather than the notion of an overlay of meaning typically associated with grammatical metaphor, the potential exists here for more dramatic forms of semantic shift. Examples of semiotic metaphor categorised as ‘parallel semiotic metaphor’ and ‘divergent semiotic metaphor’ are given in the text analysis.     

Brand response analysis: a Peircean semiotic approach

The purpose of this article is to organise and simplify the concept of brand response. It is a development of the authors' semiotic approach to the concept of brand, which uses a triadic model based on the Peircean levels of analysis of the sign. The original model distinguishes three levels: identity, object and response. This article briefly describes the first two levels and goes into more depth with regard to brand response. From the theoretical point of view, the authors' approach inverts the survey method used to ascertain positioning, image and value of the brand. Firstly, brand response is associations occurring in the minds of the reporting agent. Associations create awareness, and not the other way round, which is the concept underlying the current theory of brand equity. From the point of view of practice, it is the authors' belief that the main contribution made by the triadic semiotic model applied to brand response is that of the importance of “firstness”, i.e. the most immediate response possible. It is with a concern for ease of application that this article hopes to contribute by making brand analysis and assessment more intelligible. This article should be seen as an exploration into the possibility of simplifying the analysis of brand response. Its originality lies in the effort to return to the very reason for the existence of branding. The authors are convinced that by choosing differentiation as the guiding principle of their approach they have valorised what is essential, which is sometimes lost in more complex modelling.     

For Whose Benefit? Fear and Loathing in the Welfare State

This article contributes to the debate on the relationship between marketing and propaganda through an analysis of social marketing as a mode of governing in permanent campaigning. The working hypothesis is that social marketing operations are agitational rather than propagandistic. The conceptual approach stems from a comparison of propaganda and marketing with Fordist and post-Fordist modes of production and governance. The research into the role of agitation involves an empirical study of the UK government campaign against benefit fraud, the most expensive of its kind. Using a combination of methodologies, the political context is framed through a discourse analysis that charts the historical emergence of the problem of benefit fraud and the material effects of this discourse on welfare spending allocation, content analysis is used to identify correspondences between different newspapers’ rhetoric and policy under different governments, and semiotic analysis helps to decode the message of the campaign against benefit fraud, as it relates to the overall government's strategy on this issue. The study offers insights into the political strategy of the government of New Labour between 1997 and 2010 and its resort to agitational techniques, exposing the limitations of government marketing and public relations in the context of an overall crisis of its political legitimacy, in both economic and political terms.     

Mapping Cultural Auracy: The Sonic Politics of The Day the Earth Stood Still

Anna Kassabian writes in Hearing Film that “classical Hollywood film music is a semiotic code, and that it can and should be subjected to various semiotic and cultural studies methods, such as discourse analysis and ideology critique” (p. 36). This paper examines the sound of a particular Hollywood film—the B-Grade 1950 science fiction “classic” The Day the Earth Stood Still (1951)—in order to perform the kind of analysis Kassabian demands; but also to argue that the analysis needs to encompass not only music, but all sonic elements of the film. Furthermore, the paper argues for development of a cultural auracy that will complement studies of verbal and visual literacies in multimodal and multimedia texts.     

Backjumps: writing,watching, erasing train graffiti

This article deals with mobile semiotics. First and foremost, it discusses mobility as a semiotic device. The analysis engages with backjumps, a genre of train graffiti that draws inventively on various forms of movement. The term backjump refers to any fairly elaborate graffiti piece painted on trains in traffic, notably during the trains’ extended stops at terminal stations. The examples focus on the Stockholm metro, where a rigorous anti-graffiti policy has been firmly in place: graffiti is quickly cleaned off trains and a range of strategies is implemented to keep graffiti writing under wraps. By slyly inserting graffiti into the metro system, the mobility-driven backjump practice allows graffiti writers to temporarily subvert this semiotic regime. Furthermore, the forms of semiotic mobility at play are not limited to the movement of the trains. As the present study shows, mobile backjumps are entangled in other patterns of mobility, which jointly underwrite a number of interlinked semiotic processes.     

Reproduction of social classes at the level of production relations

This article starts with a discussion of the capitalist production relations which are defined as the relations binding two types of agents of production and the means of production. These relations are considered from the point of view of ownership, expropriation of value, and function performed. The ownership element is given the determinant role in the sense that the owner of the means of production is also the expropriator of surplus value (exploiter) and he who performs the function of capital (non-labourer). Vice versa for the non-owner of the means of production who is also the exploited and the performer of the function of labour (labourer). There is in this case correspondence between the determinant and the determined elements. Thus the two fundamental classes under capitalism are defined in terms of correspondence among the three aspects of the production relations. But the concept of determinantion implies both correspondence and contradiction between the determinant and the determined elements. The middle classes are identified in terms of contradiction among the three aspects of the production relations. Thus, there are agents (‘the new middle class’) who do not own the means of production and who perform either only the global function of capital or both this function and the function of the collective worker. This analysis of the capitalist production relations provides the opportunity for introducing the concept of positions (a fractional unit of the capitalist production process) which has both a technical content (given by the technical division of labour) and a social content (just as the capitalist production process rests on certain production relations, so does each of its fractional units). Thus, an agent of production relations and thus can be placed, at this level of abstraction, in the class structure. Subsequently, the thesis is submitted that the reproduction of classes depends on the production of both positions and agents, where the former has the determinant role. This implies that there can be a discrepancy between positions and agents at the level of the economic, i.e. that there can be a discrepancy between the value of an agent's labour power and the value required by a position. The devaluation of labour power is an important example of such a discrepancy. A distinction is made between two types of devaluation of labour power: wage goods devaluation and devaluation through dequalification. It is the latter which explains the reduction of skilled to average labour (technical dequalification of positions) and thus also the disappearance of the global function of capital (social dequalification of positions). Thus, it is the latter form of devaluation of labour power which must be used to explain the proletarianization of the new middle class. A comparison is drawn between the two types of devaluation of labour power and the circumstances are discussed which can give the dominant role either to one type of devaluation or to the other. The rest of the article uses this conceptual framework to interpret the changes undergone by the Italian new middle class since its origins.     

Defensive semiotic strategies in government: a multimodal study of blame avoidance

In this article, I argue that when officeholders try to avoid blame, they often engage in defensive performances which can be analysed in terms of how they exploit various multimodal semiotic resources to ward off (potential) criticism: the setting, appearance, and manner chosen for a particular interaction with an audience; and both verbal and non-verbal ways of arguing, legitimising, framing, and representing social actors and actions. I apply these analytic categories to interpret the data gathered during fieldwork at a major training event of British government communicators. By combining insights from multimodal discourse analysis, discourse-historical studies of organisational behaviour, and recent research into blame avoidance in public administration, I demonstrate how certain semiotic strategies used by officeholders have an effect of backgrounding the ideas about any possible norm violations that government communicators may have been associated with in the eyes of critical audiences, such as lying, spin doctoring, and using tax money for propaganda campaigns that may not actually serve the interests of the public. I suggest that analysts of government communication should pay more attention to the defensive practices of government insiders, and study in great detail how these practices are incorporated into everyday behaviour through professional training.     

Semiotics of pride and profit: interrogating commodification in indigenous handicraft production

This study investigates the shifting terrain of pride, profit and power relations in minority language communities under contemporary globalisation. While “pride” associates linguistic-cultural heritage with identity and preservation, “profit” views these as sources of economic gain. In contemporary late capitalism, “pride” seems to be increasingly giving way to “profit”. Arguing that this transformation needs to be interrogated in terms of complexity and that a detailed, multilayered semiotic analysis can open a privileged window for such an inquiry, this study combines critical multimodal discourse analysis and an ethnographic approach to analyse processes of semiotic commodification in handicraft production in the indigenous minority language community of the Sámi in northern Lapland. The investigation focuses on the activities of an innovative Sámi artist and entrepreneur, and within these a range of paper notebooks, which are, although designed by the Sámi artist and sold in her handicraft shop in Lapland, produced by women living at the border of Thailand and Laos. The investigation illuminates two critical shifts: how the move towards profit can open up space to contest the ownership of pride within an ethnic community. Second, how this move makes way for new, globalised modes of production of ‘indigenous handicrafts’ and creates global vectors of power, engaged in both empowerment and exploitation, in the production of both pride and profit. The study thereby contributes to the understanding of the increasingly complex power relations and the ambivalence and multiple effects of practices constituting the apparent shift from “pride” to “profit”.     

“What does development stand for?” A socio-semiotic conceptualisation

This paper argues for conceptualising development in terms of social semiotics. It points out that development studies is moving towards to human-centred rather than program-centred, technique-centred or engineer-centred thinking, that it has moved from a focus on technical transfer to dialogic interaction and that it has moved towards a postdevelopment imagination. On the basis of these moves, which all assume that development entails meaning-making, the article explores the contributions that biosemiotics and sociosemiotics can make towards conceptualising development. The article argues that semiotics provides development scholars with the conceptual tools to include both the material and the ideal in development thinking. It proposes a semiotic response theory of development in line with aspects of complexity thinking and semiotic realism.