SISP2023
  click here to login into www.sisp.it

SISP Conference 2023

Sections and Panels

back to Sections and Panels index

Section 10 - Methodology of Research

Managers: Alessia Damonte, Federica Genovese

Read Section abstract
The SISP Section on Methods for Political Science is home to proposals delving into any aspect of empirical methodology.
It welcomes submissions about issues in, and applications of, concept analysis, measurement, computational techniques, modeling, research design, causal inference, and theory development and testing, regardless of their innovativeness.
Applications can discuss either single- or multi-method strategies in any substantive subfield of political science and cognate disciplines.
Proposals that improve the dialogue among frequentist, logical, Bayesian, or radically subjectivist approaches to empirical knowledge are encouraged.
The Section also welcomes discussions on replicability, Open Science and the FAIRification of political data, and teaching methods for political scientists.
Besides, proposals that address the Conference theme, and proposals with ties to other Sections are especially welcome.
The Section will consider proposals for individual papers and posters as well as complete panels, roundtables, and open workshops.
 

Panel 10.1 Text as data for the study of political institutions


How Text as Data methods can be used in the study of political institutions? Texts have always been a fundamental source of data for political scientists to do research on parties, national governments, and international organizations and other key institutional actors. Thanks to the spread of the Internet, we now have easy access to an unprecedented amount of textual material. In order to be able to extract meaningful information from such a large quantity of text documents in an efficient way, in recent years, political scientists have extensively employed methods of automated text analysis or as, they are also known, “text as data”. As this label suggests, these techniques entail the transformation of unstructured words into structured data to be analyzed through statistical tools and models. Text as Data methods are usually distinguished according to the key research purpose (scaling-classification) and the extent of input required to the researcher in the estimation process (supervised-unsupervised). However, the field is in constant evolution as more sophisticated tools emerge and existing ones are employed in new contexts.

The main aim of this panel is to discuss original and innovative applications of Text as Data methods for the study of institutional actors as well as new developments in such methods. In such a way, the panel provides a significant contribution to the diffusion of automated text methods in Italian political science. The panel welcomes contributions from both the fields of Comparative Politics and International Relations, with a substantive focus on institutional actors, the way they position themselves, and communicate such positions. Submissions employing any sort of method of automated text analysis, including scaling algorithms of text, topic models and supervised classification models, are considered. Contributions adopting state of the art tools such as semi-supervised topic models and word embeddings are particularly appreciated. The panel is open to different theoretical approaches underlying the hypotheses to be tested through Text as Data methods.

To sum up, the panel will deal with issues such as:
• Original applications of text as data in political science
• Comparisons across text as data methods
• Validation of measures extracted through text as data techniques
• Potentialities and challenges of text as data in the study of institutional actors

Chairs: Alice Iannantuoni, Valerio Vignoli

Discussants: Silvia Decadri

Exploring Cooperation on the Treaty on Non-Proliferation of Nuclear Weapons through Text as Data
Valerio Vignoli, Michal Onderco, Franz Johann Eder, Martin Senn
Abstract
Preventing the spread of nuclear weapons is essential to improve global security. The Treaty on Non-Proliferation of Nuclear Weapons (NPT), adopted in 1968 and entered into force in 1970, is still the cornerstone of international cooperation on this issue. The NPT is fundamentally based on three pillars: non-proliferation, nuclear disarmament and, peaceful uses of nuclear technology. Existing literature has extensively explored cleavages across states about the treaty and its implementation based for instance on possession of nuclear weapons and regional patterns. Most of these studies are based on a qualitative reading of a limited set of documents. Barnum and Lo (2020) provided a significant methodological contribution to this debate, analyzing country statements during all the NPT Review Conferences (RevCon) and Preparatory Committees (PrepCom) between 2000 and 2018 through the scaling algorithm Wordfish. However, their method of choice does not allow them to identify separately conflicts across each dimension of the treaty. Other than considerably extending and refining their data, we offer a more comprehensive and nuanced picture of cooperation over the NPT, leveraging on Latent Semantic Scaling (LSS), a semi-supervised and multidimensional scaling method. Not only this article aims to enrich the understanding of global regimes of nuclear proliferation, but also contributes to the research on Text as Data methods, comparing different techniques to measure conflict between political actors.
Negotiating civilian protection: Who shapes the language of United Nations Security Council resolutions? (1990-2019)
Andrea Knapp
Abstract
The role of member states in shaping the decisions of international organizations (IOs) has been the subject of extensive academic debate. Two perspectives dominate the literature on the interaction between these complex bureaucracies and their members. The conventional view, rooted in realist thought, posits that IOs "serve" the demands of their most powerful members. However, more recent scholarship has contested this instrumental vision of IOs and has stressed the agency of small states and middle powers in influencing decision-making. The United Nations (UN), where members with contrasting policy preferences must reach a consensus on critical issues of international peace and security, represents a relevant example of this puzzle. The outcomes of these negotiations are formalized in Security Council resolutions, which serve as the normative and operational basis for any UN mandate. Consequently, all agents should have a major incentive in shaping the language of resolutions according to their preferences. Because a substantial share of decision-making unfolds in the organization's corridors rather than in committees, quantitative research has seemingly accepted the notion of the complex and impenetrable "politics of resolution drafting". This has caused a major gap: Who determines the language of the resolution and thus shapes UN decision-making? What is the role of the P5, the penholders and the elective members? This article draws on theories of IO vitality and agenda-setting to dissect the politics of language negotiations during particularly contentious events: conflicts where the risk of mass atrocities warrants foreign intervention. Constructing an original dataset, this article matches 20,000 speeches by the seventy states serving the Council between 1990 and 2019, UN bureaucrats (i.e., Secretaries-General) and representatives of external organizations (i.e., ICC, AU) with 1,000 corresponding outcome resolutions. The Cosine Similarity algorithm is employed to measure the textual distance between the single speeches and the resolutions, which identifies the sources from which the language of outcome documents is replicated. This analysis contributes to the existing literature by (1) providing a comprehensive study of the influence of individual states on decision-making in humanitarian crises and (2) enabling the examination of cross-sectional and longitudinal trends in the power politics within the Security Council.
The politics of central banking: Using text as data to analyse technocratic responsiveness
Michele Scotto Di Vettimo, Christel Koop
Abstract
Independent central banks have come to invest increasingly in engagement with citizens. These interactions are meant to work in two directions, with the organisations informing as well as learning from citizens. Yet, whilst several studies have assessed the impact of central bank communication on citizens, we know little about whether any learning takes place at the side of the organisations. As they are deliberately insulated from political and public preferences, independent central banks are unlikely to follow citizen input in their policy choices. Yet, it is possible -- and in fact suggested by the organisations -- that some agenda responsiveness takes place; that is, that central banks' agendas take into account societal issue priorities. Our study assesses this possibility in the case of the Bank of England, one of the most engagement-active central banks. Relying on keyword-assisted topics models (keyATM) and vector autoregression (VAR) models, we analyse the influence of the topics covered in newspaper articles about the Bank -- our proxy for societal issue priorities -- on the topics of both the internal policy meetings and external communication of the organisation. Our results show that the attention the Bank dedicates to particular topics can be traced back to societal issue priorities in most of the domains related to the Bank's mandate, and that agenda responsiveness is stronger in the period after the financial crisis. Our results contribute to a better understanding of the politics of central banking -- and of non-majoritarian governance more generally.
The Politics of Peer Evaluation in IOs: Evidence from Text and Action in the OECD Donor Assistance Committee
Alice Iannantuoni, Simone Dietrich
Abstract
The OECD Development Assistance Committee (DAC) is the premier forum for international deliberations on best practices for official development assistance––i.e., foreign aid. One of the key tools at the organization’s disposal is a peer-review system, whereby each DAC member state gets its aid giving program evaluated by two other members every few years. Do these evaluations accurately reflect the practices of the reviewed donors? Do certain members receive harsher reviews than others? Do examiners exhibit favoritism when reviewing certain donors? We address these questions with a novel corpus including the universe of DAC peer review reports, 1962-2020. We show that, while the content of the reviews reflects the actual aid giving practices of the reviewed donor, there exists important heterogeneity: newer donors appear to be reviewed less harshly than traditional donors, and the practices of the examiners play a role in how they evaluate the same practices in the reviewed donor’s aid program. These findings contribute to the literature on the governance of international organizations, particularly with regard to the monitoring of compliance through peer reviews. They also further our understanding of the role of the international donor community in shaping development cooperation practice.
Ukraine in parliamentary debates
Simon Hug
Abstract
The invasion of Ukraine by Russia in early 2022 has caught many politicians and observers by surprise. A war in Europe, in a country bordering European Union member states, has led to a flurry of activities in parliaments, dealing with sanctions, military support, neutrality, etc. The debates surrounding these activities have also highlighted divisions and cleavages among political actors. Relying on speeches in the parliaments of the Canada, European Union, Poland, Switzerland, and the United States over the last 14 months allows us to show that important divides appeared in all four settings, highlighting the pertinent domestic dynamics.
 

Panel 10.2 Survey experiments in International Relations and Comparative Politics


Experimental designs are spreading across all political science fields, especially due to their ability to identify causal effects with accuracy from single or multiple randomized controlled trials. Relying on factorial approaches or conjoint experiments, amongst others, scholars have targeted different research questions, from how citizens react to norm violations during war to public attitudes toward foreign aid and women combatants, and from the relations between political ideology and postures toward immigration to how voters choose among candidates. In addition to substantial social and political science questions, many studies already have focused on specific methodological issues in experimental research, including response biases, temporal stability and structural reliability, as well as topic relevance and measurement errors.

Despite the growing academic interest in experiments, many substantial and methodological questions remain open. For instance: Do (and why) citizens react differently to atrocities during war in terms of their support for military aid – e.g. in Ukraine today – depending on their political preferences and personal traits? How does exposure to political phenomena such as immigration, crime, gender diversity, foreign aid, or ethnic conflict affect citizens’ political ideologies? Or again, how can we more effectively avoid and/or correct measurement error biases and social desirability biases with experimental designs? To advance this crucial research agenda, this panel invites contributions that theoretically, empirically, and methodologically explore different experimental settings in International Relations and Comparative Politics. In particular, we are interested in issues such as:

• Current methodological challenges in experimental research
• New theoretical and methodological approaches to factorial and conjoint designs
• Theoretical works on different aspects of experimental designs
• Substantial, empirical works that employ experiments to address questions in International Relations and Comparative Politics
• First-draft designs of experiments (not conducted yet)

Chairs: Matteo C. M. Casiraghi, Francesco Olmastroni

Discussants: Alice Iannantuoni

A Distorting Mirror: Ideological preferences and mis-perceptions of economic inequality
Elisa Volpi, Nathalie Giger
Abstract
Economic inequality is highly debated nowadays, yet little political action has been taken to tackle increasing economic disparities. Scholars have argued that one explanation might be people’s distorted perception of income inequality. The origins of this mis-perception are still unclear. Specifically, the role of ideology remains under-explored and the little research focuses almost exclusively on the United States. However, if distortions in perceptions of inequality have an ideological leaning, this could have severe consequences for how these views get translated in the political system. Taking advantage of a new survey on inequality perceptions, we are able to assess how citizens’ ideology affects their inequality evaluations across 13 Western countries. We test how ideology influences people’s ability to assess the overall level of inequality around them. In a second study, we verify the causal mechanism through an innovative survey experiment leveraging respondents’ ideological identification. The contribution of this work is double-fold: firstly, we shed new light on the phenomenon of inequality by looking at its psychological roots and secondly, we examine one of its potential explanation comparatively.
Project: Ex-Combatant Remobilisation and The Role of Mid-Level Commanders
Ana Vilhelmina Verdnik
Abstract
In the aftermath of civil wars, why do some combatants embrace peace while others choose to re-arm? Despite the growing interest of the international community to prevent ex-combatants from ‘going bad’ through disarmament, demobilisation, and reintegration programmes (DDR), there is a lack of clear understanding of why individuals re-mobilise. To address the gap, the project proposes a new analytical framework highlighting the importance of ex-combatant networks, that is the entities connecting former foot soldiers, mid-level commanders (MLCs), and leaders. By highlighting the hierarchies and interdependencies within ex-combatant networks, the project identifies mid-level commanders (MLC) as pivotal agents of ex-combatant remobilisation. MLCs’ importance stems from their unique role as organisers and coordinators within the networks. Due to their intermediary role, they possess extensive organizational knowledge, making them hard to substitute, turning them into de-facto ‘bottle-necks’ of remobilization. To demonstrate the power of MLCs and the conditions under which they are likely to remobilise, the project collects extensive survey data on the nature and extent of ex-combatant networks, conducts interviews with mid-level commanders to reveal the conditions under which they are willing to act autonomously and completes in-filed experiments to demonstrate disproportionate leverage of MLCs on beliefs of rank-and-file fighters. The work is fielded in Colombia, Liberia and Bosnia and Herzegovina (BiH). The three cases are chosen due to the variation in their ex-combatant networks. While the ex-combatant network is informal, patronage-based, and state-elite dependent in Liberia, it is largely institutionalized in BiH through veteran pensions and state employment. Colombia is a separate case entirely, with DDR initiatives stepping in to provide opportunities to MLCs and their fighters. The project is therefore able to ensure external validity by demonstrating the importance of MLCs in three different settings, decoupling it from characteristics of each ex-combatant network.
Protest Violence in the Eyes of International Audiences
Polly Chan, Samuel Liu
Abstract
When protesters in a resistance movement escalate or de-escalate violence, how do international actors change their support for the movement? Nonviolent resistance is known to receive broader domestic and international support, but protester use of tactics rarely stays static. Peaceful protesters can turn to violence when demands are unheard; violent protesters may fall back to nonviolent tactics in the face of repression. Would peaceful protesters who escalate violently suffer a smaller legitimacy cost because they have demonstrated their peaceful intent? Or is nonviolent resistance immediately denounced once protesters turn violent? To answer this question, we conduct a preregistered survey experiment on YouGov (n = 1,678) in the United Kingdom in June 2022. Consistent with previous findings, we find that support for an overseas movement is higher when it is peaceful. However, peaceful movements lose tactical legitimacy for escalating violence, while violent movements can redeem legitimacy for de-escalating. This finding has direct implications for policy support. UK citizens are less willing to sanction the repressive target government when protesters escalate, but they are more willing to give asylum to protesters when they de-escalate.
The Consequences of Trust in Diplomatic Negotiations: Disentangling Interstate and Interpersonal Trust through Experimental Vignettes
Nicola Chelotti
Abstract
Does trust matter in diplomatic negotiations? What does trust allow diplomats to do (or not do) in negotiations? The paper explores this question by using an original and unique dataset of surveys with UN diplomats, based in the national permanent missions to the UN, in New York. In particular, we look at two types of trust: a) interstate trust, i.e., trust that is shared intersubjectively between states; b) interpersonal trust, i.e., trust that is shared by individual diplomats. Does one type of trust matter and the other not? Or better, do they interact and how? Does interpersonal trust only play a role within the context of states that possess a trusting identity relationship, or do trusting interpersonal relationships between diplomats transcend state-level distrust in facilitating cooperation? In order to gauge the relative influence of interpersonal and interstate trust, we manipulate these two forms of trust in experimental vignettes. Our design is a 2 (trust in the country: low vs. high) by 2 (trust in the diplomat: low vs. high) between-subject design. We test the effects of interstate and interpersonal trust upon four types of negotiation processes and exchanges (influencing behaviour; logrolling; information sharing; flexibility).
 

Panel 10.3 Visuality in Comparative Politics and International Relations


Previously neglected, the study of visual artifacts has gained momentum across a variety of subfields, ranging from political communication to security studies. Scholarship on visuality displays meaningful differences, which largely overlap with broader epistemological divides between positivist and post-positivist social science. While most political scientists have sought to objectively operationalize visual items and largely employed large-N, quantitative studies, visual International Relations scholars have mainly leveraged critical theory and focused on the in-depth, interpretive examination of few selected artifacts. Despite these differences, visual Comparative Politics and International Relations alike provide tremendous insights.
In such a booming sub-field, however, many substantial and methodological questions remain open. How should visual items be studied? Do colors serve as reliable ideological cues for organizations like political parties, social movements, and rebel groups? Is the display of extremist symbols a reliable indicator of organizations' radical agendas, or do they merely serve as visual cheap talk? Why do certain organizations converge or differ in the use of specific flags and logos?
To advance this crucial research agenda, this panel invites a broad range of contribution on visual politics, its theoretical underpinnings and implications, as well as the methodological challenges attached thereto. For instance, the issues we are interested include, but are not limited to:
• New theoretical and methodological approaches to visual politics
• New empirical contributions that use visual items (photos, artwork, flags, maps, logos, memes, and others) as evidence
• First-draft designs of visual studies that have not yet been conducted

Chairs: Matteo C. M. Casiraghi

Discussants: Eugenio Cusumano

All Shades of Green: Chromatic Variation and Political Alignment in European Green Parties
Federica Genovese, Dafni Kalatzi Pantera, Roi Zur
Abstract
Political parties carefully strategize their visual brands, starting from the design of the most fundamental piece of their identity: the logos. Party logos possess multiple features, but one most critical cue is their color. Nascent research in political science studies logo's variation of color across countries and time. Empirical studies have shown consistent chromatic patterns in left-right party families. This paper contributes to this literature on political symbols by focusing on a less investigated but politically rising group: European green parties. Conventional wisdom suggests that these parties show maximum color consistency (isomorphism). Challenging this view, we show that European green parties vary significantly in terms of color patterns and, therefore, ideology association in recent years. Using a new dataset of green parties’ logos and state-of-the-art color analysis techniques, we also show the domestic and international correlates of green parties’ logo variation. This research has important implications for the associations and divergences across green and classical left-right parties. The results also suggest under which conditions green parties are more or less likely to express their identity following the homonymous color.
Crowds and smiles: Visual opportunity structures and the communication of European political leaders during the Covid-19 pandemic
Gaetano Scaduto, Moreno Mancosu
Abstract
Although the relevance of images has been a traditional topic in political communication studies (Rosenberg et al., 1986), the proliferation of social media platforms increased the academic interest in the topic over the last few decades (Bucy and Joo, 2021; Lilleker et al., 2019; Schill, 2012). We propose a theoretical framework, the Visual Opportunity Structure (VOS) theory to explain patterns of employment of specific visual elements in specific situations and environments in political communication. We borrow the concept from the discursive opportunity structure (DOS) theoretical argument (Koopmans and Statham, 1999; Koopmans and Olzak 2004; McCammon, 2013; De Bruycker and Rooduijn, 2021; Ernst et al, 2019; Salmela and von Scheve, 2018). We argue that, for politicians, employing specific visual features is determined by their appropriateness to the broader socio-political environment, similar to what DOS predicts to happen in discursive features. To test empirically our framework, a dataset containing all the Instagram posts that all the leaders of parties represented in the ninth European Parliament (156 leaders coming from 28 European countries) published in the period from March 1, 2019, to December 31, 2021 has been collected (n=71,652). The images have been subjected to an automatic analysis performed with the Microsoft Computer Vision API and Face API (Peng, 2021), two image and emotion recognition algorithms that have been employed to identify two features that are likely to change in the wake of the pandemic: the presence in the picture of smiling human faces and the presence of large groups of people. Through the use of multilevel logistic regression models, we show that the employment of visual features that might be inappropriate in moments in which the pandemic hits harder is negatively correlated with the pandemic waves: political leaders drastically reduce the presentation of the “inappropriate” visual attributes when the pandemic waves hit stronger and rapidly re-introduce them in the periods between two waves. This pattern is shown to be stronger when leaders of parties in government are concerned with respect to leaders of opposition parties. This paper aims at shedding light on visual political communication strategies employed by leaders and accounts for their most fine-grained features. In addition, our contribution also provides insights into the possibilities and limitations of the use of computer vision algorithms in analyzing visual political communication on social media. References Bucy, E.P. & Joo, J. (2021) Editors’ Introduction: Visual Politics, Grand Collaborative Programs, and the Opportunity to Think Big. The International Journal of Press/Politics, 26(1): 5-21. De Bruycker, I. & Rooduijn, M. (2021) The people’s champions? Populist communication as a contextually dependent political strategy. Journalism & Mass Communication Quarterly, 98(3): 896-922. Ernst, N., Esser, F., Blassnig, S., Engesser, S. (2019) Favorable opportunity structures for populist communication: Comparing different types of politicians and issues in social media, television and the press. The International Journal of Press/Politics, 24(2), 165-188. Koopmans, R. & Olzak, S. (2004) Discursive opportunities and the evolution of right-wing violence in Germany. American Journal of Sociology. 110(2): 198–230. Koopmans, R. & Statham, P. (1999) Ethnic and civic conceptions of nationhood and the differential success of the extreme right in Germany and Italy. In Giugni M (ed.) How social movements matter, Minneapolis. University of Minnesota Press: 225-251. Lilleker, D.G., Veneti, A., Jackson, D. (2019) Introduction: Visual political communication. In Veneti A, Jackson D, Lilleker DG (eds.) Visual political communication. London. Palgrave Macmillan: 1-13. McCammon, H. (2013) Discursive opportunity structure. In Snow, D. (ed.) The Wiley‐Blackwell Encyclopedia of Social and Political Movements. London. Wiley. Peng, Y. (2021) What makes politicians’ Instagram posts popular? Analyzing social media strategies of candidates and office holders with computer vision. The International Journal of Press/Politics, 26(1): 143-166. Rosenberg, S.W., Bohan, L., McCafferty, P. & Harris, K. (1986) The image and the vote: The effect of candidate presentation on voter preference. American Journal of Political Science: 108-127. Salmela, M. & von Scheve, C. (2018) Emotional dynamics of right-and left-wing political populism. Humanity & Society, 42(4): 434-454.
How to analyse a political image? The role of iconology, semiology and intermediality: towards a new methodology. The case of populist campaign posters
Morgane Belhadi
Abstract
In the last forty decades, the political realm has become increasingly celebriticised, mediatised, and stylised (Corner & Pels 2003, Dakhlia 2011, 2015) since political communication has entered the postmodern era (Norris 1997). This shift is particularly well evidenced via visual images in the way ideologies, political organisations, and leaders represent themselves. In line with this assertion, our paper intends to provide with an innovative multimodal methodology to analyse the part played by visual images in politics with a comparative perspective. Our methodology consists in three steps: to isolate motifs, iconographic and linguistic elements contained in an image; to identify clues showing that they are all connected to convey a message which bears either one or multiple meanings, willingly or unconsciously; to replace the elements in a larger, multifaceted context -- political, historical, social, economical. As a matter of fact, not only does this methodology takes into account the specificities of the context in a situation of communication; it also relies on two major intellectual legacies, namely iconology in art history, and semiology in linguistics, communication and media studies. These disciplines are not usually invested in political science and therefore we think that they can be of interest to explore the formation of ideologies, political beliefs and mythologies, and the success of emerging political movements for instance. To this end, our purpose is to apply tools and techniques employed in iconology and semiology to the study of political objects, in order to seize the evolution, redundance, and novelty of visual patterns at work. Indeed, the main objective is to examine the recurring motifs, shapes, fashions, occurrences, and innovations that arise and respond to each other in time and space from one medium to another (Panofsky 1939), what Jürgen Müller calls the process of intermediality (2006). Our approach aims to reveal the "symbolic forms" (Cassirer 1929) and mentalities of an era, where the material image, the medium, and the immaterial, symbolic, abstract, ideological image (Gervereau 2020) are combined and intermingled. As French and Italian semiologists Roland Barthes and Umberto Eco have noted, an image is always inscribed in a context and often relates to a textual component (Barthes 1964); it is also composed of a series of citations, blending old and new references, oscillating between standardised forms and new ones (Eco 1994), in order to strike and draw the viewer's attention. In other words, to understand thoroughly the complexity of an image, it requires to analyse it as a "mixed media", to quote art historian W. J. T. Mitchell (2018). In this respect, and in order to verify the efficiency of our methodology of visual, political objects, the case of the campaign poster is particularly relevant as it seeks to attract various types of electorates and to play an active role in the public sphere. Although an ancient medium, the printed poster displayed on the streets, as we have already stressed in previous works (Belhadi 2022), is constantly in use during a campaign as part of the parties' political communication ritual. But more importantly, it is highly and systematically exploited by populist parties wishing to gain legitimacy and visibility. Through recognisable posters, slogans, logos, as well as powerful, affective and dystopic narratives, and so on, populist parties create a visual mythology and an aesthetic of their own to reach their goals (idem): to effectively serve their main ideology, spread it and, eventually, extend their electorate. This is what a few comparative examples extracted from our corpus of French far-right National Front/Rally's (Front/Rassemblement national) and far-left France Unbowed's (la France insoumise) populist campaign posters will reveal. From 1972 to 2023, the National Rally, former National Front, created by Jean-Marie Le Pen on the basis of far-right parties and financially supported at its creation by the Movimento Sociale Italiano (as the logo of the flame on posters show), has gone through a social and political shift, alined with a change of name and of leader (Marine Le Pen). All the while preserving its conservative, exclusivist, anti-immigration positioning, the National Rally has attempted to renew its image. But what is now called by the media the de-demonisation effect had, in fact, already started in the 1990s when number 2 National Front leader by the time Bruno Mégret wanted to introduce more consensual themes to the party in the bid to soften its ethos and to attune its racist, xenophobic stances (Igounet 2017). On the other side of the political spectrum, France Unbowed was born in 2016, on the basis of the Left Front (2008-2015). The latter was formed by a coalition of several left-wing parties, among them the Communist Party. The revolutionary connotations, and narratives of an angry, internationalist, unsubmissive people, well-known in the communist ideology and iconography, are back in style in France Unbowed's rhetoric through its posters. However, France Unbowed, especially since 2022 parliamentary posters campaign, seeks to encompass this heritage by introducing new themes and political personnel, a kind of universal "phi" logo (that stands for "philosophy", "wisdom"), in order to gather all the sensibilities of the left-wing. Neglected for long, we believe that visual politics, along with new methodologies and transdiciplinarity (Bleiker 2018), is now significantly integrated in political studies, for they shed a new light on the understanding of political phenomena, in particular the populist one growing all over Europe.
 

Panel 10.4 Survey experiments in political and social scientific research (I)


This panel will explore the use of survey experiments: an increasingly prevalent methodological tool in political and social scientific research due to their capacity to isolate causal effects and more rigorously test theoretical arguments. Such experiments have been widely used in recent years to examine a diverse range of phenomena across various sub-fields including public opinion, values and attitudes, and political behaviour. However, their increasing usage has also seen their employment in research designs where they are arguably not ideally suited or even necessary. The panel will focus on identifying the strengths and limitations of survey experiments, in various potentially suitable research designs, and address a range of methodological issues. Besides classic issues of attaining (internal and external) validity, there are further questions to be discussed regarding potential representation and measurement errors in experimental studies. The panel will also seek to discuss the use of different types of treatments, sample characteristics, and the interpretation of results. We welcome contributions from scholars who have employed different forms of survey experiment (vignette, conjoint, list, and so on) in social scientific research. Overall, this panel aims to showcase the true value of survey experiments and promote a deeper understanding of their strengths and limitations.

Chairs: Silvia Keeling, Fred Paxton

Discussants: Riccardo Ladini

The sequential-argument design: A new way of capturing the interchange in political debate and its impact on public opinion
Robert Johns
Abstract
The standard design in persuasion experiments is to provide one argument and test respondents' reactions to it. Sometimes multiple arguments are tested -- but alongside rather than against each other. Occasionally (e.g. Gibson 1998, Sniderman and Jackman 2006, Petersen et al. 2010), respondents endorsing one argument in a standard survey question are presented then with a counter-argument. What is missing from the public opinion literature are survey experiments that reflect the to-and-fro of real world political exchanges, whether that be in parliamentary debates, TV studio confrontations or squabbles on social media. This failure of the typical design to reflect the dynamic interchange of real-world debate comes at a cost in terms of the scope as well as the external validity of survey experiments. A design capturing that interchange would enable the testing of more arguments and, crucially, public reactions to sequences of arguments and counterarguments. Which point clinches the debate? Can 'rational' or cognitive arguments counter emotional appeals? What matters: having the first word to frame the debate or the last word? In this paper, I make the case for a new *sequential-argument* experimental design which can address all of these questions and more. It is explicitly designed to replicate the multiple stages of argument and counter-argument in political debate. Crucially, by manipulating not only the content and order of arguments but also the point at which respondents are lifted from the debate and asked the outcome variable questions, we can measure the power of each individual argument and the particular combinations that 'win' debates. I will present two variants of the design, one based on social media exchanges and one on a TV studio debate. The substantive application in each case is to the debate between security and civil liberties that has been often used in the one-shot counter-arguments literature (e.g. Davis and Silver 2003; Petersen et al. 2010).
Inequality and political action: an unlikely combination? An experiment on how to make people act on inequality
Elisa Volpi, Nathalie Giger
Abstract
High economic inequality has many negative consequences but an electoral reaction in form of higher support for redistribution-friendly parties is not observed. In this paper, we take a different approach and study direct political action against it. We address two questions: (1) who is likely to become politically active against inequality? and (2) can the provision of information change this and increase the likelihood to become active. Based on a four-country survey and experiment, we show that personal salience and ideological leaning are important predictors to get active. Providing information on the topic helps and this effect is more pronounced for citizens scoring low on political sophistication. Our findings have important implications for how to make economic inequality a more relevant issue and the role information can play in this process.
Why Does Representation Matter? The Role of Descriptive Representation in Policy Justifications
Mirko Wegemann
Abstract
Voices demanding representation of marginalized social groups are getting louder. Despite high hopes, descriptive representation does not automatically translate into substantive representation. In the real-world, we can observe two sides of the same coin. On the one hand, representatives of marginalized groups like the first German trans-MP's, Tessa Ganserer and Nyke Slawik, take a leading role in justifying the extension of group-related rights. On the other hand, politicians like British Home Secretary Suella Braverman, daughter of two Indian migrants, have been applauded by anti-immigrant organizations for defending deportation flights to Rwanda. These empirical examples raise the question of how both group supporters and group adversaries react when marginalized social groups become active participants in democratic procedures. Until now, research on policy justifications has mainly investigated the content of justifications. In contrast, less emphasis has been put on speaker-related characteristics. However, these characteristics appear important since they can serve as source cues. Source cues are likely to elicit emotional responses by the audience which, in turn, can affect the evaluation of a policy justification. Although research has demonstrated that parties regularly appeal to social groups and that group cues affect citizens' attitudes, there is a lack of studies analysing the effect of a source's social group affiliation on citizens’ evaluations of policy justifications. To close this gap in research, this project zeroes in on the role of social group affiliation in policy justifications. It starts from the general assumption that citizens can infer from socio-demographic characteristics whether politicians are affected by a proposed policy. For instance, politicians with a migration background have been or will be affected by migration policy to a larger extent than politicians without that background. From existing research, we know that those politicians who are presumably affected by a policy are being perceived as more trustworthy. In general, this makes them more persuasive to citizens. For instance, women who discuss legislation on gender equality are being perceived as more credible and citizens are more likely to follow their stances on the policy. At the same time, this main effect is likely to be conditional on group liking. On the one hand, citizens tend to agree with the justifications of politicians if they like their social group. On the other hand, in some cases, even those citizens who dislike a social group may follow representatives of that group. If politicians turn against their alleged group interests, they are effectively legitimizing the views of group adversaries. Returning to the example outlined above, politicians like Suella Braverman who argue against the substantive interests of a group they represent descriptively may gain substantial support from citizens who dislike that group. To test my theoretical assumptions, I rely on a single-profile conjoint experiment that has been conducted in Germany. The experiment follows a within- and between-subject design randomly assigning respondents to a social group cue, policy position, and different policy justifications. The stimulus material comprises of twelve different fictional scenarios in which participants will be introduced to a policy decision that is justified by politicians with different socio-demographic characteristics. The article contributes to the scholarly field in two main ways. First, it improves our understanding of descriptive representation in the democratic process. More specifically, it scrutinizes whether politicians who are representatives of social groups affected by a policy can shape the acceptance of policy decisions. Second, it shows how the nomination of social representatives can elicit different responses by the audience conditional on group liking.
 

Panel 10.4 Survey experiments in political and social scientific research (II)


This panel will explore the use of survey experiments: an increasingly prevalent methodological tool in political and social scientific research due to their capacity to isolate causal effects and more rigorously test theoretical arguments. Such experiments have been widely used in recent years to examine a diverse range of phenomena across various sub-fields including public opinion, values and attitudes, and political behaviour. However, their increasing usage has also seen their employment in research designs where they are arguably not ideally suited or even necessary. The panel will focus on identifying the strengths and limitations of survey experiments, in various potentially suitable research designs, and address a range of methodological issues. Besides classic issues of attaining (internal and external) validity, there are further questions to be discussed regarding potential representation and measurement errors in experimental studies. The panel will also seek to discuss the use of different types of treatments, sample characteristics, and the interpretation of results. We welcome contributions from scholars who have employed different forms of survey experiment (vignette, conjoint, list, and so on) in social scientific research. Overall, this panel aims to showcase the true value of survey experiments and promote a deeper understanding of their strengths and limitations.

Chairs: Silvia Keeling, Fred Paxton

Discussants: Francesco Colombo

Activating social and political identities through political communication memes
Mauro Barisione, Silvia Keeling
Abstract
This paper investigates the effects of political communication memes on public opinion, with a focus on how social and political identities are activated among users of the main seven social media and messaging platforms (Facebook, Instagram, YouTube, TikTok, Twitter, WhatsApp, Telegram). A strong body of research focused on the relationships between social media and social and political identities (Settle 2018, Barberá 2020, Bail 2022, West & Iyengar 2022). While this includes the relationships with memes, research has yet to focus on the interplay between platform vernaculars, in the form of memes, and how they way they are framed interacts with different social and political identities. To do so, we have designed pairs of memes specifically for these experiments, focusing on their intertextuality, the blending of images and written text, and the salience of the issue. While memes within the same wave revolved around the same topic, experimentally we appealed to different political and social identities by framing the topic differently (e.g.: left and right political self-positioning). We considered a variety of topics including European and Italian politics, LGBTQI+, and sustainability. We used as an entry point into topics specific focusing events: highly mediatised events shortly preceding the wave and tangent to a relevant topic. We then analyzed the effects of these political communication memes on social media users by measuring changes in their degree of identification with a series of social and political groups depending on the timing (priming effect) and the way it was presented (framing effect). Our repeated cross-sectional survey experiments were conducted over six months in Italy in 2023, with over 10,000 interviewees. This approach complements traditional repeated cross-sectional survey experiments mainly in two ways: waves have been repeated multiple times per month to emphasize shorter-term effects at a diachronic level; most waves feature an experimental section, with two memes manipulated around a series of topics and focuses around salient issues. Methodologically, findings point at priming mechanisms being stronger than framing effects consistently across topics and waves. Contextually, we underline the struggles involved into designing frequent waves with time-sensitive objectives, such as the difficulty in including and operationalising issues around focusing events (creating social media content/memes for the experiment that could be somehow appealing to political and social identities or divisive) and in coordinating the logistics behind data collection (e.g.: scheduling data collection ¬with data providers; deciding timetables does not always go hand in hand with focusing events; decision to draw attention on focusing ‘dates’ such as bank holidays with a political relevance and conclusions to draw around this element). We conclude that exposure to both counter-attitudinal and congenial social media contents must be studied not only in relation to ideological polarization but also to a more selective and context-dependent activation of social and political identities. Our findings contribute to the understanding of the role that social media contents, and more particularly political communication memes, plays in reinforcing or defusing social and political identities, and suggest that this has important implications for the quality of the public spheres in the digital age. Keywords: Identity, Social Media, Political Communication, Memes, Public Opinion, Survey Experiments References Bail, C. (2022). Breaking the social media prism: How to make our platforms less polarizing. Princeton University Press. Barberá, P. (2020). Social Media, Echo Chambers, and Political Polarization. In N. Persily & J. Tucker (Eds.), Social Media and Democracy: The State of the Field, Prospects for Reform (SSRC Anxieties of Democracy, pp. 34-55). Cambridge: Cambridge University Press. Settle, J. E. (2018). Frenemies: How social media polarizes America. Cambridge University Press. West, E. A., & Iyengar, S. (2022). Partisanship as a social identity: Implications for polarization. Political Behavior, 44(2), 807-838
Rebel Tactics and Local Public Support. Experimental survey evidence from Southern Thailand.
Margherita Belgioioso
Abstract
How do rebels' tactics affect local public support for rebels and their leaders? The local legitimacy that rebel groups enjoy is difficult to observe directly in contested areas. Existing literature on rebel financing, rebels' governance, and the tactical choice of dissent assumes that rebels' local legitimacy depends on the use of violent and/or non-violent tactics. However, empirical work testing this mechanism directly remains limited. Systematic knowledge of how the local support of the population in contested areas varies depending on the tactics used by rebels can inform how to shape stable peace talks and effective negotiations. This is because existing empirical work shows that rebels with a wider base of support are associated with more stable peace agreements since they possess a more coherent organizational structure able to deter potential spoilers. We provide a hard test of how different violent and non-violent tactics used by rebels affect the support of the local population for rebel groups with two survey experiments in the conflict-affected provinces of Thailand. We map the response of local public opinion to specific tactics by providing survey respondents with randomized informational text varying the tactics used by rebels, and then compare how much support locals profess for rebels. We test the expectation that non-violent tactics increase the local legitimacy of rebel groups by examining four discrete nonviolent tactics, and the expectation that the use of violent tactics against soft targets actually decreases local public support, as several rationalist studies assume. Finally, we test the effect of the use of internal "democratic" processes, arguing that rebels' use of elections to designate leaders increases the local legitimacy of the group.
Food for (political) thought: investigating factors and behaviors associated with political inferences from apolitical cues in Italy
Gaetano Scaduto, Fedra Negri
Abstract
Research on political stereotypes in the US has shown that individuals rely on apolitical cues, such as cars or clothes, to infer the political preferences of others (Carlson and Settle, 2022; Lee, 2021). Based on these results, this work introduces the concept of politicultural linking (PL), defined as the process through which cultural signals are used to make inferences about other people’s political preferences. We explore whether this behavior is observable in a multi-party European context such as Italy through a survey vignette experiment. We test the association between ideology, news media exposure, cultural consumption, and affective polarization with politicultural linking. Moreover, we explore which cultural items are more likely to generate these inferences, and what are the consequences for personal interactions and political conversations. Associations between political and cultural preferences, whether originating from associative issue ownership (Walgrave et al., 2012), observed correlations (Rogers, 2022), or social projection (Ames, 2004), function as “classificatory schemes” (Bourdieu, 1996: 175) to infer the ideology or partisanship of other individuals and thus play a fundamental role in the detection phase of political discussion (Carlson and Settle, 2022). We expect that people having certain characteristics would be both more motivated and able to infer political preferences from apolitical cues. Data has been collected through a survey vignette experiment on a sample of the Italian population (N = 1092). Participants' sociodemographic characteristics, political preferences, personality traits, and cultural consumption habits are recorded. Participants are presented with a vignette of a person choosing a specific menu at the restaurant and are asked to express their opinion about the political preferences of this person, whether they would like to have coffee with this person, and whether they expect a political conversation with them to be pleasant. The cultural field in which PL will be observed is that of food. Food will be classified through two dimensions: Typical/Ethnic and Vegan/Carnivore. The scientific literature on the topic points to a consistent alignment between political ideology and preferences for ethnic (versus traditional) food (Buechner et al., 2022; Guidetti et al., 2022) or vegan (versus meaty) food (Dhont and Hodson, 2014; Rosenfeld et al., 2023). Through this project, we will observe if these food dimensions are consistently used to infer the political ideology of others. Our results lay a significant association between preferences for vegan and ethnic food and political ideology, while preferences for a meat-based diet are associated with right-wing ideology. Factors associated with people’s tendency to do PL are investigated through a series of logistic regression models. We found a positive association between PL and ideological self-placement, high levels of news media exposure, and high levels of cultural consumption, while we found no association between feeling thermometer-based affective polarization measure and politicultural linking. Finally, we observe how politicultural linking is associated with the likelihood to interact with others and the expectations regarding the pleasantry of a political conversation through a series of logistic regression models. Our results confirm that inferring political preferences from apolitical cues is associated with a change in these attitudes. Our study contributes to research on political stereotypes and cross-group contacts, showing how inferring political preferences from apolitical cues reduces cross-partisan contacts and conversation, thus contributing exacerbate political polarization. References Ames, D. R. (2004). Strategies for social inference: a similarity contingency model of projection and stereotyping in attribute prevalence estimates. Journal of personality and social psychology, 87(5), 573. Bourdieu, P. (1996). Distinction a social critique of the judgement of taste. Routledge. Buechner, B. M., Clarkson, J. J., Otto, A. S., & Ainsworth, G. (2022). Political Ideology and Cultural Consumption: The Role of Flexibility in Shaping Liberal and Conservative Preferences for Global-Local Experiences. Journal of the Association for Consumer Research, 7(3), 266-275. Carlson, T. N., & Settle, J. E. (2022). What Goes Without Saying. Cambridge University Press. Dhont, K., & Hodson, G. (2014). Why do right-wing adherents engage in more animal exploitation and meat consumption?. Personality and Individual differences, 64, 12-17. Guidetti, M., Carraro, L., & Cavazza, N. (2022). Dining with liberals and conservatives: The social underpinnings of food neophobia. Plos one, 17(1), e0262676. Lee, A. H. Y. (2021). How the politicization of everyday activities affects the public sphere: The effects of partisan stereotypes on cross-cutting interactions. Political Communication, 38(5), 499-518. Rogers, N. (2022). Politicultural Sorting: Mapping Ideological Differences in American Leisure and Consumption. American Politics Research, 50(2), 227-241. Rosenfeld, D. L., Rothgerber, H., & Tomiyama, A. J. (2023). When meat-eaters expect vegan food to taste bad: Veganism as a symbolic threat. Group Processes & Intergroup Relations, 13684302231153788. Walgrave, S., Lefevere, J., & Tresch, A. (2012). The associative dimension of issue ownership. Public opinion quarterly, 76(4), 771-782.
 

Panel 10.5 Measurement and Indicators in Political Science


In recent decades, the discipline of political science has undergone a steep quantitative turn. This transformation has been facilitated by methodological advances in statistical analysis and availability of data. While commonly used quantitative approaches are growing in complexity, the importance of high-quality data and indicators remain often overlooked. Yet, using and producing high-quality data is fundamental for truly pushing political science forward in this quantitative era. Today, the well-known saying ‘garbage in, garbage out’ holds more weight than ever before.

Currently, there is a lack of extensive research on political science measurement. To address this research gap, this panel seeks to attract papers that focus on the measurement of political science phenomena from various perspectives. We encourage in particular the submission of papers that propose and develop new indicators to be used in any political science subfield; evaluate the quality of existing indicators and datasets; reflect on data evaluation methods and measurement validity; and more broadly, contribute to the advancement of measurement in political science.

Ultimately, the objective of this panel is to foster more rigorous quantitative political science research by bringing together experts working on political science measurement and by providing an opportunity to discuss currently relevant issues on the topic.

Chairs: Andrea Vaccaro

Discussants: Guido Panzano

Ideological polarization in Western European party systems: a new dataset
Vincenzo Emanuele, Bruno Marino
Abstract
Unlike other key party system properties such as fragmentation or volatility, ideological polarization lacks a publicly available dataset that can be set as an established and widely recognized source among the political science community. This is probably because, for a long time, the political party research on this topic has employed different measures to operationalize the same concept. Recently, the Dalton’s index of ideological polarization has become an almost standard reference in the field. The index considers the position of each party on the left-right scale and their relative size. It ranges from 0 to 10, where 0 means that parties in the system hold exactly the same left-right position and 10 means that they are all located at the two extremes of the scale. Yet, notwithstanding the establishment in the literature of this widely shared measure, current studies employing ideological polarization usually rely on ad-hoc gathering and calculations based on multiple sources for the left-right placement of parties. This paper aims to fill this gap by introducing a novel publicly available dataset of ideological polarization in Western European party systems. The dataset provides measures of ideological polarization for each parliamentary election and legislature in 20 Western European countries since 1945. It does so by calculating four different measures of ideological polarization: two based on multiple expert surveys (related to votes and seats, respectively) and two based on comparative manifesto data (again, related to votes and seats, respectively). The paper illustrates the process of data collection, the data sources, and the methodological choices. Then, it presents the results in terms of historical and comparative trends of ideological polarization in Western Europe and discusses the similarities and inconsistencies between our results and those of other studies.
Measuring Spoiler Effects in Parliamentary Elections
Daria Boratyn, Dariusz Stolicki, Wojciech Słomczyński, Stanisław Szufa
Abstract
An electoral spoiler is usually defined as a losing candidate whose removal would affect the outcome by changing the winner. So far spoiler effects have been analyzed primarily for single-winner electoral systems. We consider this subject in the context of parliamentary elections, where the election outcome of interest is the allocation of seats among competing parties. In such elections, there is no longer a sharp distinction between winners and losers. Hence, we propose a more general definition, under which a party is a spoiler if their elimination causes any other party’s share in the outcome to decrease. We show that spoilers are ubiquitous under some of the most common electoral rules, and that proportionality is the only seat allocation rule which is spoiler-proof in all cases. However, spoilers in parliamentary elections vary in terms of their impact on overall results. We introduce first measure of such spoiler impact and demonstrate its application to a number of recent elections.
Question-order effect in the study of satisfaction with democracy. Lessons from three split-ballot experiments
Zsófia Papp, Pál Susánszky, Andrea Szabó
Abstract
This study looks at question-order effects in measuring Satisfaction with Democracy (SWD). Particularly, we are interested in whether the relative position of Satisfaction with the State of the Economy (SWE) question affects responses to SWD. Our research question is informed by the European Social Survey’s core questionnaire, in which the SWD question is the last one in a battery of satisfaction measures. We conducted three independent split-ballot experiments in Hungary between March 2021 and May 2022. We coded two versions of the questionnaire with different question orders: (1) SWD – SWE (not primed) and (2) SWE - SWD (primed). The questionnaire randomly directed respondents to either of them, applying simple randomization. We report a significant and substantial negative priming effect that possibly leads to a systematic underestimation of SWD. Importantly, we find no question-order effect in the measurement of SWE. The analysis further reveals a contrast effect: when the SWD question is primed, the difference between SWE and SWD means increases. Our final recommendation is that researchers put the SWD question before the SWE item to avoid question-order bias.
Reconsidering the Empirical Measurement of Social Trust
Zoltan Grunhut, Akos Bodor
Abstract
Social trust was originally conceptualized in the field of Social Psychology (see the works of Rosenberg and Rotter). However, much of the key thinkers, who contributed to the framing and reframing of trust theories, are political scientist. Just to mention a few important names, Fukuyama, Hardin, Hooghe, Levi, Newton, Putnam, Rothstein, Stolle, Uslaner or Welzel may come into our mind. Social trust is considered as a generative source to political participation, political engagement and political efficacy, as well as to civic actorness. Since the 1990s an impressive pool of literature has been devoted to the objective to better theorize and more adequately measure social trust. In connection with these efforts almost all of the relevant social surveys – including the World Values Survey, the European Social Survey, the European Values Study, the International Social Survey Programme, and the Eurobarometer – have been inserted one or more items for the empirical addression of social trust. So, there are varying concepts related to the phenomenon just as there are extensive datasets to statistically analyze its specificities (in different levels, longitudinally, comparatively, in association to other social-political dimensions, etc.). Despite of this rich background, there are still fundamental disagreements about the notion. Some claim that social trust cannot be taken seriously as a scientific concept (see the interpretations of Hardin and Noteboom). These critical voices emphasize that trust is not a social mood, but an intimate and intensive individual feeling, thus it is related to exact actors and situations. Others - among them, for instance, Fukuyama, Rothstein, Putnam and Uslaner - believe that social trust is more than just the accumulation of individual subjects’ trusting emotions; it is the bedrock of the structural, cultural and institutional architecture of a society. The current paper cannot aim to comprehensively review the theoretical background of social trust. It rather concentrates on its empirical measurement. Of course, there is enormous literature, including critical revisions, on the measurement of trust too. However, what is missing is the comparative examination of diverse measuring tools in different surveys from the same periods and countries. Accordingly, as a first step the paper presents a descriptive analysis of the most common tools of measuring social trust, among them the so-called standard trust variable, the trust index, and the radiuses of trust from the above mentioned surveys. As a main finding of this section, the paper points out that various tools measure the level of social trust with seemingly reliable consistency, yet if we compare the results of different tools then important in-society and between-societies discrepancies can be explored. This finding has motivated us to move on and propose a new approach for the better (more reliable) measurement of social trust. As it was already mentioned above, in the literature there are numerous criticisms of the empirical identification and interpretation of social trust. Studies that try to reflect on these critical remarks by different kinds of refinements usually address only particular issues regarding to survey tools & techniques, operationalization, and statistical methods, without more comprehensive revisions. This paper, on the contrary, strives to support its reflections on how to reconsider trust measurement by theoretical contributions. In order to achieve that, the proposed argument invokes Piotr Sztompka’s concept about the culture of trust, and his distinction between trust and trust functions. So, the second part of the paper elaborates on how the interlinked consideration of trust and its micro- and macro-level manifestations (identified as trust functions by Sztompka) can help to better understand the phenomenon of trust towards others. Then a comparative statistical analysis describes a more complex empirical framing of trust based on open-source databases from the European Social Survey.
 

Round table

Panel 10.6 Roundtable: Teaching Research Methods for Political Science


Research methods enable and constrain learning. The methods selected for training students in political science define the purpose and specificity of its manifold fields. Yet, we know little about the best practices of teaching methods across the discipline. This MetRiSP roundtable invites a discussion with Luigi Curini (Unimi), Adrienne Héritier (EUI), Pierangelo Isernia (Unisi), and Fedra Negri (Unimib) on the methodological training for political scientists in Italy and its capacity to yield adequate knowledge of old and new political phenomena.


I metodi della ricerca rendono possibili e vincolano particolari tipi di conoscenza riguardo entità e fenomeni di interesse. Di conseguenza, i metodi selezionati per formare studenti alle professioni della scienza politica definiscono obiettivi e specificità del suo multiforme campo in relazione ai settori disciplinari affini.
La Tavola Rotonda MetRiSP invita a una discussione con Luigi Curini (Unimi), Adrienne Héritier (EUI), Pierangelo Isernia (Unisi), and Fedra Negri (Unimib) sulla attuale specificità della formazione metodologica per la scienza politica in Italia e la sua capacità di generare conoscenza adeguata di vecchi e nuovi fenomeni politici.

Elenco interventi:
Fedea Negri - Università degli Studi di Milano-Bicocca
Luigi Curini - Università degli Studi di Milano Statale
Pierangelo Isernia - Università di Siena
Adrienne Heritier - European University Institute

Chairs: Federica Genovese

 

Panel 10.7 Text-as-data in political science: new research and advancements


In recent years, the number of works in the field of policy research that make use of text-as-data and quantitative methods has grown significantly. On the one hand, more and more scholars have identified textual sources as an untapped wealth of information. On the other hand, technological advances have made the processing and analysis of this type of data more accessible. From voter manifestos to speeches, from social media data to press releases, quantitative and automated textual analyses refer to disparate text types. The models and tools used for exploring texts are equally diverse, from topic extraction to the study of sentiments. Although cross-fertilisation with the computational sciences has opened new and exciting scenarios, the core tasks of quantitative textual analysis in the social sciences meet different research needs and cover discovering new phenomena or trends, their measurement, prediction and inference. Innovations in textual analysis and computational improvements have prompted more and more political scientists and those from adjacent disciplines to use text as data to answer more or less familiar questions. These advancements and innovations create unprecedented conditions and challenges for research in this field. How are text-as-data contributing to social science tasks? This panel welcomes studies that use quantitative textual analysis to answer new or established research questions and show how automated tools are employed, including their benefits and limitations, to study a wide range of topics. Papers can also deal with how new data sources can contribute to theoretical developments or help to bridge theoretical divisions through evidence in the field.

Chairs: Matilde Ceron, Jessica Di Cocco

Discussants: Jessica Di Cocco

Lost in Translation? Exploring the Potentials and Pitfalls of Text-as-Data Methods for the Cross-national Comparison of Local Spatial Development Policies
Theresia Morandell
Abstract
Text-as-data methods enjoy increasing popularity in comparative politics research not only due to their convenience in analyzing large quantities of textual data but also due to their potential in facilitating cross-national comparative research in multi-lingual contexts. The paper leverages in particular on this latter aspect of text-as-data methods by presenting a comparative analysis of local spatial development policies across 10 European-OECD countries spanning various language contexts. The research question guiding the analysis is the following: To what extent do medium-sized European cities address the broad set of physical and functional linkages (urban-rural relations) which typically exist between urban centers and their suburban and rural neighboring municipalities? As cities continue to grow beyond their administrative boundaries to form highly connected city-regions with their surrounding municipalities, the management of urban-rural relations has emerged as a strategic policy objective in the policy sector of spatial planning. The paper relies on text-as-data methods, structural topic models in particular, to conduct a large-n comparative analysis exploring if the European academic and policy discourse on urban-rural relations has translated into the contents of planning policy at the local (city) level. The aim of the analysis is to explore i) to what extent and how urban-rural relations occur as a topic in local spatial development policies adopted by medium-sized European cities, and ii) how the outcomes under point i) vary according to the characteristics of the territorial and policymaking context in which the analyzed policies were adopted. The paper thereby relies on an original corpus of 224 policy documents adopted by a sample of 109 medium-sized cities across 10 European-OECD countries, spanning nine languages. Medium-sized cities (also referred to as intermediate cities, or i-cities) are a theoretically interesting type of settlement to study in this regard. In the relevant literature, they are discussed to be regional centers for the provision of administrative, economic, cultural, infrastructural and planning functions, servicing both urban and rural populations within their broader region. It is this assumption of strong linkages of the intermediate city to its surrounding region which renders it a particularly relevant category of settlements to focus on in this analysis. Intermediate cities have only recently begun to enter the academic and policy agenda and are still relatively under-researched as compared to their bigger metropolitan counterparts. The points of innovation of this research are two-fold. First, it breaks with a tradition of adopting predominantly small-n case study approaches to analyzing instances of urban-rural relations and city-regional coordination in spatial policymaking. Secondly, the paper contributes to exploring the promises and pitfalls of text-as-data methods for social science research in the European multi-lingual context where language barriers pose a marked obstacle to cross-national comparative research of textual data. This paper taps into a line of social science research which leverages on the rapid advances in machine-translation software to overcome language barriers, by translating textual data written in a variety of source languages into English as a common reference language on which the actual analysis is performed. However, machine-translation is not error free. There is the potential that systematic differences in vocabulary use between the source languages continue to persist in the machine-translated text corpus and bias the model estimation. We may end up modeling variations in word use to specific language contexts rather than detecting topics as semantically coherent concepts across linguistic borders. Researchers may counter this problem by relying on the statistical properties of structural topic models which allow to include information on a text document’s source language directly into the model estimation, thereby controlling for systematic differences in vocabulary use across different languages. The paper test the limits of such an approach by extending the number of source languages to be included for analysis, as well as by applying it to an analysis of policy documents in the highly technical policy sector of spatial planning. Spatial planning is characterized by a distinct and specialized vocabulary which may differ considerably across various national planning contexts. Is there an upper limit to linguistic variation which can be accounted for in topic modeling?
Style Matters: Unveiling Gendered Communication Patterns of Italian Members of Parliament
Giovanni Pagano, Luigi Curini, Silvia Decadri, Fedra Negri
Abstract
Research on gender and political rhetoric has long suggested that male and female legislators exhibit distinct communication styles. Previous findings - which mainly indicate that women tend to adopt more concrete orientations in policy discussions, and to display less antagonistic language than men - have rarely been substantiated using extensive longitudinal data. In this study, we make two significant contributions to this literature. Firstly, we assess whether observable gendered differences exist in parliamentary debates, specifically whether Italian female Members of Parliament (MPs) differ in their communication style when compared to their male colleagues, and under what conditions these differences occur. Secondly, we enhance existing approaches that measure language style and content by employing a semi-supervised text analysis method, which enables us to assess multiple dimensions of communication style on a unique dataset spanning over seven decades (1948-2023) of Italian parliamentary debates. Our analysis takes into consideration potential variations across political parties, disparities between MPs in leadership positions and those with a less prominent status, government/opposition dynamics, and the impact of politicians’ idiosyncratic characteristics. Our approach produces a more nuanced understanding of gendered patterns in parliamentary speech, thereby shedding light on the role of gender in shaping political discourse and representation.
To which extent do news media influence issue and policy responsiveness? Cross-national evidence from Europe.
Simon Luck
Abstract
To which extent do news media influence responsiveness? Different lines of research have provided evidence on the extent to which policy and parties are responsive towards citizen more broadly, and the conditions which moderate the opinion-policy link more specifically. However, no single study has systematically studied the effect of news media on responsiveness. Why is this? On the one hand, scholars have demonstrated a variety of media effects, making it necessary to identify the set of media effects most relevant for responsiveness. On the other hand, the empirical finding of substantial media effects on responsiveness is fundamentally compromised by endogeneity: media are influenced by policymakers and public opinion to the same extent that policymakers and public opinion are influenced by news media. Finally, conducting a cross-national study is challenging as it is highly data and computationally intensive. To test for substantive media effects on responsiveness and to overcome the issue of endogeneity, I focus on the sentiment as well as the speed of sentiment change in news reporting. Policy and political communication scholars both hold that tone in news coverage not only allows to detect changes in levels of political accountability but also forecasts policy changes. The literature on responsiveness and media effects is reconciled using the rational anticipation mechanism: politicians respond to voter preferences to avoid being voted out of office. Accordingly, the media is expected to have an impact on responsiveness as it provides politicians with cues on issues to which they should respond to. This is also consistent with evidence that politicians are bad at gauging public opinion and are therefore open to, if not dependent on, cues. It implies that re-election-minded politicians depend on information that reduces uncertainties and informs them on which issues it might be worth engaging with public preferences. Sentiment as well as the speed with which sentiment of media coverage changes is one such cue, as they can incentive politicians to be responsive. Lastly, using the speed of positive and negative sentiment changes also promises to mitigate problems caused by endogeneity, as there is less potential for a dialectical relationship between the speed of sentiment change in different media outlets and the degree to which policymaking activities of politicians relate to public preferences. While testing this hypothesis, this study also builds on the distinction between issue attention allocation - are politicians aware of public preferences or on whose preferences do they focus? - and positional policy congruence - how much do decisions reflect citizens' preferences? In recent years, evidence increasingly suggests that it is easier for policymakers to respond to citizens preferences while setting the agenda as compared to at the decision-making stage where it appears harder to put words into action. Therefore, the effect of sentiment and sentiment change will be tested both for issue attention allocation and position policy congruence to provide a comparative perspective. The research design combines different datasets. While parliamentary speech data will be used to measure issue attention allocation, policy data is adduced for positional congruence. Responsiveness measures are constructed using public opinion data in combination with parliamentary and policy data. Newspaper articles are used to operationalise news coverage, with the help of which various measures of sentiment change are created. A novel database of more than 500,000 news articles was compiled from various archives and publicly available sources, mainly obtained through web scraping. The research design implements the analysis for a range of different policy areas covered between 1978 and 2017 for several European countries. Sentiment change will be measured using a self-written R function which is able to detect the number of positive and negative changes in sentiment. This function performs sentiment analysis using both a dictionary-based approach and Latent Semantic Scaling. Applying different methods does not only allow to add further robustness to the results but also enables an evaluation as regards the performance of text-as-data methods to detect sentiment in news articles written in different languages. A series of regression models are estimated in which the degree of convergence between public opinion and issue attention as well as the degree of convergence between public opinion and policy are the dependent variables. Sentiment, sentiment change, and speed of sentiment change are the main independent variables of the analysis. A number of control variables will complete the model as to connect well with the existing literature and to evidence the substantiveness of media effects among other possible mediating factors. Using an extensive and original cross-national dataset, the analysis promises not only to circumvent several methodological issues, but also to provide the first substantial evidence of the impact of news media on responsiveness in Europe.
 

Panel 10.8 Experimental Approaches to Understanding Politics


Politics is complex and alternative causal mechanisms are frequently debated in Political Science. Internal validity aiming to understand causal paths remains a central goal for political scientists, yet a difficult one. As a result, identifying the underlying conditions and mechanisms that determine individual attitudes and behavior toward political actions and decisions is critical. Experimental approaches are becoming more popular as a fruitful approach for researchers to test causal hypotheses and draw inferences on political dynamics. This panel welcomes rigorous and innovative experimental research - in-lab, in-field, and survey experiments- on politics ranging from political participation to political conflict. The focus will be on presenting theoretical frameworks and the latest evidence on political processes and dynamics using experiments, as well as assessing the effectiveness of experimental designs as an empirical technique for identifying, exploring, and analyzing these problems.

Chairs: Polly Chan, Ana Vilhelmina Verdnik

Discussants: Polly Chan

The multi-dimensional character and appeal of anti-establishment politics: an experimental approach
Fred Paxton
Abstract
The appeal of anti-establishment parties has upended European politics. Particularly among voters with populist attitudes, support has dramatically risen for candidates who emerge outside of political institutions, mobilise new issues, and rhetorically challenge elites. Yet, while the widely-used label ‘anti-establishment’ clearly implies a number of different traits, the distinct appeal of each of them has not yet been individually assessed. A central issue is of a conceptual nature: the specific components covered by the ‘anti-establishment’ label still need to be unpacked and systematically clarified. To address this issue, we produce a conceptual framework of the multi-dimensional character of anti-establishment politics. To then assess the appeal of these dimensions to voters, we leverage a conjoint survey experiment that allows us to causally identify the effects of various anti-establishment candidate traits on vote choice. Moreover, we distinguish the effect between individuals of high and low populist attitudes, in combination with their post-materialist ideological orientation. We compare results across six European countries (Denmark, Germany, Hungary, Italy, Romania and the United Kingdom), which enables us to also consider the effect of populists being in government for voters’ preferences. This paper therefore aims to make both a theoretical contribution – by clarifying the multi-dimensional character of ‘anti-establishment’ politics, and an empirical contribution – by demonstrating the varying appeal of these dimensions across a wide range of individuals in different contexts.
Norms and Public Approval of the Use of Force: A Conjoint Experiment on Norm Robustness in Comparative Perspective
Matteo C. M. Casiraghi, Elizabeth M.f. Grasmeder
Abstract
A wealth of qualitative literature on the strength and durability of norms has investigated how violations of these standards influence public support for war. However, these theories have been rarely tested via experiments – an approach that would enable scholars to compare the impact of different violations across different types of military operations. Through a conjoint experiment on a representative sample of the US population, our findings highlight a counterintuitive story about norm robustness in relation to alternative, traditional logics used to explain public support for war. While the violation of strong norms (i.e. nuclear taboo) significantly reduces public approval for the use of force, other norms have different impacts. When combatants transgress a weak norm (i.e. anti-mercenary norm), the effect on public support vanishes, and when international regulations are disappearing (i.e. targeted killings) their violation increases public support. This is also a negative story, since leaders' respect of norms does not have any influence of patterns of public approval. Even strong norms, however, always matter in relation to an instrumental logic of cost. Particularly when endured costs increase, the impact of norm violation on public approval decreases. Lastly, support for leaders who respect or violate norms change dramatically depending on the audience: crucially, citizens with more knowledge about international norms are more sensitive – in terms of approval of the use of force – to violations of both strong and contested norms, a result that generates important implications for policy-makers and practitioners that work in international humanitarian law.
What is Unfair about Inequalities? Economic Inequality Dimensions and Fairness Perceptions
Francesco Colombo
Abstract
Previous research on attitudes towards inequalities and the demand for redistributive policies has produced conflicting results. Empirical evidence suggests that objective levels of inequality only influence attitudes when perceived as unfair. In this paper, I test how different types of information on economic inequality influence fairness perceptions. While previous research has viewed economic inequality as a unidimensional phenomenon, inequality is multidimensional in nature, as reflected in the wide range of existing measures. Building on existing inequality measures, I examine how information on different dimensions of inequality influences fairness perceptions. Using a well-powered, pre-registered conjoint experiment, I identify the main dimensions explored in previous literature, replicate treatments, and add new ones. I estimate how each inequality dimension influences fairness perceptions and explore how different social groups evaluate these dimensions, focusing on evaluations of inequality of resources vs. opportunity. Additionally, I examine whether a focus on equalizing opportunity makes inequality of resources more acceptable. I provide a set of information based on real-life distributions of different dimensions of inequality across US states to provide realistic inequality profiles. Overall, the findings contribute to a deeper understanding of how information on different inequality dimensions influences fairness perceptions and the demand for redistributive policies.