Methodology of the complex monitoring of daily TV news using a database
NGO "Detector Media" monitors the quality of daily TV news since 2003. The research covers the main (those broadcasted at the prime time) newscasts of central TV channels. In 2018, the list of TV channels was as follows: Inter, 1+1, Ukraine, STB, ICTV, 5th Channel, "UA:First" and "112 Ukraine". This list may be supplemented or amended.
The subject of monitoring is compliance with the standards of the news reporting, manipulation, "parquet" (official announcements without the news value) and materials with signs of order. The product of this research is a qualitative analytical review published in the section "Monitoring of daily news" on the MediaSapiens website.
The comprehensive monitoring expands the subject of research. In addition to determining violations of standards and signs of order and "parquet", it records:
- news topics (possibility to determine the proportion of the news on various topics in broadcasts);
- geography of the news (specific regions, Kyiv, the whole Ukraine, international news);
- compliance with the standards of the news reporting;
- general sentiment of materials (negative, neutral, balanced, positive);
- presence of expert opinions in materials (with indication of experts);
- in-studio guests (on those TV channels where there are guest studios in the news);
- references to government institutions - number and sentiment;
- references to political forces - number and sentiment;
- references to specific politicians (candidates for presidency) - number and sentiment.
The research methodology involves the analysis of texts (transcription) of materials of the main TV newscasts, broadcasted on the central TV channels. If necessary, experts check transcriptions against video materials. Full text of a material and link to a video (if it is publicly available) is also added to the database.
A unit of the analyzed content in the research is an information story or message. Stories and messages are analyzed with intros and outros of a host, if any. They assess the stories of socio-political, economic and social topics. The stories on other topics (sports, entertainment, culture, weather, etc.) are considered in case the experts noticed manipulation or signs of hidden advertising in them.
The main products of the research are analytical review (qualitative analysis), in which the author focuses on the most serious violations of standards, materials with signs of order and "parquet", and a database that is used for quantitative and qualitative analysis of the news product at various data samples.
Standards of the news reporting
Compliance with the following standards is assessed:
- quick delivery;
- balance of opinions;
- authenticity (references to sources);
- separating facts from opinions;
Principles for analyzing compliance with these standards in the TV content are described in detail in the Methodology for monitoring television and radio programs of NGO “Detector Media” (section 2.1). Information about (non-) compliance with the standards is also added to the database. In addition to compliance with the balance of opinions, a specific list of parties, to whom they gave the floor in a material, is added to the database.
"Parquet" and materials with signs of order (censorship)
Principles for determining signs of "parquet" and order (censorship) are specified in the Methodology for monitoring television and radio programs of NGO “Detector Media” (section 3). An author of the monitoring review describes the signs of "parquet" and order, giving the required context and references to sources, if necessary. The database includes the information on whether the story is classified as "parquet" or showing signs of order.
General sentiment of a material
An expert determines what the general sentiment of a material is: negative, balanced (both positive and negative), neutral, positive.
Positive sentiment is the submission of facts, statements or actions potentially contributing to the support of politicians or officials, other subjects involved in materials. Materials having significant and newsworthy information and no signs of order can be positive. At the same time, the sentiment of a criticism of the authorities’ or other opponents’ emotional speeches and statements that have signs of populism, that is, will promote the support of the speaker by a mass audience, can be determined as positive.
Negative sentiment is potentially capable of causing a critical attitude towards individuals or events, their rejection. Can be determined in the context of the submission of quotations or actions of a person, the presence of authorial evaluations in the voice-over, extremely negative comments on a politician, official, other person, organization, event, idea in the material itself from opponents, experts, etc.
Sentiment is considered to be balanced in case if there are both negative and positive evaluations in a material, position of both supporters and the opponents, strengths and weaknesses listed, without creating a manipulative advantage for one side or one vision of a phenomenon or event.
Sentiment that does not contain either negative or positive evaluations is neutral.
Information to be added to the database
Region. Whether the events referred to in a material occur in one of regions of Ukraine (oblast, Autonomous Republic of Crimea plus Sevastopol, and the city of Kyiv), or they concern the whole of Ukraine, or whether it is about events outside of Ukraine. Several options can be selected if stories are collected from different regions. If it is not possible to determine the connection with the region, the field remains empty.
Newsworthy information. A brief statement of the main point of the news, story, event, to which the material is devoted. Search in the database by keywords will allow you to determine which channels covered and which ignored a particular event. If TV channels covered the same event, the wording of the newsworthy information must be identical, except when materials have critical differences in the focus and interpretation of the event content.
Topics. Which thematic group the material belongs to. The basic list includes the groups of "politics of Ukraine", "international politics", "economy of Ukraine", "international economy", "religion", "crime", "emergency", "war on the Donbas", "Crimea", "social topics" and "culture". The list of thematic groups may be amended or supplemented.
An in-studio guest. Names of people invited to the studio for a live interview or speech.
References to the government institutions. Whether it is about the President, Prime Minister, Speaker of the Verkhovna Rada, Secretary of the National Security Council and Defense, Verkhovna Rada, Presidential Administration, Cabinet of Ministers, parliamentary factions / groups (without indication), non-parliamentary parties (without indication) in a material. This list may be amended or supplemented.
By choosing one of the items, the monitoring expert indicates the sentiment of the mention: negative, balanced (both positive and negative), neutral, positive. You can select more than one option from the list. No references are added to the database that do not have a meaning in a material.
References to specific politicians. Whether it is about one or more politicians, who are the most ranking according to the results of sociological surveys or often appear in the media, in a material. After choosing one of the items, the monitoring expert indicates the sentiment of the reference: negative, balanced (both positive and negative), neutral, positive. These include syncs of these politicians or quotes displayed on the screen. We propose to include Anatoliy Grytsenko, Andriy Parubiy, Andriy Sadovyy, Arseniy Yatsenyuk, Borys Kolesnykov, Vadim Novinsky, Vadim Rabinovich, Viktor Medvedchuk, Volodymyr Hroisman, Yevgeniy Murayev, Mykhailo Dobkin, Oleh Lyashko, Oleh Tyahnybok, Oleksandr Vilkul, Oleksandr Turchynov, Oleksandr Shevchenko, Petro Poroshenko, Serhii Kaplin, Yulia Tymoshenko and Yuriy Boyko to the basic list, which may be amended or supplemented. You can select more than one item. References having no meaning in a material and being superficial are not considered.
Syncs / references to party politicians. Whether a comment, quotation or reference to a politician that represents one of the listed political forces is present in a material. The list that may be amended or supplemented includes the political forces that appear in sociological polls as the most popular, as well as those that often appear on TV. After choosing one of the items, the monitoring expert indicates the sentiment of the reference: negative, balanced (both positive and negative), neutral, positive. We propose to include the Petro Poroshenko Bloc, the People's Front, the Batkivschyna Party, the Civic Position, Samooborona, Oleh Lyashko’s Radical Party, the Opposition Bloc, the Opposition Platform - For Life, Svoboda, Agrarian Party of Ukraine, People's Power Party, Democratic Alliance, Ukrop, Vidrodzhennia, People's Control, Our Land and Socialist Party of Ukraine to the basic list. This list may be amended or supplemented. References having no meaning in a material and being superficial are not considered.
Expert opinion. Whether the experts’ syncs or quotations are used in a material. You can choose from the options "is", "no" and "not required". The last option concerns materials where, according to the monitoring expert, there is no need to involve experts in explaining the nature of the news or assessing the position of the newsmakers.
Experts. The experts’ names who were involved in a material in the form of syncs, indirect speech or quotations.
Format of overview publications
An overview publication of materials that came out during the week is scheduled for the next week (roughly Friday or Saturday). It includes:
1. Analytical review of the main topics covered by TV channels, with presentation of the necessary context and, if any, a list of the silenced or not fully covered topics and events. The review is structured chronologically, but the structure can be changed for the thematic one if the author of the monitoring considers it appropriate. In addition to questionable materials, the monitoring author highlights the highest quality, professional or exclusive ones.
2. Calculation of questionable materials. It includes:
- materials with signs of political or commercial order;
- parquet materials: statements, comments, participation in solemn events, awards distribution and greetings with holidays from officials and People's Deputies where the speaker comes to the foreground, the content of an event is not covered sufficiently, there are no comments from other contributing persons, there is praise (self-praise) of achievements of the speaker or the body he represents;
- promotional materials for their own media groups projects: extended materials about other programs, series, films produced or broadcasted by the channel or media group to which it belongs. This does not include short announcements of socio-political programs, which continue the broadcast after the news.
3. The database analysis. After filling the database for a sufficiently long time, it will include an infographic displaying key news features within a week. The infographic format and the list of data that will be included in it will be determined later. The data will also be used for monthly, quarterly and annual final analytical publications, which will allow you to see a more panoramic picture of changes in the information policy of the television news services.