Metrics on internet and society: research sources

Nothing like getting to know some of the reality meters of the internet and society to do evidence-based research!

Metrics and research

A moment of much tension in a research is that early stage, when we know so little about the subject that it seems as if we are talking about an unexplored field (and we want to escape the Dunning-Kruger effect). It’s when generally those who have more wisdom or good academic advice at their disposal will look for research already done about it.

After all, starting a new research rarely makes us the only reference in a subject. Most of the time, it’s like getting into a conversation where several people have talked about a lot of that subject before us.

In addition to articles and concluding papers, which generally raise what is already discussed in the scientific format on the subject, an important source can be indicators related to the reality that we want to study. Not everyone is going to discuss the same specific research problem or from the same angle we’re looking at – but that’s where metrics become even more interesting, because they provide a foundation for the discussions we want to have. They can even open up new paths that, without that data, would not have been imaginable.

For example, if I want to talk about digital inclusion, I am interested in knowing how many people have access to the internet in the reality I am studying, and perhaps what are the most frequent obstacles, and what use is made, etc. Thus, indicators allow, to a certain extent, to leave the speculative field and start with an evidence-based picture of reality. And this can be fundamental for responsible research, committed to its methodological quality.

Before, something about data

I will soon present some initiatives for measuring internet issues that help those who want to study issues of the internet and society to formulate good questions and reflections to be explored. But, before encouraging this dive into the data, I wanted to draw attention to some precautions that this type of source requires.

Many situations are difficult to map and every meter has limitations. Thus, using data from an indicator as evidence or justification for a survey requires that it is appropriate to that context. You cannot talk about the whole world based on data from two or three countries, for example, or about all social networks based on data from one of them. Sometimes it is enough to point out the limitations of the evidence; in others, it is necessary to rethink whether there is even a way to study that problem.

It is very important to observe the methodology of elaboration and application of the metrics of an indicator, in order to assess how well it is adequate to provide a reliable picture for that research. If all the thinking done is backed up by poor quality data, the risks of it being wrong are great. Teh same if good quality data is extrapolated to a context different from that which it allows us to verify. It is necessary to point out this limitation and the need for further studies.

When looking at quantitative data, one must avoid assuming correlation or cause-and-effect relationships without doing robust studies on the subject. For example, if a graph of a country shows the increase in per capita income over time and another shows the increase in requests to remove content on social networks, this does not mean that these factors are necessarily associated, or even that one causes the other. It is necessary to analyze the relevance of what we are going to infer by looking at the indicators and avoid what is known as a spurious correlation.

Still, it is wise to consider that some data would be very important, but it has not yet been collected. In this case, we cannot fill this gap with our opinion, as it is not yet possible to make statements about some issues, and that is fine. It is normal, in research, to identify that we do not know something yet. When this is so, we can even suggest, as a result of research, some new indicator to be measured and observed. But, anyway, we are lucky to have several indicators that assess important issues of the internet and society and that can be excellent allies in supporting our research. Let’s get to them!

Metrics and where to find them

It is impossible to talk about indicators about the internet without remembering CETIC (acronym for the Regional Center for Studies for the Development of the Information Society). In Brazil, it works as a kind of observatory since its creation in 2005, on different aspects of information and communication technologies in the country. The indicators that CETIC brings are grouped in periodical publications that monitor the use of ICT from different perspectives, based on large-scale interviews with a sample of the population. Want to know how the internet is being used at home by people? At TIC Households you will find a detailed report. How do children and teenagers use the internet? TIC Kids online offers some perspectives. In addition, it covers 9 other themes and areas. In addition to annual reports, the CETIC website provides a data viewer, which allows you to specifically search for information of interest.

Still talking about the Brazilian context, ANATEL (acronym for the National Telecommunications Agency) provides several data panels that include indicators related to mobile or broadband internet service providers. In addition to data on consumer complaints, it provides quantitative access to companies’ services and the status of important infrastructure aspects for internet connection.

Another interesting indicator is the Network Readiness Index (NRI), which ranks more than 130 countries since 2006, when it was created by the World Economic Forum, and analyzes how ready they are to benefit from ICTs, based on 4 pillars: technology, governance, people, impact. It covers topics such as artificial intelligence, internet of things and the role of the digital economy in achieving sustainable development goals.

XpA Metric, developed by Article 19, is designed to monitor global freedom of expression through 37 indicators and assesses around 161 countries. It encompasses five spheres of assessment, which comprise transparency, protection, civic space, digital and media. From them, the Global Expression Report is produced.

The Digital Access Index (DAI) was created by the Global e-Sustainability Initiative (GeSI) in 2018 and seeks to establish the relationship between internet access and the achievement of the goals for sustainable development adopted by the UN in 2015. It is divided into three axes: connectivity, technologies, digital solutions, and allows visualizing data in connection with sustainable development goals (positive, negative, indefinite).

The ICT Development Index (IDI), created in 2008 by the International Telecommunications Union, combines 14 other indicators and seeks to measure the levels of: ICT development over time, its progress in developed and developing countries, the digital divide and the potential of ICT to support the growth and development of countries.

Freedom on the Net is an annual report that, since 2009, has evaluated various conditions of freedom in various facets of internet use. There are 21 questions that monitor, among other things, restrictions on the flow of national information across borders, and rank countries by their index of internet freedom. It provides detailed reports on each country with the respective sources of information. It is worth noting that IRIS, in 2017, contributed to the Brazilian report.

Ranking Digital Rights is an indicator of the accountability of corporations (internet platforms and telecommunications companies) and their compliance with international and human rights standards in relation to users. It is divided into three axes: governance, freedom of expression and privacy, with details of the questions that make up the assessment of each indicator within these categories and the respective answers for each company.

More sources needed

This list that I presented is not closed, but it gives an idea of the type of metric that already exists and the diversity of approaches to the theme of internet and society. Several of them are described in the first volume of the Digital Inclusion Glossary. That book is also an excellent support material for anyone who talks (or wants to talk) about the internet and society. My last advice is that you go beyond the paths indicated here and, if you find an interesting source, always share it with fellow researchers!

The views and opinions expressed in this blogpost are those of the author. 
Illustration by Freepik Stories.