Using Sensor Data for Safety and Quality of Life

By Marijn Biesiot, Tim Jacquemard & Rinie van Est | PDF-version | This article is the first in a series | photo: Harold Versteeg - Hollandse Hoogte 

Eyes and Ears Everywhere

Examples of digital sensors include digital cameras and GPS on a smartphone. Sensor data can help the police to do their job better, for example by means of “smart” detection and effective enforcement.[1] Sensor data also enables private individuals and businesses to carry out investigative work or take action to make their own living environment safer and more liveable.

The extensive use of sensors raises various questions. Who can actually keep an eye on whom? What about safety and privacy considerations? We describe how sensors are used to improve safety and quality of life. [2] We first discuss which sensors the police use, after which we outline four forms of monitoring using sensors: surveillance, sousveillance, horizontal surveillance, and self-surveillance. We show the diversity of sensors with which the police, municipalities, individuals, and businesses are trying to make society safer and more liveable. We then explain how sensor data is used in various core tasks of policing. On that basis, we identify five trends in the way sensors are used to improve safety and quality of life. We conclude with a brief summary of the insights obtained.

Sensors used by the police

At the end of the 1990s, sensor data consisted mainly of video images from surveillance cameras in the street.[3] Currently the police are experimenting with cameras offering facial recognition and “anti-sensor sensors” that detect devices intended to disrupt signals (i.e. ”jammers”). At the moment, these innovative technologies are being used only by specialist police teams, but there are three directions of development: in recent years, digital sensors have become more extensive, more mobile, and smarter.

More extensive

Traditional imaging cameras and microphones can “look” and “hear”.The functions of digital sensors have become more extensive, and they have in fact digitised all our senses: they can also “smell”, “taste”, and “feel”. In fact, digital sensors can often perceive more than people can, for example detect the presence of metal and read DNA for forensic investigation purposes.[4] The police are also studying the use of digital sensors that can track smartphones via Wi-Fi signals (“Wi-Fi-sniffing”).[5]

More mobile

The Dutch police installed cameras in police cars for the first time in 1975.[6] Since then, digital sensors have become smaller and smaller. There are now even nanosensors, such as the “e-noses” that are used in the port of Rotterdam to detect unhealthy air and dangerous gases.[7] As a result, digital sensors have become more mobile. They are present in smartphones, can be worn on the body (for example smartwatches and smartglasses), and can be controlled remotely. “Bodycams”, for example, are video cameras attached to a police officer’s uniform, allowing the control room to watch and listen with him/her live. In the Netherlands, the first police bodycams were used by the Maastricht force in 2008.[8] Hundreds are now in use, albeit still experimentally.[9] The police also use helicopters and drones equipped with various sensors, such as a heat-sensing camera to identify cannabis plantations and a video camera that can provide sharp-focus images of a large area.[10]

Smarter

An example of a “smart” camera is one that can recognise faces. It not only records video images but, for example, can potentially recognise the faces of football hooligans in those images.[11] Another example is Automatic Number Plate Recognition (ANPR). This camera is “smart” because it has software that can read car number plates and compare them with those in a database. The police are also currently testing smart video cameras that can detect motorists who are making phone calls or using an app while driving.[12]

Four types of sensor surveillance

Besides the police, private individuals, businesses, and municipalities are also using sensors to improve safety and quality of life. The police have about 500 to 1000 cameras and municipalities have over 3000 surveillance cameras in the street. By comparison, individuals and businesses own some 1.5 million security cameras.[13] If necessary, the police wish to make use of the data from those sensors, which is why they ask individuals and businesses to register their security cameras. The idea behind this Camera in the Picture project [Camera in Beeld] is that if the police have a list of all the cameras, then when an offence has been committed they can quickly retrieve all the data from those near the scene of the offence.[14] Some 200,000 cameras have already been signed up for the project.[15] The project is a form of public-private cooperation aimed at making society safer. It also shows that people are not only monitored by the authorities but also regularly operate cameras themselves.[16] From the perspective of the individual, there are four types of surveillance using sensors:

1. Surveillance

Surveillance means monitoring “top-down”. It involves authorities monitoring individuals and property. Camera surveillance in the street is a familiar example. In this project, we will consider surveillance by the police, municipalities, and businesses. Like the police, municipalities and businesses also use other types of digital sensors to improve safety in the neighbourhood. More and more municipalities are experimenting with sensors in the context of “smart city” projects,[17] for example for crowd control. In 2017, the City of Amsterdam began using Wi-Fi sensors and smart camera surveillance to determine how busy it was in the streets. The aim was to control Christmastime crowding.[18] The City of Eindhoven is working with the police, businesses, and knowledge institutions to reduce nuisance and incidents in the Stratumseind nightlife area. They are experimenting there with acoustic cameras that not only measure sound level but can also warn the officers on duty if aggression is detected.[19] “Smart city” projects involve municipalities collecting data about individuals within public space. Amsterdam and Eindhoven have formulated principles for the careful handling of this kind of sensor data.[20] Geonovum (a government foundation) has also proposed a set of rules for sensors in public space.[21]

Camera surveillance by businesses is not limited to office buildings and parking spaces. ProRail, which looks after the country’s rail infrastructure, has installed smart cameras along railways to detect copper thieves or people trespassing on the line.[22] All seven hundred of Dutch Railways Safety & Service staff will soon be wearing a bodycam, which they can turn on in the event of an unsafe situation.[23] Another form of sensor technology is to be found in the security scanners at Amsterdam Airport Schiphol, which use millimetre wave technology to check whether travellers are carrying prohibited articles on their person.[24] Some companies are monitoring their employees with digital sensors.[25] The BestDriver app developed by DHL Express, the City of Rotterdam, and CGI (an IT company), for example, utilises sensors and serious gaming to encourage their drivers to drive more sustainably and safely.[26]

2. Sousveillance

In the case of sousveillance, it is citizens who keep an eye on the authorities. The term refers to surveillance “from the bottom up”, for example when a member of the public uses their phone to film action by the police. There are a lot of “arrest films” on YouTube. These can play an important role in criminal cases, such as the death of Mitch Henriquez after he was arrested by the police, using force, in The Hague in 2015.[27] Bystanders filmed the arrest while Henriquez was on the ground, and his family released photos taken at the hospital. His death led to demonstrations in the Schilderswijk district of The Hague against police use of force.[28] Two officers were convicted of assault resulting in the death of Mitch Henriquez.[29]The various video images and photos were used in the court case to show exactly what had happened.

3. Horizontal surveillance

People who “spy on” one another, for example by peeking in on their neighbours with a drone,[30] are carrying out horizontal surveillance, also sometimes referred to as acting as “little Big Brothers”.[31] TV programmes showing “idiots on the road” present footage from digital cameras mounted on the dashboard of cars. In the Netherlands, there are already some 250,000 cars with a “dashcam” of this kind.[32] In addition to security cameras inside their house and at the front door, people are also installing sensors that can detect movement and attempted break-ins. The HomeWatch [ThuisWacht] system available from the Interpolis insurance company, for example, consists of a sensor package with a smart camera and sensors for a house’s doors and windows. When the occupant is not at home and the sensor detects a movement at the door, it automatically transmits an alert to their mobile phone. If the movement involves the postman or a spider’s web, they don’t of course need to take action, but if a potential intruder is revealed, the occupant can use an app to call on friends or neighbours or a professional security service to respond.[33] The Neighbourhood Watch arrangement [WhatsApp Buurt Preventie] allows residents to alert one another to suspicious situations in the neighbourhood, for example by sharing photos of suspicious persons.[34]

4. Self-surveillance

People can also use devices and applications with digital sensors that help them follow rules for safety and quality of life, for example the Fair insurance [Fairzekering] and the ANWB motoring organisation’s Safe Driving [Veilig Rijden] app. Sensors keep track of a person’s driving behaviour and those who drive safely are given a discount on their car insurance. The objective is to improve road safety.[35] That is a type of self-surveillance. This system of sensor technology does still leave the user freedom to decide what to do with the feedback. If you enable ASR’s Drive Safely app[36] or KPN’s Safe Lock app, [37] the system prevents WhatsApp and other messages from coming through while you are cycling or driving.

The sensor data value chain

Many of the examples discussed in this article actually go beyond just data collection by means of digital sensors; they involve digital technologies that process that data and also take action in the physical world. The “cybernetic loop” helps to understand how this works.

The cybernetic loop

The three stages in the cybernetic loop are collecting data, analysing it, and applying it (see Figure 2).[38] The cybernetic loop visualises the entire value chain of digital sensor data. To take one example: Automatic Number Plate Recognition (ANPR). An ANPR camera collects data: it continuously records video images of cars on a highway. The camera is connected to software that analyses the images collected. The software can identify number plates and compare them with a database in which they are linked to car owners. When the algorithm signals a hit, various applications are possible, for example by a police officer seeing the hit and taking action. The “smart” camera can also take action itself, for example by automatically issuing a penalty notice for speeding.

The three stages in the cybernetic loop broadly correspond to three core activities of the police: witnessing, investigating, and enforcing:

Collect data: witness

Digital sensors act as a kind of digital witness in situations in which quality of life or safety is under threat. The information provided by the sensors supports and enhances observation by the police. Individuals, businesses, and municipalities collect a great deal of sensor data that can help to increase public safety. The police are required to gain their consent in order to access that data. That is also the case if the police want to view the images from registered security cameras in the context of the Camera in the Picture project. In addition to sensor data, the police collect information from public sources (such as YouTube and Twitter) and closed sources (such as court data and banking systems). They also request participants in platforms such as Burgernet[39]and Amber Alert[40]to provide information about missing persons or suspicious individuals.

Analyse: investigate

The police can then analyse the sensor data that has been collected and perhaps combine it with data from other sources. Investigating involves searching the data for patterns and suspects. This analysis can be carried out by police analysts but also by means of artificial intelligence and “big data” analysis. In Roermond, the police, scientists, and the city authorities are carrying out an experiment to prevent gangs of roving criminals from pickpocketing, shoplifting, and burglary.[41] The designer outlet in Roermond attracts many gangs from Eastern Europe, especially Romania. The police are investigating how digital sensors (such as acoustic cameras, camera surveillance, ANPR cameras, and Wi‑Fi trackers) can help to identify and track such gangs sooner. The datasets are linked and analysed at a data centre at Eindhoven University of Technology. Manually developed algorithms are used to identify patterns in the data, for example whether a car has come from Germany, whether it has a Romanian number plate, and how many people it is carrying.

Apply: enforce

Based on the analysis, the police then proceed to intervene. Enforcement can also focus on preventing crime. The aim of the Roermond experiment is to detect suspicious behaviour at an early stage and thus prevent an offence taking place.[42] In order to do this, the police collect a great deal of data, including about a large number of innocent motorists in Roermond. Automated data analysis helps the police determine what is abnormal behaviour and what is in fact suspicious behaviour. The algorithm allocates points to cars based on a list of criteria for suspicious behaviour. The more points a car has that is on its way to the outlet, the greater the likelihood that the police will head it off. To safeguard privacy, data from non-suspect cars is deleted after the analysis has been carried out.[43] Practice has yet to show how often the algorithm produces a correct hit.

Five trends

The cybernetic loop makes clear how sensor data is used in various core activities of policing: witnessing, investigating, and enforcing. This way of thinking can also be applied to other parties that use sensor data to improve safety and quality of life, such as businesses, municipalities, and private individuals. Not only do they collect sensor data that may be of value to the police but they can also analyse the data themselves and take action based on it. We have identified five trends in the way sensors are used to improve safety and quality of life.

Trend 1: There are more and more police sensors and sensor data

The police are increasingly working with sensors. At the end of the 1990s, they were already using sensors, for example cameras to photograph crime scenes and surveillance cameras in the street. However the number and type of digital sensors available to the police has grown enormously. Officers on the beat are supported by digital sensor technology. This has its effect on the officers’ role. They take decisions partly on the basis of information from sensors. In that sense, police action is informed by data. It is conceivable that in the near future sensor data will play a more directing role. Can one then still say that the police officer takes the decision him/herself?

Trend 2: The police are automating some core activities by means of smart sensor technology

A smart sensor can also actually perform core tasks of the police. What we actually mean by “smart sensor” is an automated robot system. The definition of a robot is “a machine that can perceive, think, and act”..[44] 

In that case, the three stages in the cybernetic loop are automated. An ANPR camera can collect images continuously but it is still a police officer who decides what action is needed based on analysis of these images. However, fully automated intervention is also conceivable. For example, the Central Judicial Collection Agency (CJIB) could have ANPR cameras automatically issue fines when tracking motorists. The ANPR camera system would then collect the data, analyse it, and take action without human intervention.

Trend 3: Individuals, businesses, and municipalities are collecting more and more sensor data

Over the past twenty years, the use of digital sensors has increased enormously. Individuals and businesses own a thousand times more security cameras than the police. That is without counting all the smartphones and other sensors. What that means in practice is that there are a very large number of digital witnesses that can support the work of the police.

Trend 4: The police are seeking new forms of cooperation so as to use sensor data from the public to improve safety and quality of life

In this article, we provide various examples of how the police are cooperating with private individuals, businesses, and municipalities to improve safety and the quality of life by using sensor data. In Roermond, the police are working with the municipality, Eindhoven University of Technology, and the Public Prosecution Service. This is a form of cooperation within the public domain. The police are also working with private parties, i.e. individuals and businesses. They are looking for new ways to access sensor data from private parties, such as in the Camera in the Picture project. This is a form of public-private cooperation with individuals and businesses. Another example is when the police encourage individuals to do investigative work themselves.

The police are developing new digital platforms that help private individuals to do their own investigative work. The SamenZoeken [Search Together] app helps people search more “smartly” for a family member, friend, or neighbour who has gone missing. The police officer who devised the app explains: “It’s a fundamentally different way of looking at participation by the public. You don’t ask individuals to help the police investigate someone; we help private individuals with their search.”[45] If the police then take over a search, individuals can easily provide them with the information already collected. Another example is Automon, which will be launched in 2019.[46] This is a kind of Pokémon Go for stolen cars:[47] ANPR cameras in the street recognise the number plates of stolen cars and automatically send an alert to nearby Automon players, who then start looking for the car. The first person to find it receives a reward.[48] An app has even been announced that gives individuals tips so they can play detective themselves after a minor incident, such as vandalism or burglary (Sherlock[49]).

Trend 5: Private parties do their own investigative work and enforcement with sensor data

The use of sensor data by the police and municipalities involves sensor applications within the public domain. Private parties (individuals and businesses) are also using sensors and sensor data to make their own living environment safer and more liveable. Not only do they collect sensor data to share with the police (as witnesses), but they also analyse it themselves (i.e. investigating), and take action themselves (i.e. enforcement). Let’s take the examples of a car insurer and “do-it-yourself-policing”.

Someone who takes out the ANWB’s Safe Driving insurance policy is given a sensor system in their car that collects data every time they drive and keeps track of their driving style. Driving far too fast, for example, is classed as unsafe.[50] The safer someone drives, the bigger the discount they get on their car insurance. The terms of use state that the ANWB can terminate the insurance if the insured party doesn’t drive safely: “You can see whether you drive safely by the colour codes in the feedback that you receive from us. We work with a series of categories that are each linked to a colour. … If you receive six red notifications in the speed section within a year and/or one black notification for speed, we may cancel your insurance.”[51] The car insurer therefore takes action based on an analysis of sensor data and it terminates the insurance of someone who drives in an unsafe manner. There are also additional benefits: in recent years, the “dongles” - a USB stick containing a SIM card so as to receive mobile Internet on a PC or laptop - that are installed in insured persons’ cars have enabled the ANWB to trace a number of stolen vehicles, while speeding feedback has alerted parents to their children using the car without permission.[53] 

Individuals cooperating with the police is not something new. What is new, however, is that they use sensors not only to provide the police with information but also to investigate and enforce for themselves. This kind of “do-it-yourself-policing” is at their own initiative.[54]“Jaime the Villain-Spotter”, for example, has been tracking pickpockets and shoplifters in Amsterdam as a hobby for years now.[55] He searches the streets for people behaving suspiciously, follows them, calls the pickpocketing team of the Amsterdam police, and waits for the police to arrest them. He films all this and last year started uploading the films to the Internet. His “Villain-Spotter” [Boevenspotter] YouTube channel has more than fifty thousand followers. The police don’t mind Jaime sometimes chasing suspects and taking back the stolen property, as can be seen in one of the film clips. “He does so at his own responsibility. A private individual can basically make a citizen’s arrest of someone who is caught in the act.”[57]

”Taking the law into your own hands”

Questions arise in the case of digital participation by the public in which individuals not only provide information but also carry out their own investigative work and sometimes enforcement. Besides success stories, there are also warnings from experts regarding citizens who “take the law into their own hands”, obstruct the police, end up in risky situations because of their investigative work, are traumatised by finding a body, or who infringe the privacy of others.[58] “Jaime the Villain-Spotter” now makes sure that the pickpockets who he films are not recognisable on screen. If a suspect is recognisable, it may have consequences for criminal prosecution. A group of youths (“the head kickers”) who assaulted a man in Eindhoven in early 2013 were given a reduced sentence because of video footage. The court ruled that their privacy had been violated because the video images were shown on television.[59]

In conclusion

This exploration of the use of sensors reveals a complex network of “sensor surveillance”. Individuals are not only monitored by the police and other institutions (surveillance), but also use cameras themselves in various ways (sousveillance, horizontal surveillance, and self-surveillance). Collecting sensor data, analysing it, and subsequently taking action can be carried out by public and private parties and can also take place (partly) automatically.

When we look at the entire sensor data value chain, we see five trends in the use of sensors to improve safety and quality of life:

1. There are more and more police sensors and sensor data;
2. The police are automating some of their core activities (witnessing, investigating, and enforcing) by means of smart sensor technology;
3. Individuals, businesses, and municipalities are collecting more and more sensor data;
4. The police are seeking new forms of cooperation so as to use sensor data from the public to improve safety and quality of life;
5. Private parties are carrying out investigative work and enforcement with sensor data.

Sensor data is thus being used by public parties (the police and municipalities), private parties (individuals and businesses), and public-private partnerships. In the next article we will explore in greater detail what form these various social practices take.

These five trends raise various questions. Do individuals have different attitudes to a police officer with a bodycam and to a Dutch Railways security guard with one? Does it matter to someone whether they receive a “ticket” from a police officer in the street or from a smart ANPR camera? Would people – if so requested by the police – be prepared to add a function to their security camera that can recognise faces or recognise people by the way they walk? How do people feel about leaving pickpockets and burglars recognisable in film clips uploaded to YouTube? What do people think about businesses taking on enforcement tasks using sensors?

This project investigates what aspects people consider when thinking about these kinds of questions. When do they think using sensor data is a good thing (or not), and why? The fact that this can literally become a very personal matter is shown by the public debate about security scanners at Schiphol Airport. When they were introduced ten years ago, there was a fuss about “scanners that show you naked”. A body scan showed a person’s body and whether they had prohibited items hidden under their clothing. However, the face and private parts were blurred. Why did some people prefer to be scanned while others preferred to be frisked by a security guard in the old-fashioned way? Was it because of the way they had to “expose” themselves in the scanner or because of other emotions and arguments? More about that in the next article.

[2] The study “Public perspective on the use of sensor data for safety and quality of life” [Burgerperspectief op inzet sensordata voor leefbaarheid en veiligheid] focuses on safety and quality of life in a broad sense. “Quality of life” refers to minor offenses, such as littering. On the other end of the spectrum are more serious crimes that create a major sense of insecurity, such as mugging, threats, assault, and serious forms of crime such as drug trafficking and trafficking in human beings. This study therefore concentrates on the task of the police, which primarily involves enforcing the rule of law and providing assistance in the event of an emergency. When choosing to discuss a practical example, we therefore always ask ourselves whether a situation is involved in which one would call the police.

[6] Flight, S. (2016). “Politie en beeldtechnologie: gebruik, opbrengsten en uitdagingen”. Justitiele verkenningen 42, no. 3, pp. 68-94.

[8] Flight, S. (2016). “Politie en beeldtechnologie: gebruik, opbrengsten en uitdagingen”. Justitiele verkenningen 42, no. 3, pp. 68-94.

[9] De Leeuw, P. & I. Nap (2018). Programma Sensing. Concept programmaplan 2018-2019. Draft, version 2.0 19 February 2018.

[10] Wide Area Motion Imagery (WAMI) is explained on pp. 75-77 of Politie Academy (2018). Kennis voor de politie van morgen. Een conferentie over onderzoek bij, naar en voor de politie. The Hague: Sdu publishers.

[13] Leenaers, H. (ed.) (2016). De Bosatlas van de veiligheid. Groningen: Noordhoff Publishers, p.31.

[44] Van Est, R., D. Bunders & I. Korthagen (2017). Rise of Robot City Politics: the State of Affairs in the Netherlands. The essay was presented at URBAN AUTOMATION, an international workshop in Sheffield, UK, on 4-6 September 2017.

[46] Hoorweg, E. et al. (2018). Vertrouwen en wantrouwen in de digitale samenleving. Trends in veiligheid 2018. Utrecht: Capgemini.