privacy and the surveillance explosion

As surveillance technologies become more ubiquitous, are we using them for good or for evil? The answer is Yes. The president of the World Future Society offers an overview of who is watching us and why.

It has long been a cliché that evolv- ing technologies drive social change. In part, this process is ener- gized by expanded capabilities for responding to a perceived problem or need. But the terrorist attacks of September 11, 2001, also certainly qualify as a change driver, especially in the area of proactive intelli- gence-i.e., early identification and effective response to security threats.

One of the leading areas of techno- logical response to this perceived need has been the explosive growth of surveillance capability (especially video surveillance). This has hap- pened largely at the government level, but the technologies are in- creasingly used by commercial en- terprises and private citizens, as well.

Surveillance encompasses the mon- itoring of behavior, movement, or other dynamic states (mostly involv- ing people) for the purpose of influ- encing, managing, directing, or pro- tecting. While most often used in crime and terrorism prevention, sur- veillance is also used for such work as epidemiological oversight by the Centers for Disease Control and Pre- vention, for example. Construction sites, warehouses, commercial office buildings, and parking lots also commonly have surveillance tech- nology installed for the protection of property.

One of the challenges facing sur- veillance technology is its common political association with negative images of continual spying on citi- zens in George Orwell's classic novel 1984. Since 9/11, however, public ac- ceptance of surveillance has risen in the United States, and the rate of ap- proval for public surveillance cam- eras continues to rise (now around 70%). Products like Microsoft's Kinect, which offers movement, voice, and gesture recognition (largely for video game applica- tions), have also increased public comfort and acceptance.

On the other hand, state laws that regulate audiotaping and videotap- ing of individuals, especially in pub- lic settings, are more in flux. The practice continues to face challenges from pro-privacy organizations.

Surveillance is a costly activity for cities. New York City's "ring of steel," which consists of 200 cameras in lower Manhattan, has already cost the city $200 million. And it's not just "Big Brother" (government) watching us: "Little Brother" surveil- lance is also growing-e.g., private systems installed in places such as retail stores. For example, critical Boston Marathon bombing footage came from a nearby Lord & Taylor department store, and footage of the shooting of U.S. Representative Gabrielle Giffords came from an ad- jacent Safeway grocery store.

Of course, as the use of this tech- nology increases, so do related pub- lic policy questions. Many feel that camera systems are most effective in the investigation stages of criminal activity, where parameters are clearly established, rather than the deterrent stage, where the boundar- ies on use are much vaguer. Support- ers of these systems point to the lack of adequate public education on the deterrent side.

Cost effectiveness and privacy is- sues will definitely continue to be debated as camera systems grow. For example, urban camera and drone systems are often installed for the stated purposes of traffic control, but then converted later to general sur- veillance. In fact, 12 U.S. states have banned speed cameras, and nine have banned red-light cameras. The District of Columbia announced plans in September 2013 to double its traffic-camera system, which had added $85 million to District coffers in traffic enforcement fines during FY2012. In a Washington Post survey, 63% of respondents supported red- light cameras and 53% supported speed cameras. System defenders cite an Insurance Institute for Fligh- way Safety report finding that red- light cameras have reduced fatal crashes from running lights by 24% in all cities, and reduced all types of fatal intersection crashes by 17%.

Information Enhancement and Interpretation

Surveillance cameras can already track thermal differences, distin- guish details and color in very dark and low-light situations, detect mo- tion, control access, and provide re- tail point of sale (POS) observation, as well as identification and recogni- tion. Using biometrics at POS does not require central storage. You carry the biometric information on a card with your ID and then use it to ver- ify identity as part of the authentica- tion process.

What now represents the cutting edge of surveillance technology is intelligent analysis of the visual im- ages being collected. A federal research program called Mind's Eye is part of a visual intelligence pro- gram that the Defense Advanced Research Projects Agency (DARPA) is developing. Mind's Eye is a series of visual algorithms designed to pre- dict human behavior, including threatening behavior or dangerous situations.

While it is still several years from full deployment, Mind's Eye will de- scribe actions in surveillance areas us- ing text messages and offer possible interpretations. It is also designed for use by unmanned vehicles in danger- ous or uncontrolled areas. The pro- gram is even able to interpolate unob- served off-camera actions to make judgments. DARPA's research partner in this program is Carnegie Mellon University's National Robotics Engi- neering Center.

Operating in parallel to this is the development of a Spatiotemporal Activity Reasoning Engine at , which focuses on pulling out and assessing the "nouns" and "verbs" in any visual image. Inher- ent questions in this process are: What are we seeing? What is happening here? and What should we be doing in response?

One special area of interest in this arena is the prediction of, and re- sponse to, potential sexual assault and other interpersonal crimes in public and commercial spaces. Inter- preting and reacting to social inter- action as it slides into assault or lar- ceny is a potential challenge even for human monitors; it may prove diffi- cult to establish viable guidelines that navigate the boundaries be- tween failure to flag criminal activity and excessive false positives.

Enhancing Visual Surveillance With Biometric Identification

When we consider the identifica- tion and recognition aspects of vi- sual surveillance, we still need to ask Who are we seeing? and Are they who they seem to be? This challenge is be- ing addressed by melding biometrics with video surveillance, and some highly creative solutions are under development.

Biometrics of all types are already being used for identification and verification, including fingerprint, face, iris, speech, and DNA analysis using hand-held portable devices in combination with fixed systems. The National Institute of Standards and Technology (NIST) is now working on protocols for Web biometric sys- tems and cross-modality testing, such as simultaneous face and iris capture plus no-touch fingerprint capture. These biometric and visual tools could also be used to authenti- cate e-commerce and POS transac- tions. The U.S. federal standard is now a maximum of 20 seconds to capture a fingerprint scan, and the commercial market is likely to push this limit much lower.

This growing role of software in analyzing video feeds in real time naturally leads to a decreased need for 24/7 personnel, with their well- known limits in attention span. This software never sleeps, and it notifies supervisors and response personnel to a critical incident immediately.

In essence, human memory is be- ing outsourced to mobile technology for the capture and assessment of still and video images and decision making based on their content. Or to put it another way, decision support is moving increasingly toward deci- sion automation in commercial settings.

* Facial-recognition technolo- gies. Verification is matching a real- time image against a claimed iden- tity by utilizing a library of certified images. Recognition involves match- ing an image against an identity li- brary to provide a name. The aver- age person has the capacity to recognize at most 1,500 faces at any one time. A NIST report from 2012 states that the capacity of facial- recognition technology has im- proved by two orders of magnitude over the last 10 years, while recogni- tion errors have been dropping by 50% every two years. Clearly, its use will continue to increase.

U.S. programs are modest com- pared with those in China. The na- tional Golden Shield program aims to use facial recognition to track every one of China's 1.3 billion people. The state of the art in China is now the ability to verify identity using facial-recognition software from up to 500 feet away.

Another public-policy question that is gaining momentum is the practice of building third-party fa- cial-recognition files from Facebook personal profiles. While social net- works now connect one-third of the world population, these connections are often informal and abbreviated, directed to everyone and to no one. Nonetheless, they greatly facilitate data mining.

Data mining is the use of statistical techniques and programmatic algo- rithms to discover previously unno- ticed relationships involving data, usually relating to individuals or groups. This includes economic and social behavior patterns, which may be suggestive of other less construc- tive activities and therefore useful in surveillance.

This data mining capability has been called "big data" in the media, and it is perceived as being able to peel away the social covering and re- veal the personal secrets of us all. Wired magazine editor Chris Ander- son argued in a 2008 issue of his magazine that big data has now ren- dered the scientific method obsolete. If one throws enough data into an advanced machine learning tech- nique, he argues, all correlations will be revealed, thus explaining everything.

While a preference for observation over modeling is still evolving, the definition of big data offered by IBM seems useful in its clarification: Big data is characterized by volume, va- riety, veracity, and velocity. This means that volume (data set size) alone is not enough. These massive data sets must be sufficiently rich to cover the range of factors one wants to investigate, sufficiently depend- able to accurately reflect real-world factors, and sufficiently nimble to avoid it taking the rest of one's life to answer a large data question.

* Other biometrics developments. The future of biometrics seems quite extravagant. For example:

- Body-odor research, under way at the U.S. Department of Flomeland Security, including its use as a lie de- tector (tracking odor changes, like galvanic skin response in the poly- graph).

- 3-D imaging being developed in Japan to capture gait and walking style (90% accurate) and barefoot print analysis (99.6% accurate).

- Keystroke signature (how a pass- word is entered) using speed and rhythm patterns in combination with other factors.

- Palm vein patterns (already be- ing used in some U.S. schools) scanned by a near-infrared reader without physical contact.

- Nose profile based on shape and size (not as accurate as iris scan).

- Japanese use of human posterior patterns, developed for antitheft sys- tems in the seats of automobiles (98% accurate).

- Ear print of outer and inner ear (99.6% accurate).

- DNA, which is still too expensive but is coming down in cost very rap- idly.

Portable smartphone platforms for biometrics are also developing rap- idly, as is the market penetration of the smartphone itself, which recently surpassed the "feature" cell phone in global unit sales.

Smartphone prototypes in devel- opment include capabilities such as 3-D viewing (without special glasses), above-screen projection, and a maneuverable viewer point of view. All of these capabilities will al- low security-system monitoring at much greater levels of detail.

Secure data and secure sites will be accessible through multiple-level biometric authentication and verifi- cation tools. Fingerprint identifica- tion from screen touch, facial recog- nition from phone cameras, and voice recognition are all expected to be part of this package. Screenless smartphones (which reduce battery size and weight) will be activated by touch or voice via a cloud interface. When a screen is needed, any nearby computer will allow display of that data on its screen.

In the past, identity could be es- tablished by things we carry (driv- er's license, passport, etc.) and things we remember (passwords, PINs, etc.). But these are quite vul- nerable to theft and hacking. Biomet- rics, while more secure, will also fol- low an individual through a lifetime, thus raising concerns about privacy and unauthorized uses.

Physical attributes such as retina patterns, vein patterns, and hand configuration are harder to alter than behavioral attributes. But behavioral biometrics programs will be able to assess emotional states from such factors as facial cues, posture, ges- tures, and the tone, pitch, and speed of speech. The International Biomet- rics and Identification Association is working toward identity-manage- ment protocols that will be both ap- propriate and effective.

So far, however, the rate of techni- cal innovation has far outpaced the rate of social invention, especially outrunning legal and ethical struc- tures, as Chris Dede of the Flarvard Graduate School of Education has observed. A working protocol now exists whereby U.S. and UK law en- forcement and intelligence services can activate the speaker on a cell phone and thus listen to any conver- sations taking place nearby. RFID (radio frequency identification) chips normally used for warehouse secu- rity, now also enable employers to track personnel.

* Drones. Unmanned aerial ve- hicles and aerial surveillance drones such as the MQ-9 Reaper can iden- tify the heat signature of a human body from a distance of 60 kilome- ters (about 37 miles). The prospect of such drones in private hands is alarming. Who will be empowered to use them, and who will be ob- served? At present, drones are still treated by the FAA like model planes. No flights are permitted above 400 feet (about 122 meters), and commercial use is prohibited.

The ever-greater use of domestic drones in law enforcement raises the policy possibility of cameras now, weapons later. At what point might public demonstrations become tar- get-rich environments?

An interesting industry that has sprung up in response to the private sector surge in commercial drones is in countersurveillance equipment. These include heat-signature-cloak- ing garments (initially funded on Kickstarter) and face paint (known as CV Dazzle), plus mouth inserts to foil facial recognition. In addition, privacy Web sites and blogs dis- pense advice about turning off cell phones and removing the battery to foil GPS tracking, or encasing the phone in a customized foil packet marketed under the name OFF.

Privacy Issues and Public Concern

U.S. courts have ruled that the use of facial-recognition technology may be restricted under the Fourth Amendment's prohibition of "unrea- sonable search and seizure." Yet, at the same time, Americans' First Amendment rights may cover the capturing of images as a protected form of self-expression, so long as this does not involve tracking an in- dividual in a public space. The ques- tion here is the reasonable expecta- tion of privacy in specific situations. U.S. Senator Al Franken (Democrat- Minnesota) strongly supports the position that people have a funda- mental right to privacy, but regula- tion is lagging behind technology- which is not desirable.

Public opinion swings widely in the United States, as seen in re- sponses to revelations concerning massive surveillance by the National Security Agency (NSA). Those dis- closures have increased public con- cerns about protection of privacy rights, as reflected by an August 2013 poll by the Associated Press- NORC Center for Public Affairs Research. This survey shows a con- tinued concern about public safety and terrorism (related to the 9/11 at- tacks and subsequent events), but many people have reservations. Only 53% said that the U.S. govern- ment does a good job of ensuring freedoms, compared with 60% two years ago. And 61% felt that the im- pact of 9/11 has grown to the level of "a great deal," compared with 50% in 2011. On the disapproval side, 61% surveyed said they oppose government collection of the online communications of citizens.

In many cases, U.S. government policies seem contradictory. For ex- ample, NSA practice between 2006 and 2009 directly violated guidelines set by the Foreign Intelligence Sur- veillance Court. The Family Educa- tional Rights and Privacy Act limits the release of student records and personal identifying information to third parties. On the other hand, the use of cameras in U.S. public schools (often operated by private contrac- tors) has grown steadily-to well over 50% today.

Another concern is the rapid trans- fer of surveillance monitoring to the cloud. While this may decrease over- head and installation costs-Internet protocol (IP) versus closed circuit television (CCTV), or digital versus analog-questions about data secu- rity remain. The digital IP pathways provide easier access to the surveil- lance stream for users, two-way communications, inexpensive and easy digital storage, improved image quality, ease of maintenance, and system scaling. Of course, open IP frameworks also offer easier access by unwanted third parties, which has been a downside of the digital cloud since its inception.

It is common for public opinion surveys in the United States to re- port that, while most responders think the government should be able to collect DNA in order to catch criminals and terrorists, they also ob- ject to that government having sam- ples of their own specific DNA. Flowever, one recent Supreme Court ruling, which defined DNA collec- tion as a legitimate police practice, reinforces its use even by local police departments. The Communication Assistance for Law Enforcement Act also enables searching of a wide range of conversations for trigger words or phrases that suggest suspi- cious behavior, as well as monitoring visits to certain Web sites or commu- nications with suspicious individu- als or groups. This is largely done through text scanning, capture, and analysis programs, such as Carni- vore and Coolminer, both developed by the FBI.

In response to the growth of these sorts of capabilities, groups like the Electronic Privacy Information Cen- ter, which was founded in 2001, have grown up to deal with such issues as airport body scanners and biometric data collection and retention. Also influential is the Identity Counsel In- ternational, headed by Joseph Atick, which focuses on the social impact of technology and the commoditization of identity data.

The definition of what constitutes identity data continues to expand. It now includes the ability to track smartphone user location, personal networks, likes/dislikes, and com- mercial behavior. It is coming to in- clude a fuller understanding of how connected humans influence and persuade one another, such as how word-of-mouth marketing could be efficiently managed and even be used to build networks that provide peer-to-peer purchasing advice.

Finally, there are potential political consequences to consider, including developing the ability to identify in- fluencers in any network, track be- havioral indicators of growing men- tal illness, and track the spread of memes, especially political ones (based on the epidemiological con- cept that disruptive ideas are a form of social contagion).

Actual epidemiological behavior (e.g., the spread of influenza) can also be tracked, based on electronic movement and communications pat- terns using smartphone information. Even obesity is becoming somewhat predictable through social patterns. Accordingly, public health, urban planning, and marketing strategies might be guided by examining be- havior relating to the use of smart tech.

In the same manner, disaster-relief monitoring could be shaped using mobile-phone tracking and reporting (during the last Haitian earthquake), and geo-tagged media could be used to enhance public safety at a variety of levels (as after the Boston Mara- thon bombing).

Surveillance for Health and Social Services

One of the great challenges for ser- vices in the twenty-first century is their cost. Medical testing without doctor visits is one of the develop- ments under way to meet this chal- lenge. Medical assessments such as sonogram and blood work can now be managed at home by the patient through innovative smartphone apps.

Less-expensive medical-monitor- ing environments can be provided at home by what have been called Granny Pods in the United States. These are portable manufactured housing, or auxiliary dwelling units, that can be equipped with cameras, vital sign sensors, Internet connec- tions, and climate management. Most commonly placed adjacent to a family home, they provide addi- tional bed space that was otherwise unavailable. A number of private companies are now offering them in the United States and Europe.

All of these technical stopgaps rely on the growing acceptance of tele- health and the belief that computers continually tracking blood pressure, glucose levels, and other indicators can yield adequate health-care moni- toring 24/7. The biggest criticism is that such remote monitoring is ster- ile and unnuturing and thus unlikely to improve a patient's physical or psychological well-being. This may be especially true for patients with heart disease or other serious chronic conditions, such as diabetes and pulmonary disorders.

On the cost side, evidence from the UK suggests that such monitor- ing could produce a yearly savings of some £2,300 per patient (about $3,600), compared with the cost of traditional institutional care. This is certainly relevant in this age of gov- ernment fiscal constraint.

From Big Brother to E-Government

The growth of e-government ini- tiatives around the world has been steady and transformative. Starting in cities such as London, the initial role of surveillance was largely re- lated to public and traffic safety. But with the growth of Smart Cities and other technology-driven policy en- deavors, the integration of inter- active video into a wide range of governmental services seems very likely.

Some analysts see 2020 as the year where broadband connectivity hits the 4G level (100+mbps) for all smart personal devices worldwide, allow- ing public interfaces to function at a new level. This is already being demonstrated in the growing cloud capacity of governments, such as the UK's G Cloud. It would also enable the fully measured society and the Internet of Things planetwide. A sensor network like the one being prototyped by HP Labs-where such factors as light, temperature, humid- ity pathogens, pesticides, etc., can be measured and recorded at nearly every point on the globe-will en- able us to test models and monitor patterns in a wide range of fields. Zoom panoramic technology captur- ing actual cityscapes could be com- bined with digital modeling to update planning documents in real time.

On the interactive side, smart- phone application interfaces could enable two-way citizen input and di- alogue. Civic organizations provid- ing repositories for crowdsourced data could serve as "urban observa- tories." Systems already in place in- clude LIVE Singapore!, which runs real-time open data streams that allow city residents to tag and track trash pickups. Transport systems for food, energy, or water services could also become part of a transparent tracking system as the world's cities get smarter.

One of the few downsides to this vision is that most of this tracking will likely be done via the Internet. While management of the Internet is internationalized, its regulation and restrictions are still largely national.

What's next for e-government at the regional and global level remains unclear. However, both the govern- mental and market-based motiva- tions are robust, and solutions are likely to be crafted in relatively short order. ?

Cluster of cameras overlooks Times Square in New York, part of the city's "Ring of Steel" surveillance system. The network of cameras is designed to spot suspicious behavior or unattended packages, for example, reporting in to a central police command center.

DARPA's Mind's Eye program attempts to develop visual intelligence for unmanned surveillance systems.

in essence, hunnn nenoRy is beinq outsouRced to nobile technoLoqy For the cnptuRe nnd nssessnent oF stiLL nnd video mnqes nnd decision nnKinq bnsed on theiR content.

Among the biometric technolo- gies that the U.S. National Institute of Standards and Technology (NIST) has been developing for identity verifica- tion are (clockwise from top left) iris scanning, wireless finger- print data transmission, voice- print matching, and personal identity verification cards.

uith the qROUth of snnRt cities nnd otheR technoLoqy-dRiven policy endenvoRS. the inte- qRRtion of inteRRctive video into r uide Rnnqe of qoveRnnentnl seRvices seens veRy liKely.

About the Author

Timothy C. Mack is presi- dent of the World Future Society and executive editor of World Future Review. His last article for THE FUTURIST, "Foresight as Dialogue," was published in March-April 2013. E-mail [email protected].