Artificial intelligence and machine learning can identify threats to an organisation – but at what cost to privacy and whistleblowers?
Troubled by something deeply unethical going on at work? Or maybe you’re plotting to leak sensitive information on the company that just sacked you? Either way, you best think twice before making your next move because an all-seeing artificial intelligence might just be analysing every email you send, every file you upload, every room you scan into – even your coffee routine.
The latest wave of cyber-defence technology employs machine learning to monitor use of the ever-expanding number of smart household objects connected to the Internet of Things – shutting down hackers before they’ve broken into corporate databases or whistleblowers before they’ve forwarded on information to the media.
One of the leading proponents is cyber-defence company Darktrace, founded in 2013 by former British intelligence officers in Cambridge and today featuring 370 employees in 23 offices globally. The company is targeting growth in the Asia-Pacific, where regional head Sanjay Aurora is promoting Darktrace’s Enterprise Immune System at the CeBIT Australia conference in Sydney on 23 May.
In an interview ahead of the conference, Aurora tells the Guardian that the Internet of Things, the interconnected everyday devices such as the smart fridge, offers more vulnerabilities to be hacked than ever before – but also more ways to scan for threats.
“In newspapers there is not a single day where we don’t read about an organisation being breached,” he says.
“At a time when even coffee machines have IP addresses, many people in security teams don’t so much as have visibility of the network.”
Where cybersecurity normally functions as a barrier to keep out previously-identified threats, Aurora says Darktrace technology behaves more like a human immune system.
“Once you understand the devices and people, once you notice subtle changes within the network, you establish a pattern of life, and whether it is lateral movement or unusual activity – maybe an employee using a device they don’t normally use, or a fingerprint scanner acting unusually – the immune system notices and takes action, detecting these things in network before they become a headline,” he says.
Darktrace’s package includes a 3D topographical real-time “threat visualizer”, which monitors everyday network activity, and the responsive Antigena system, which can decide for itself to slow systems down to give security personnel time to stop a potential breach, cut off network access to particular individuals, or mark specific emails for further investigation.
“Let’s say an employee is made redundant and becomes a potential information threat, the machine will intelligently determine what is the problem, assess the mathematical threat and then decide what action is to be taken,” Aurora says.
Darktrace claims its Enterprise Immune System has reported over 30,000 serious cyber incidents in over 2,000 deployments across the world, offering up examples such as an employee who was disgruntled about their company’s Brexit plans and was caught before they could leak the information. Another case was put forward by Darktrace co-founder Poppy Gustafsson at the TechCrunch Disrupt conference in London last year. Gustafsson cited the case of attackers sending a truck into the warehouse of a luxury goods manufacturer after uploading their fingerprints to the company’s system in order to bypass the biometric scanners.
“It’s one of the few attacks where a criminal has given their fingerprint ahead of time,” she said.
Darktrace is well on the way to establishing itself in Australia ahead of the CeBITbusiness tech conference, already boasting clients such as national telecommunications provider Telstra.
According to a Telstra spokesperson, the company “joined forces with Darktrace in 2016, adding it to a suite of complementary security technologies which are designed and utilised to protect customer and corporate information and the Telstra network. Darktrace, along with our other technologies, people and processes, strengthens Telstra’s internal security through its ability to detect anomalous activity and its ability to visualise all network activity, resulting in a reduced time to detect potential threats.”
The move has attracted concern from Communication Workers Union (CWU) national secretary Greg Rayner, who says the union was not consulted on the introduction of the technology.
“That’s disappointing and arguably a breach of Telstra’s obligations under the current enterprise agreement,” he says.
“They’re supposed to consult on changes that will have a significant effect on the workforce. Telstra employees have been subjected to increasingly intense electronic monitoring in recent years, including scrutiny and recording of their online activities at work. We are obviously concerned that this technology will allow further intrusions into employees’ day-to-day working lives.”
Telstra has history in regard to unions and whistleblowers – in 2008 former employee Jim Ziogas was fired after being connected to a leak to the media of internal plans to de-unionise the workforce.
Whistleblowers Australia vice-president, Brian Martin, doesn’t have a lot in common with Darktrace, but he does share a fondness for immune-system analogies. “Whistleblowers are antibodies for corruption in organisations,” he tells the Guardian. “If it were possible to prevent leaks (and that remains to be shown), this might only allow problems to fester until they become much worse. Think of what happened to Volkswagen, which lacked any whistleblowers or leakers and paid a much larger penalty than if its emissions fraud had been exposed years earlier.”
He says invading the privacy of workers has the potential to create resentment and undermine loyalty, and that a lack of independent monitoring means there are serious questions regarding the effectiveness of Darktrace’s Enterprise Immune System, particularly in regard to false positives and false negatives.
“The damage to morale done by falsely accusing an employee of planning to leak documents can be imagined,” he says.
“How about this option? Adapt the software to monitor the e-communications of top managers to see whether they are planning reprisals against whistleblowers. How do you think they would like that?”
Devised as it was by former MI5 and GCHQ agents, inspired by the challenges they were facing in counterintelligence, Darktrace technology is also an interesting proposition for governments, but the company is more coy about the countries that it counts as clients than the businesses it services.
For its part, a spokesperson for the Australian Signals Directorate (ASD) – the department of defence intelligence agency that bears the slogan “reveal their secrets, protect our own” – refused to confirm or deny use of Darktrace technology, telling the Guardian it does not “provide commentary on capability or use of commercial products”.
There are certainly plenty of rivals to Darktrace technology also promoting their cybersecurity platform’s integration of the latest machine learning capabilities, including CrowdStrike, Symantec and Cylance.
Then there are Darktrace’s true rivals – hackers themselves. Thomas LaRock, technical evangelist at IT company SolarWinds, warns that machine learning is a tool that can be used to attack just as easily as it can be used to defend.
If it is possible to use machine learning to build a model that helps them launch cyberattacks with greater efficiency, then that’s what you can expect to happen,” he says.
“Think of this as a spy game, where you have agents that go from one side to another. There is bound to be a person somewhere right now working on machine learning models to deter crime. One day they could be found to be working for the criminals, using machine learning models to help commit crime.”
Aurora defends the use of machine learning at Darktrace, arguing this is one game companies cannot afford to opt out of.
“If you look at the way the threat landscape is moving, it is just simply humanly impossible using conventional methods – the only way to react to these threats is AI and machine learning,” he says.
“We are proud to achieve on that front – pure, unsupervised machine learning, as employee behaviour changes. That is the secret sauce – continuously evolving and learning.”
Avots: The Guardian
