Pattern of life analysis: how timelines uncover hidden behaviors

Pattern of life analysis is a powerful technique used in investigative applications. It helps investigators understand the habits or behaviors of persons of interest based on large quantities of observed data. In this blog, we look at how timeline analytics can help in this scenario. We’ll use KronoGraph – our timeline visualization toolkit – to reveal the hidden patterns in our data.

Pattern of life analysis, powered by the KronoGraph timeline visualization toolkit
Pattern of life analysis, powered by the KronoGraph timeline visualization toolkit

The pattern of life visualization challenge

There are many wide-ranging reasons why pattern of life analysis might help your use cases:

  • Finding insight in anonymized datasets – it’s a popular technique in signals intelligence (SIGINT) and open-source intelligence (OSINT) methodologies. Applying pattern of life analysis to metadata generates intelligence from communications without needing to see the content of the communications themselves.
  • Suspicious activity and fraud investigation – understanding ‘typical’ behaviors makes it possible to identify unusual activity.
  • Predictive intelligence – if we know how criminals usually act, we can make pre-emptive operational decisions. For example, analyzing airline ticket purchases made by known drug traffickers may reveal patterns. This can help border surveillance investigators decide which flights to intercept.

If you want to see how KronoGraph fits into your investigative toolkit, take a look at 5 popular use cases for KronoGraph timeline analysis.

Scale wrapping makes pattern of life analysis possible

We recently launched version 1.1 of KronoGraph which includes a simple but extremely powerful scale wrapping feature. When combined with KronoGraph’s innovative heatmap view of large quantities of temporal data, scale wrapping instantly shows us behavioral insights from a soup of activity.

Want to know more?

Get an in-depth introduction to visualizing timelines with KronoGraph.

Download the white paper

To explain how it works, imagine we collect data on emails sent by staff in an organization, and we classify them by type (support, out-of-office, sales, etc.). Each email has a timestamp, making it perfect for temporal pattern analysis. A year’s worth of data loaded into KronoGraph might give us something like this:

Visualizing an organization's email traffic as a timeline
Visualizing an organization’s email traffic as a timeline

At first glance, we can see very different patterns depending on email type. For example, promotional emails are more sporadic than support emails, but are there more patterns hidden in this dataset?

Scale wrapping lets us choose a specific time period (a day, a week, a month, etc.) and show where in that time period each event happened. If we switch to a ‘day’ view, our heatmap shows that the various world regions tend to send their support email at specific hours within each day, but there’s no such daily pattern for promotional emails:

Visualizing the same dataset by time of the day
Switching our timeline scale to reveal email traffic by time of day

Switching to a monthly view shows that promotional emails tend to happen on the first and 15th day of each month (although a glance at the initial yearly view shows that they take a brief break in August):

The same email data, presented by day of the month
The same data, presented by day of the month

Let’s take a look at a larger, more realistic dataset to see what other patterns we can spot.

Behavioral analysis of anonymized communication data

For the following examples, we’ve used the wonderful Radosław-email dataset (Michalski, Radosław, 2020, “Manufacturing company email metadata and corporate hierarchy”) which provides the timestamps of some 82,000 email communications between around 170 employees of a mid-sized manufacturing company.

Typical of intelligence data, we only have partial information: we know when the emails were sent, and we’ve anonymized IDs of the participants, but we don’t know who the participants are, or what their emails were about. Nevertheless, KronoGraph’s pattern of life visualization gives us a great deal of insight into behaviors.

We’ll start by simply loading the data into KronoGraph, with an entity row for each employee and an event for each email. We don’t see a great deal initially, but we can identify some employees who send very few emails (perhaps people on the factory floor with limited access to email clients?) and we notice a few dark spots in July and August during holiday season:

Radosław-email dataset, visualized in KronoGraph
Radosław-email dataset, visualized in KronoGraph

Now let’s switch on scale wrapping, and set it to a scale of one day. Here’s the result:

Changing the scale reveals the daily email traffic patterns
Changing the scale reveals the daily email traffic patterns

Immediately we spot familiar sequences and can start to analyze temporal patterns in the data. The heatmap reveals when employees typically start and leave work. We can see which employees send emails late into the evening and which ones stop communicating at 5pm sharp.

Digging deeper into our pattern of life analysis

Notice the unusual bright spots in the top corners of the picture? They appear to show regular communication between a small number of employees at around midnight local time. What’s happening here?

Zooming into this interesting area, we see that it’s made up of extremely regular emails from one employee (employee #1) to a group of others. At this time of night, and with this regularity, it’s almost certainly an automated email, perhaps a regular automated report?

Focusing on employee #1, and their regular late-night group email
Focusing on employee #1, and their regular late-night group email

I can now highlight this report in yellow, and switch my scale wrapping to ‘week’ mode. This shows me the wider context and typical pattern of these reports:

The group email, highlighted yellow, follows specific weekly patterns too
The group email, highlighted yellow, follows specific weekly patterns too

They are most common on a Monday evening. By the weekend, they’ve pretty much faded out.

Run pattern of life analysis on your data

Get started with a KronoGraph trial

Request a trial account

Enriching data for behavioral analysis

We’ve been able to spot working hours, vacations, late-night working and automated reports, all from a completely anonymized dataset. But pattern of life analysis gets even more exciting when we can enrich the dataset with more information.

This version of the Radosław dataset contains some useful information for enrichment – an organizational chart of the company, which lets us identify groups: managers, regular employees, the CEO, etc.

We’ll use KronoGraph’s powerful types feature to color code the ends of each email based on the position in the organization of the sender and recipient. To keep things simple, we’ll focus on a smaller number of employees. Here’s the result, with scale wrapping set to one day:

Coloring our entities by their position in the organization: CEO, managers and employees
Coloring our entities by their position in the organization: CEO, managers and employees

It’s the same daily pattern, and we can clearly see employee #1’s nighttime group emails to the right of the picture. We can also use color to see how patterns differ between different groups.

For example, the CEO is shown at the top in yellow. We can see that this CEO follows a different pattern to most employees, who often start earlier and work later. Focusing on the CEO with a simple double click filters just their activity with the rest of the organization:

Focusing on the CEO highlights employees who communicate with them most
Focusing on the CEO highlights their individual communication habits

Two employees in particular have very yellow timelines, implying that they talk to the CEO directly and frequently.

Let’s focus on the relationship between the CEO and manager #68. Zooming in brings out individual events from the heatmap.

The communications between the CEO and manager #68
The communications between the CEO and manager #68

This shows us that manager #68 tends to come in an hour earlier than the CEO and gets a huge number of emails from employee #85 before the CEO arrives. Perhaps getting briefed on projects before a morning meeting?

This kind of pattern of life analysis reveals valuable insight into who usually does what, and when.

Needles in the haystack

Pattern of life investigations are not just about spotting normal behaviors. We can also look for patterns that surround unusual activity.

Switching to an annual view of our email communications, we notice an unusual black spot in employee #13’s activity. Almost certainly a summer holiday (but a long one – two months!).

pattern of life analysis makes it easy to spot anomalies in data
Pattern of life analysis makes it easy to spot anomalies in data

Let’s zoom in and see what activity preceded and followed this gap.

You can see that on either side of the break, the employee sends emails to a red entity – a manager. One before leaving (“I’m off now, see you in two months!”) and a short exchange a week after returning.

A final farewell before employee #13 heads off for a long break
A final farewell before employee #13 heads off for a long break

This ability to move seamlessly through the data, spotting trends and unusual patterns, makes our interactive timeline tool uniquely powerful. Go from a bird’s eye view of an entire dataset, down to a detailed view of a single sequence of events with just the roll of a mouse wheel. The data looks good and reveals insight at every scale. These are just some of the reasons why KronoGraph is so compelling for visual pattern analysis.

Bringing it all together

Using a timeline to investigate behavioral patterns is powerful. Even more powerful is the combination of a timeline with other visualization techniques, such as the link charts provided by our KeyLines and ReGraph toolkits.

To give just one example: if you fuse geospatial metadata, such as cell tower location, with the pattern analytics of KronoGraph’s scale wrapping, not only can you spot unusual behaviors, but you can also see how the movements of a person of interest correlate with those behaviors.

In other words, where was the subject going and what were they doing when the unusual activity took place?

Combining the timeline with geospatial link chart adds more context to our pattern of life analyis
Combining the timeline with geospatial link chart adds more context to our pattern of life analysis

This kind of Activity-Based Intelligence is a powerful technique, made possible by the interactive visual analytics of tools like KronoGraph, KeyLines and ReGraph.

Getting started with these toolkits is simple. Just sign up for a free trial, or get in touch to find out how we can help you build the best investigative application for your users.

More from our blog

Visit our blog

Registered in England and Wales with Company Number 07625370 | VAT Number 113 1740 61
6-8 Hills Road, Cambridge, CB2 1JP. All material © Cambridge Intelligence 2021.
Read our Privacy Policy.