(Re-)Building trust in Personalisation

How would you like to get news personalised to your taste? If this question doesn’t appeal to you, it’s not a surprise. While the concept of offering online content fitting to the person viewing it is nothing new, there is a great level of mistrust in the idea for different reasons.

Ever since social networks became popular, we are all familiar with the idea of seeing what we like most. Or at least the equivalent of what an algorithm thinks we do. Facebook is the most well known provider of such a timeline. And it’s also Facebook that is one of the main reasons people are questioning the content they are seeing: We don’t know whether the information we get is correct, whether it’s complete or if we’re missing out on something. Nor why exactly we are being shown this information - we don’t have control over it.

(Results from the online survey done at Deutsche Welle clearly show the wish for more transparency)

What is personalisation?

Personalisation can take a multitude of different forms. A timeline like in Facebook or Twitter is only possible with an unlimited number of news items at your disposal. Something news companies, especially local ones, usually don’t have. But personalisation can also mean to just show people a limited number of items chosen specifically for them based on certain preferences. Focusing on a specific area, a certain time of the day, serving news just in a specific length or just addressing specific topics appealing to the reader.

CPN’s aim and how we approached it

In short: We want to build trusted personalisation as a service for news media. Meaning, we want to find an approach that is trustworthy through transparency and control, that doesn’t give you the feeling of missing out on important topics, that does put you in a filter bubble, by feeding you only one side of the story and that does give you control over what influences what you are seeing. Sounds complicated, doesn’t it? Here is how we have approached the issue.

We first took a close look at what is already out there, learning that media companies have been late to the game. Social networks and marketing have long ago started personalising their offers. Media companies have yet to find the right approach, and while there are a lot of experiments going on, it seems clear, that there isn’t one overall solution.

So we decided to get the info first hand and invited our audiences to help us. In co-creation sessions (at VRT and DIAS) and through online surveys (DW), we reached out to real users and asked for their opinion on personalisation. We invited both producers and end users for the workshops and contacted audience members of DW for the online questionnaires.

In all three cases, participants were open for the idea of personalising news individually in particular because of the information overload that we are all facing online these days. But it also became very clear, that users wanted a transparent system, not a black box. Understanding the decisions made by the system, was a key requirement, followed by the preference to being able to influence those decisions by keeping control over one’s own data.

(Part of a scribbled scenario created during a co-creation session at VRT)

The workshops were also used to create some first ideas of how a new personalisation system could address the users’ needs - proving once more that personalisation is just no one-fits-all solution. Based on the personas created before the sessions, the workshop participants discussed different scenarios for “news junkies”, different categorisation options on hard facts versus more emotional aspects and how to best set up user profiles following new approaches.

The online survey gave very similar results, also stressing the information overload and underlining the need for a transparent system, but also highlighting the trust issue. It seems like people are rather reluctant to give out certain personal data due to (bad) past experiences. Following after the workshops the surveys also allowed to already test the acceptance towards certain ideas from the workshops ro avoid the fear of missing out (FOMO) or Filter Bubbles. The majority of the participants shared the fear but also favored the suggestions to limit those effects, for example by introducing alternative items, not matching the users profile.

The real challenge

After analysing all the material from the sessions and surveys, the project team transformed the information into a structured set of first requirements, addressing the main pain points identified in the previous steps. Those requirements are now the basis for the first CPN prototype to be developed by the technical partners in the project. Building this prototype will not be the issue, as the project already has a set of modules in place to set up the basics in creating user profiles, analysing media items and matching both with each other.

The real challenge will be to turn this into a personalisation algorithm, that really produces good matches, while working transparent enough for the users to understand why they are shown certain results - an aspect strongly tied also to the new European Data Privacy Regulations (GDPR) . Keeping the users in the loop (e.g. through regular evaluation sessions) is only going to be half the work. Coming up with the right ideas to create smart profiles and correctly match news items to interests is the other part. By letting the users participate along the way, we do hope to find the right solutions for that and keep the system transparent enough to build the necessary trust in it for it to work successfully, while keeping data privacy a top priority.  

Beyond the state-of-the-art: innovative components in CPN

Compared to neighbouring media industries, the news publishing industry has been relatively reluctant to adopt personalisation in their offering. Understandably, since a lot of industry specific ethical, journalistic and business considerations come into play.

Nonetheless, the way is being paved by companies like The Washington Post, with its Arc Publishing platform, while Adobe, with recent personalisation options in their AEM publishing software, and others following suit.

CPN wants to actively take part in this evolution and assist European news publishers to adopt personalisation. Even though it’s still early days, we are convinced we need to look beyond the state-of-the-art to offer enough value in the near future. That’s why, in parallel with the current solution that is being developed, we will research the following innovations that are new to news publishing or show great potential:

• Content adaptation

• Contextualised datasets

• News bots, smart speakers and digital assistants

• Radical innovations in trust and transparency

Content adaptation

Recommendation engines might be very useful, revolutionary, and possibly even overdue in the news publishing industry, but their technology is hardly innovative. Under the bonnet, they use the content items as separate “units of information”, created in a one-size-fits-all fashion, not tailored to any specific target audience.

CPN wants to research if adapting the content inside those news items would have a positive impact on the users’ engagement and understanding of the news. In other words, would content adaptation, combined with recommendation methods - let’s call it multi-level personalisation - have an even greater impact than recommendations alone?

News articles have plenty of elements and attributes that can be adapted to the preferences of each individual reader: the title, the header image, the length, the complexity, the tone-of-voice, the required knowledge, … All these could be adapted by software algorithms. Or at the very least, those algorithms could assist the adaptation, so the journalist can make the last call and decide which versions would suit the different target audiences.

We know Netflix uses content adaptation for their artwork. We know that leading voices in news innovation like Titus Plattner have been advocating this type of advanced personalisation. And we believe our consortium has enough combined know-how and talent to put this to practice. VRT’s newsroom will be the perfect testbed to run these experiments.

External and highly contextualised datasets

Context is king! No one wants to read a deep diving opinion piece on the Iranian nuclear crisis for breakfast. Or maybe some readers do? Then we want to know, in order to serve them as well as possible.

(source)

CPN wants to research the use of external and highly contextual datasets to improve the types of content you’d engage with in each context. This will reduce friction, improve the return of the content offering and, ultimately, the understanding of the readers.

Personal data like the current occupation, a person’s emotions, moods and value sets, but also data that are more external in nature, like trends, the weather, traffic info and impactful social affairs like crises or elections, could contribute to the creation of intelligent contextual awareness for prediction algorithms.

We are aware that such data are very sensitive and should be handled with the utmost respect for privacy. However, when the user allows it, these data can be readily sourced from personal devices and sensors (smartphones, smartwatches, fitness trackers) and social media (a very good proxy for personal data traits, as demonstrated by Michal Kosinski in 2013).

CPN partner DIAS will look into the use of moods and value sets for use in personalisation algorithms, together with the other content partners VRT and Deutsche Welle.

News bots, smart speakers and digital assistants

Our research confirms that users wish for a more engaging, intuitive and seamless way to receive personalised news. That’s why we decided to include text- and voice-based conversational applications into our vision of the CPN platform.

These technologies are prognosed to grow significantly in the near future and they have the potential to provide a user-friendly and intuitive interface for CPN.

Chatbots have been around from the 50s, but are rapidly progressing today thanks to cheap hardware, advanced language processing and machine-learning technologies. Conversational apps in general are on the rise but haven’t been widely adopted in the news publishing industry. We want to know if they can be a valuable addition to the services of the CPN platform.

Voice-based digital assistants are a more recent technology. They have the potential to change product design as a whole, shift paradigms and advance our conceptualisation of human-machine interactions at large. The use of smart speakers skyrockets with the efforts of Google, Apple and Amazon pushing their devices to every household, and the software advancing so rapidly they even learn how to recognise emotions.

CPN wants to help news publishers enter that realm by researching ways to offer their content on these platforms, engage with users over time and even allow to create or update their news-preference profile through conversation, games and quizzes.

CPN partner Deutsche Welle brings the necessary experience to the table to do exactly that.

Radical improvement in trust and transparency

We realise that the technologies described above could be perceived as exceptionally intrusive. Add to that the malicious ways personalisation has been used in recent years and the trust in the news industry being at an all time low, and you have a difficult starting point for news personalisation.

That is why, along with other innovations, we are determined to radically innovate in our services’ privacy and transparency.

It is general knowledge by now, but our research confirms that questions (or even the slightest misconception) about the use of one’s personal data can raise an insurmountable barrier for the adoption of personalised offers.

Transparency and user control are of the utmost importance. We will research novel ways to deliver on these issues and to regain the user’s trust. Not only concerning their personal data, but also about the personalisation algorithms, in order to avoid pushing readers into a filter bubble.

The CPN technical partners’ know-how about security, combined with Digital Catapult’s excellence in all things GDPR, their experience in transparency modules and user control will ensure our services are solid, secure and trustworthy.

To conclude...

While this list of proposed innovative components is neither exhaustive nor definitive, we believe they are worth investigating to take CPN beyond the state-of-the-art.

CPN creating a solution for European news publishers to easily adopt personalisation will be a feat in and of it’s own, but given the rapid evolutions in this area we need to look ahead  and be ready for the future.

PS: follow us on Twitter!

CPN: Towards the creation of an Open Virtual Platform

The main goal of the CPN project is to create innovation in the way content creators can structure content production, distribution and in-depth interaction with audiences. To do this, CPN will realize an innovative Open Virtual Platform. This means that the platform will be 'open' to future third-party integrations and that it is not intended to be a physical system which media companies should install, but as a 'virtual' platform which can be adapted to the needs of media companies by connecting different services together.

These characteristics are very important because they will allow the platform to be reused even outside the CPN project and above all they will give the opportunity to enrich the CPN platform with more features in the future.

Reference Architecture

A reference architecture is a 'template solution' for an architecture implementation. It consists of a list of functions, indications and rules for interactions among the components within the architecture and with external services.

It is defined at different layers of abstraction and it is described through some 'views' that allow a better understanding of the architecture.

In CPN, the Reference Architecture document contains the guidelines and the template of the architecture that will be used to develop this Open Virtual Platform.

In particular the document describes the design and implementation choices to obtain a flexible and extensible architecture, and the guidelines for implementation, deployment and communication among components.

Microservices Architecture

To talk about microservices, we must probably start talking about Service-Oriented Architecture (SOA). A service-oriented architecture is essentially a collection of web services, usable independently, that collaborate with each other to satisfy a business process.

Microservices architecture is a variant of standard SOA, that structures an application as a collection of loosely coupled services (microservices).

Microservices is an approach to application development in which a large application is built as a suite of modular services. Each module supports a specific business goal and uses a simple, well-defined interface to communicate with other sets of services.

CPN Platform - Microservices architecture

After a thorough analysis and comparison of different architecture candidates, we have chosen the microservices architecture, which offers a series of benefits that are well suited for the CPN platform requirements:

• The microservices can be independently developed and deployed. This allows for the smooth integration of different components for platform functionalities, even if these are developed by different project partners, which will be the case in CPN.
• The microservices are designed to offer a range of 'micro' capabilities that group themselves to cover a variety of platform use scenarios.
• The microservices do not bind the platform to the use of a given technology stack*, but makes it interoperable with different operating systems.
• The microservices architecture lends itself well to both the creation of new services and the implementation of new processes, even after the deployment of the architecture. This meets the criteria of interoperability, flexibility and openness of the platform.

* A technology stack is a set of software providing the computers' infrastructure (for example the operating system)

Implementation of the microservices architecture

The image below is the 'Implementation View' of the CPN platform. This view is one of the possible views that allows to describe the architecture. In particular, it describes a possible implementation of the platform through microservices architecture. In addition to microservices, we can see the communication modules (Orchestrator for synchronous communication and Message Broker for async one) and the API Gateway, that allows external client applications to interact with the platform.

Finally, a blockchain instance is represented, which is used by some modules on the platform.

CPN Platform - Implementation View

The implementation of the microservices architecture will follow specific design patterns to ensure the efficiency, stability and scalability of the system.

Furthermore, strong and popular open source tools, as Docker, Kubernetes and Apache Kafka, will be adopted, to reinforce the concept of openness.

These tools, indeed, besides being open source, have a large community behind them and this will allow to make the platform more easily extensible.

Conclusions

In this article, we have seen the main goals of the CPN Platform and as a Reference Architecture document, it can be useful to reach them.

In particular, we have described the benefits of a Microservices Architecture and a possible implementation of the platform through this architecture.

Following this document and the instructions it provides, in the next steps of the project we will implement the first version of the platform.

Do you want to stay up to date about our activities and results? Subscribe here for the project newsletter and blog posts. Or follow us on Twitter!

3… 2… 1… Project CPN has taken off!

Group photo of the consortium members, taken at the CPN kick-off meeting.

Welcome to the CPN project (Content Personalisation Network)! With a team of broadcasters, media companies and technology providers, we work towards a new approach for personalising digital content, enabling economic impact for large and small news publishers.

Vision

Europe is scattered with media companies, large and small, that represent an enormous amount of cultural diversity. On a daily basis, millions of news content items have to find their way to millions of users. The CPN project takes up the challenge of developing a new approach to personalise the daily news offer, allowing both large and small media companies to better target their content to media consumers. As a media consumer, you will be able to better receive news, in the right format at the right time, and in a fitted context.

Objectives

With CPN, we will provide media users with a seamless and reliable news experience on different platforms and provide media companies with effective news distribution solutions. Therefore, we will develop a virtual open platform (with reference architecture) where on the one hand, media professionals are able to receive faster and more targeted cross-channel news and information distribution solutions, and on the other hand, users are able to experience more attractive and engaging news and information.

To make this happen, we aim to create novel services targeting content personalisation and real-time analytics on the platform. This will provide users with personalised news feeds, insights into the impact of news and information in their daily lives, while at the same time break them free from the filter bubble, which can contribute to more nuanced and factual opinions in larger society.

Throughout the duration of the project, we will iteratively test and validate the solution in operational real-life environments in different countries (Belgium, Germany, Cyprus, and Italy) by setting up large scale pilots.

Workplan

Take a look at our project website. There, you will find more information about the CPN project and a detailed description of the different work packages.

Do you want to stay up to date about our activities and results? Subscribe here for the project CPN newsletter and blog posts. Or follow us on Twitter!

Project Partners