Causality Link Uses Amazon Translate to Bring in Global Perspectives

Guest post by Greeshma Nallapareddy, Startup Solutions Architect and Daniel Omachonu, Fintech Startup Account Manager, AWS

As an investor, Eric Jensen, co-founder and CTO of Causality Link, was frustrated with how difficult and time consuming it was to project trends in financial markets. Too often, he found there was either no information available or only regurgitated sources, and he decided to change how investors consume information to make decisions. Eric started Causality Link to empower investor decisions with natural language processing (NLP) and provide information from around the globe in a consolidated and interactive platform.

Eric Jensen, CTO & Co-Founder

Financial markets are impacted by a multitude of global factors. To get a complete, diverse perspective on a company or industry, information from around the world must be taken into consideration. Causality Link uses Amazon Translate to process over 500 million characters in 24 languages each month. These characters come from sources all over the world including news, broker research, earnings call transcripts, filings, licensed aggregators, in-house financial market analysis and RSS feeds. The information from these millions of documents fuels Causality Link’s NLP engine which enables them to find causal relationships in financial markets.

The Causality Link Research Assistant ingests content, and an in-house algorithm establishes the likelihood that the content matches the language metadata. Texts that are identified as being primarily non-English are assigned priority levels for translation as a batch process or near real-time process. High priority texts are added to an Amazon ElastiCache First-In-First-Out (FIFO) queue based on the language of the text. An Amazon Elastic Container Service cluster with the AWS Fargate launch type hosts the service that works on the queues. These task runners can be scaled based on queue depth, and are single thread, single task, lightweight NodeJS processes saved as custom ECR images. The language-specific queues are polled breadth-first to maximize the translation throughput.

The translation process breaks articles into chunks of content, trying where possible to use natural paragraph and sentence breaks to ensure the translation model is given context that can aid the quality of translation. The reassembled article in English is updated and users interested in the content are able to view it immediately. Subsequently, the same task mechanism is used to efficiently schedule the newly available article for NLP processing, a process managed with an Amazon Simple Queue Service (SQS) queue due to more stringent requirements around reliability and the scale of distributed resources used for those processes, at times requiring many hundreds of cores.

“The philosophy of our platform has been to tap the wisdom of crowds by aggregating diverse viewpoints from around the globe. It would be extremely challenging for our small team to manage and maintain multiple ML models, supporting ontology and NLP modules targeting multiple different languages. Having a high quality, high-capacity translation solution enables us to focus our work on the best possible interpretation of English while allowing our reach to expand significantly in terms of global understanding,” says Jensen.

Before leveraging Translate, Causality Link declined opportunities to work with non-English text or had to use human translation to add it to their system as English text. A large portion of the foreign text they had licensed had been left untranslated and unused. Jensen says they “now have a great deal of flexibility and capability to work with customers whose native languages and text archives are not necessarily entirely in English.”

While there is already a large amount of translated news available for consumption, Amazon Translate allows Causality Link to get closer to the source and detect differences in the focus and emphasis of documents, as well as the causal explanations authors tie to changes in financial markets. “When we’re trying to build a consensus number or a true wisdom of crowds estimate, we want to collect information from as many different perspectives as possible, and having access to translated text is one great way to contribute to this,” says Jensen. This also enables Causality Link to tap into new markets such as Europe, where they are differentiated by their ability to process customers’ internal information which includes diverse languages. Their system is now reading tens of thousands of published news articles each day and they can monitor how information and ideas propagate across the globe. Their system can have a foreign language translated headline, NLP metadata and sentence-level quotes available in their real time APIs in under 30 seconds from their original receipt.

Causality Link has three team members that specialize in NLP. By offloading translation and elastically scaling with news flow, the AWS cloud allows these critical resources to concentrate on producing the best possible interpretation of a single language. Amazon Translate allows the Causality Link team to focus on what differentiates their business: gathering and processing text from around the world to glean valuable financial insights. While their competitors focus on using sentiment analysis to analyze statements to estimate the overall perception of a company or consolidate structured information about companies like earnings estimates, Causality Link goes beyond that, developing more robust findings that take into consideration more diverse perspectives. “We call our solution ‘Sentiment 2.0’ because it’s both much more precise (sales of Tesla Model 3 in Europe vs. Tesla as a whole) and less emotional – we aren’t trying to measure whether a company is liked per se, but rather whether their business is looking healthy or not, both in their reported numbers and the causal drivers that enable them,” says Jensen.

Causality Link has two primary types of customers: quantitative experts who wish to use their data to systematically calculate opportunities and risks within the market, and portfolio managers and analysts who leverage their software interface to support their analyses. Causality Link has gotten positive feedback from both groups on the flexibility and customizability of their system. This allows their customers to use the system to find information and events that are most relevant and important to them.

“We believe that the percentage of supported language pairs we are using is very high and tackles even very large articles gracefully. We are unique in that our system delivers insights gleaned from global news sources that can be utilized purely as ‘alternative data,’ or that can be read and interacted with by human analysts,“ says Jensen.

Causality Link’s unique use of Amazon Translate and blend of machine learning, a specialized ontology and traditional artificial intelligence allows them to provide their customers with an NLP-powered tool that guides investment decisions with news and information on an international scale. They are able to bring in more diverse perspectives from new markets where foreign language analysis is required to draw well-informed conclusions. Using Amazon Translate, Causality Link delivers a unique and differentiated solution to empower investors to make better decisions with information consolidated from around the globe.

About the Authors 

Greeshma is a Startup Solutions Architect at AWS. She helps startups leverage AWS to grow and change the world.

Daniel is a Startup Account Manager at AWS. He helps startups accelerate growth on the cloud and build for scale.