AI/ML in Investment and Risk Management: Recent Applications, Use Cases, and Implementation Challenges

Arvind Rajan
Managing Director – Head of
Global & Macro
PGIM Fixed Income


Investing is a completely different ballgame from many of the other applications where AI/ML has been spectacularly successful. It presents some unique challenges both to the data scientist and the portfolio manager. 

Here, Arvind discusses some of the most promising techniques, including natural language processing, regime classification, and pattern recognition, showing how they can be applied to investment-related use cases. 


Arvind Rajan, PhD, is a managing director and head of global and macro at PGIM Fixed Income, heading the FX, Global Bond, Emerging Markets Debt, Investment Strategy and Economic Research teams.

Watch Arvind Rajan’s full presentation here

Challenge: looking for solutions that can give customers an edge in investing. 

Once you identify what’s useful, then you can actually try to apply it to asset management and risk management. It’s often difficult to implement new technology in an organization that’s used to doing things in a different way. The asset management industry and banking industries are hundreds of years old. Integrating quantitative techniques and data to traditional businesses is a major challenge. 

What customer data is most relevant (and what isn’t)? 

We collect 1 billion pieces of data about each human each day. Yet are there even ten useful things per person each day worth knowing? Most of this data is noise, so we use AI to find the relevant bits. However, there can be tremendous inefficiencies if not guided properly. 

What can big data and analytics tell us that no one else can? 

Any new value-added information gleaned by investors from alternative data sources:  

  • Must not (yet) be available from existing sources (banks, brokers) 
  • Must not (yet) be disclosed by the management of the issuing entity (country or corporation) 
  • Must not (yet) be incorporated into the market price 

Conclusion: the majority of new information is likely to be short-lived, of limited individual importance and/or hard to extract. 

Corporations may withhold information from their investors for a number of good reasons:

  • They do not want the competition to know about it. 
  • The information may have the effect of transferring value between stocks and bonds. 
  • The information is not due to be revealed yet.
  • Management is not aligned with other stakeholders 
  • The information may pertain to what other investors think or are doing.
  • The information may be unknown to both management and to other market participants. 

Conclusion: AI/ML and Big Data initiatives try to ferret out and exploit these hidden elements. 

Recent applications (and successes) in asset management

Alpha generation

Alternative data: identify tactical trading opportunities with more timely information than traditional data sources; more robust modeling of performance drivers at company level.

Incorporating AI/ML to alpha generation requires new skills and processes. You’ll need data engineers, data scientists, quantitative analysts, and professionals with domain knowledge. 

Natural language processing: analysis of transcripts, anomaly detection of SEC filings; prioritization of news flow; identifying trending themes across sectors. There are a number of potential uses for asset managers, including: “scoring” text-based reports by their potential impact on a security, identifying “new” news, and prioritizing news flow for analysts. 

Classification and clustering algorithms: regime classification, identify similarity patterns for prior fundamentals at company level. 

Risk Management 

Cluster analysis with machine learning: identify groups of bonds that behave similarly in thin markets to predict bond liquidity characteristics 

Sentiment analysis using natural language processing: identifying tweets regarding major banks under supervision to predict changes in risks to the banking system by predicting changes to financial ratios and market levels. 

Use case: Bank d’ Italia used sentiment analysis of tweets with unsupervised methods to predict risks to major Italian banks. 

Pattern recognition with unsupervised learning: model validation for regulatory or trading purposes: identify a small anomalous fraction of simulation runs of projected scenarios generated by a stress testing model that are invalid due to data input or other reasons (these can be re-run).

Other applications 

Manager selection: data-driven approaches to better assess manager performance. Is the manager meeting certain benchmarks? 
Trading/execution: large data sets of intra-day activity can help refine trading approaches to optimize execution; street firms using NLP to scrub all trade inquiries regardless of source.

Key challenges in Integrating AI

  • AI techniques transpose human processes; what is hard becomes easy (example: large-scale computations) while what is easy may be hard (example: a leaky faucet). 
  • Enormous amounts of new data, most of which is noise.
  • Standalone AI is easier to implement but its scope is often limited.
  • Augmentation with AI is more broadly applicable but harder to incorporate. 
  • The field of investment management presents special challenges because of the large role of game theory and human behavior in financial markets. 

Implementation Changes 

Challenges in modifying staff and process:

  • Staff skill shortages: fundamental manager staff may lack analytics/data science staff skill sets 
  • Investment process incompatibility: may need major changes to incorporate data and AI/ML techniques
  • Cultural resistance: changes may encounter reluctance/resistance from fundamental research staff and portfolio managers. 

Value hard to prove: value added is hard to quantify when the investment process is partly subjective.

Challenges in using alternative data

  • Uncertain value in using alternative data: value is uncertain and dissipates quickly; signal may already be incorporated; difficult to isolate/measure incremental value. You don’t really know how much value is added by any new set of data, and you don’t know how long it will last. 
  • Data availability, quality and quantity: firm’s ability to suitably organize their internal data is limited; large data sets, particularly news and sentiment come from disparate sources and include substantial noise; most data sets have limited history, making it difficult to test through various market environments. 
  • Speed of change: AI/ML is constantly evolving — expensive to keep up, particularly given uncertain value. Continuous process requiring constant evaluation of best data sets and algorithms. 

Challenges using machine learning techniques

  • Overfitting of models: given larger number of inputs, enormous computing power, and automation of data science platforms, we need to be cautious not to overfit data. 
  • Sampling period selection: out-of-sample tests result in selecting models that work well for a specified out-of-sample period. There may be a bias to overfit the “out-of-sample” period.
  • Econometric time series data issues: economic data comes late, with lags and restatements. 
  • Black box models: many machine-learning models are non-linear and structurally opaque (example: black boxes), casting doubts on output stability, why they work and for how long. 
  • Ethical concerns: model selections may contain biases that contravene the law or company policy.

Tags   •   Finance


Leave a Reply

Your email address will not be published. Required fields are marked *


I do not even know the way I ended up right here, but I believed this post was great.
I don't recognise who you're however certainly you're going to a well-known blogger if you aren't already.


Spot on with this write-up, I absolutely believe that this
web site needs far more attention. I'll probably be back again to read more, thanks for the information!


Just want to say your article is as astonishing. The clearness
in your post is simply excellent and i can assume you're an expert
on this subject. Fine with your permission allow me to grab your RSS feed to keep up to date with forthcoming post.
Thanks a million and please carry on the enjoyable


all the time i used to read smaller posts that also clear their motive, and that is also happening with this
piece of writing which I am reading now.


Good blog you've got here.. It's difficult to find good quality writing like yours nowadays.
I truly appreciate individuals like you! Take care!!

Related Posts

Recent Posts

Comparison of Popular AI Frameworks - Comparison of Popular AI Frameworks   Introduction "A computer program is said to learn from experience E with respect to…
Here’s The Main Reason Why Most AI Projects Fail - How’s your AI project coming along? If training data challenges are getting in the way of your goals, it may…
How AI is Transforming Telehealth - As technology continues to advance, artificial intelligence (AI) has become an everyday reality. And one industry it is rapidly transforming…
The Best States for New Businesses in the AI Space - The Best States for New Businesses in the AI Space The growth of AI businesses is becoming explosive. While the…
How Levatas Teaches Spot New Tricks -   One of the key components to Levatas’ success: partnerships. Customers looking for accurate analog gauge reading and thermal heat…
How AI is Revolutionizing Education -   Artificial intelligence has become increasingly relevant in a number of major industries. We read a lot about how it’s…
Three Amazing Ways AI is Revolutionizing Healthcare - It may not seem like it was too long ago when the idea of artificial intelligence playing a major role…
How 5G is Going to Impact AI in Automation Within Telecom - During this webinar, an industry expert discussed how an automation project comes to life from the initial business problem through…
How Automation Projects Come to Life in Telecom - During this webinar, an industry expert discussed how an automation project comes to life from the initial business problem through…
The Future of AI in Marketing - During this webinar, industry experts discussed where AI in marketing was heading in the future. We’ve included a short transcription…

Popular Posts

Does Healthcare AI Meet Basic Ethics Principles? - Ingrid Vasiliu-Feltes Chief Quality and Innovation Officer MEDNAX, Health Solutions Partner Over the past decade we have noticed an exponential…
Machine Learning and Artificial Intelligence in Banking - Artit "Art" Wangperawong Distinguished Engineer US Bank Introduction Every company’s AI journey is different. We’re all trying to figure out…
Machine Learning for Pricing and Inventory Optimization @ Macy’s - Jolene Mork Senior Data Scientist Macy's Iain Stitt Data Scientist Macy's Bhagyesh Phanse VP, Data Science Macy's Overview In this…
Artificial Intelligence & Cybersecurity: Math Not Magic - Wayne Chung CTO FBI Introduction The field of cybersecurity has slowly progressed from an art to a science. It has…
AI/ML in Investment and Risk Management: Recent Applications, Use Cases, and Implementation Challenges - Arvind Rajan Managing Director - Head of Global & Macro PGIM Fixed Income Introduction Investing is a completely different ballgame…
Top AI Conferences - Interested in learning the latest in AI this year? We’ve compiled a list of the top artificial intelligence conferences in…
Machine Learning in Production: From Research to the Customer - Ameen Kazerouni Lead Data Scientist Zappos Overview In this presentation Ameen Kazerouni, the Lead Data Scientist at Zappos, walks through…
How COVID-19 is Impacting the State of AI in Banking - On this panel, industry experts (listed above) discussed The State of AI in Banking and how COVID-19 is affecting it.…
“Ask Me Anything” with Zappos’s Head of AI/ML Research & Platforms, Ameen Kazerouni - Ameen Kazerouni Head of AI/ML Research & Platforms Zappos Family of Companies Ai4 recently hosted an "Ask Me Anything" session…
The Autonomous Pharmacy: Applying AI and ML to Medication Management Across the Care Continuum - Ken Perez VP of Healthcare Policy Omnicell, Inc. Ken applies artificial intelligence (AI) and machine learning (ML) solutions to medication…