hi, I am Thomas Toller I am the US Director of Exactpro Systems
Exactpro have tested market surveillance systems
at stock exchanges, banks and leading financial market infrastructure providers worldwide
in this video we explore
how market surveillance systems are tested today and
how effective they are
then, we will cover the challenges of testing
these systems and explore Exactpro’s approach
those who will benefit from watching would
include Compliance professionals, program and project managers
and product owners of market surveillance systems
how are market surveillance systems tested today?
testing is frequently performed by members of the compliance or market control functions
themselves valuable business resources
because testing requires a comprehensive understanding of the surveillance monitoring function
how are market surveillance systems today effective?
despite the widespread adoption of market surveillance systems
to detect and deter market abuse and manipulation
we continue to see enforcement actions brought by the regulators
in January 2020, the CFTC ordered an Australian company
to pay $1 million for spoofing that took place over a period of at least 5 years
and in October 2019, the SEC charged 18 traders in a $31 million stock manipulation scheme
that spanned 6 years from 2013 to the SEC compliant in 2019
while market abuse is being prosecuted
it has often been perpetrated over a long period of time before detection
so, what are the issues with the current deployments of market surveillance systems?
simply deploying a system does not absolve a firm of its regulatory responsibility
the system needs to detect the abusive and manipulative activity and firms need to act on the alerts
the UK FCA in the a September 2018 Market Watch noted that they
“continued to observe firms with vendor-supplied systems using ‘out of the box’ and ‘industry
standard’ settings to calibrate their alert parameters”
and they further noted that they: “observed firms using average peer alert
volumes as a measure of the appropriateness of their calibration”
Exactpro has over 10 years of experience in testing exchange, CCP’s, CSD’s
and other related systems
and has taken that experience and expertise and has now applied this to
testing market surveillance systems
as with all complex systems, the best way to know your system
is working the way you want it to and not in ways you don’t
is to test it thoroughly
what are the challenges when testing market surveillance systems?
market surveillance systems are complex systems
that may include a set of interconnected subsystems
including realtime or static trading data, market data
and other structured and unstructured data
complex processing, analytics and a tool to analyze and display data and alerts
while the adoption of market surveillance systems is now widespread
proper implementation, ongoing testing and calibration of these systems
is a necessary step in making sure they are achieving the underlying goal of the regulations
that have driven their adoption which may include
enhancing market integrity and investor protection
detecting and deterring the use of, or attempted employment of manipulative or deceptive devices
price manipulation
or trading on the basis of nonmaterial public information
Exactpro Test and Automation Approaches
today we will highlight the challenges and the complexity of testing market surveillance systems
connected to trading platforms, market data providers
involving various data mining processes, alerting mechanisms
and having different degrees of process distribution complexity
Exactpro has tested a number of market surveillance systems across different markets and locations
our expertise and experience include
building and maintaining of test and regression libraries covering system platforms end-to-end
automating the testing process using proven tools and methodologies
building up a holistic approach encompassing the full cycle of market surveillance systems testing
based on our experience of testing the world’s leading Trading, Exchanges,
and Clearing platforms
market surveillance systems are typically
interconnected with numerous trading platforms market data feeds and news feeds
and the introduction or upgrade of any of those systems can seriously impact the effectiveness of
the market surveillance platform
what are the benefits of testing and optimization?
the optimization of market surveillance systems ensures that false positives and errors are minimized
meaning the monitoring team is able to be more efficient and, hence, are able to
mitigate the costs and effort of the compliance team in testing
ensure market regulatory requirements are met and documented
what are the challenges when testing a market surveillance system?
market surveillance systems are complex systems
that may include a set of subsystems including: gateways that may connect to other systems
like trading systems
market data providers, trade reporting systems
reference data systems, unstructured data
3rd-party data vendors: FX rates and regulatory news announcements
it may also include a data enrichment module to produce internally calculated statistics
and may also include an analytics module
real-time and offline alert engines to detect
abusive market behavior and generate rules-based alerts and may also incorporate machine learning
and data mining
data repositories to store structured and unstructured data
these systems are then generally linked to
a GUI that displays alerts, real-time monitoring dashboard and market replays that can integrate
3rd party dashboards
how does Exactpro approach this testing?
Exactpro focuses on automated testing and builds tools to support this approach
Sailfish is a tool used for end to end functional testing of the whole message flow in real-time
starting with the injection of data in the upstream systems like the FIX gateway and
then receiving messages from the market surveillance systems for comparison of the received message
against the expected response
where a system does not support the real-time
approach, a tool called Shsha is used to compare output from the market surveillance system
with the previously verified data output
a data consistency test is also performed
after each operational cycle of the system to compare messages from different end-points
of the market surveillance system
Alerts Testing
one of the main purposes of market surveillance systems is the prevention and investigation
of abusive, manipulative or illegal trading behaviour in the markets
there are a set of pre-configured patterns with known abusive behavior that help streamline
regulatory compliance in this area
market surveillance systems also contain modules
that allow users to create new patterns – or adjust the existing ones – and integrate
them into the system
some of the most common types of patterns include
Layering
Spoofing Marking The Close
Wash Trading And front Running
Exactpro performs the testing of the market surveillance system patterns to ensure that
the actual results match the business requirements
the test stages here are the same as in employing
the automated testing approach using the Sailfish test tool and test scenarios designed based
on the business requirements provided
Alerts Testing - Production Data Replay
but trust no one!
the requirements may be wrong, or may not fully meet ongoing behaviour
on the market and, as a result, the alert may behave abnormally
the most common samples
of such anomalies are: pattern thresholds are not calibrated
benchmarks that may be wrong false positive alerts are triggered – here
we need to look at adjusting the parameters or we are missing something - false negatives
all of the above anomalies are common, especially when a new platform is designed from scratch
so, what is the Exactpro approach here? we look at this as a Data Science challenge
we use data mining techniques to ensure that our analysts can see the number of alerts
triggered with the updated pattern based on the production or synthetic data, and then
we manage the quality of these alerts
The Next Generation of Market Surveillance Systems
realizing the advantages of cloud computing services providers like
AWS, Azure and others the financial industry and firms are now actively
migrating their solutions – including market surveillance systems – into the cloud
Exactpro test harness tools are cloud-ready and can help clients to make the migration
as smooth as possible
the financial industry is also actively leveraging
machine learning and artificial intelligence techniques
an increasing number of companies tend to apply AI to market surveillance systems for
smart market monitoring prediction
fraud detection and pattern recognition from data based on
machine learning models, instead of rules-based algorithms
so, how can Exactpro help you?
Exactpro can test your current system is performing
as intended and calibrated to your firm's needs by testing and helping you optimize
Exactpro can test the subsystems and data used in your market surveillance systems
Exactpro can test your alerts to confirm the alerts configurations match the firm’s requirements
Exactpro has cloud-ready tools to help clients when they are migrating arket urveillance systems
into the cloud, to make the transition as smooth as possible
if you found this video helpful and have some questions, please don’t hesitate to contact
myself or one of the other Exactpro team at the details below