High Stakes
The stakes for complying with the Bank Secrecy Act (BSA) continue to rise higher and higher. Regulators, including the SEC, are making compliance a priority – anti-money laundering (AML) compliance was listed as a market-wide risk in the agency’s “Examination Priorities for 2017.” Three recent cases where non-compliance was found resulted in substantial fines and even individual liability. In January, Western Union entered into a $586 million settlement agreement with multiple agencies as a result of their failure to maintain an effective AML program. Another example is Banamex USA (a bank subsidiary of Citigroup) who also failed to maintain an effective AML program, resulting in a hefty settlement with the FDIC and California Department of Business Oversight of $140 million in 2015. Foreign companies aren’t exempt from failures either; in January 2017, Deutsche Bank was fined $41 million for violations by the FRB.
Why is Model Validation Important?
The FRB, OCC and FDIC all agree that the quality of any BSA/AML model output must be evaluated to verify the model’s accuracy. Reliance is placed on the BSA/AML model every time a transaction is processed, but the BSA/AML monitoring department is not on the front line dealing with customers face-to-face. Instead, the monitoring department sits in the back office analyzing the outputs of the model for potential illicit behavior that may be happening out front. No matter the size of the institution or level of sophistication of the model, if the model isn’t functioning correctly, the results will be inadequate. As noted above, regulators have honed in on issues related to the failure to maintain an effective program and the failure to identify unlawful activity. Without validating the model, your BSA/AML compliance program cannot be deemed effective because you have no idea of the quality of output.
What’s the Validation Process?
OCC Bulletin 2011-12 (Supervisory Guidance on Model Risk Management) explains that model validation should be performed by someone independent from the development of the model and daily users. While regulatory guidance does not require the model validation to be performed by a specific party (in-house or outsourced), the validation should be conducted by someone with significant expertise and the authority to challenge the model development and specifications.
Back-testing (reverse-engineering the output of your model using a separate process/software) is the most frequently used method of performing model validation. To do this, the parameters, or rules, set up in the institution’s BSA/AML model to determine which transactional activity results in output, or “hits,” are extracted from the model. A sample period of transactional data is also extracted from the source, for example, a core application system dump. The party performing the validation then utilizes separate software with criteria set to mimic the rules of the model. The separate software is used to process the transactional data to determine what “hits” result. The validator then compares the results to the model results and investigates discrepancies. The results are then utilized to fine-tune and recalibrate the model.
One and Done?
Validation of the model is not a one-time process. The model should be validated on a periodic basis to proactively respond to technological advancements and to incorporate new product offerings by the company. OCC 2011-12 specifically states that the model should be validated at least annually as well as when material changes occur to the model. Keep in mind validation does not have to occur on the back-end; if a new product is being offered, sample transactions can be validated to ensure correct outputs from the model.
Benefits of Validation
Validating your BSA/AML model has more benefits than just crossing an item off your regulatory compliance to-do list. These can include:
- Increasing your confidence in the model’s outputs – eliminating garbage,
- Identifying your model’s limitations to determine necessary mitigating controls,
- Allowing for a proactive approach to technological advancements and new product offerings, and
- Reduction in risk of misdetection of illicit activity.
While full risk cannot be eliminated, validation of your BSA/AML model can help reduce your risk and identify model limitations and loopholes. However, technology still cannot replace human instincts. Promoting an institution-wide culture of following your gut and speaking up, even if the model doesn’t detect suspicious activity, will also help to reduce your institution’s compliance risk.
