Skip to content

End to End Financial Fraud Detection #947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

This file was deleted.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -19,23 +19,23 @@
"id": "abce482e-7537-4857-90e3-af02e665c28e",
"metadata": {},
"source": [
"<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Introduction</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
"<p style = 'font-size:20px;font-family:Arial'><b>Introduction</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial'>\n",
" In recent years we have seen a massive increase in Fraud attempts, making fraud detection imperative for Banking and Financial Institutions. Despite countless efforts and human supervision, hundreds of millions of dollars are lost due to fraud. Fraud can happen using various methods, i.e., stolen credit cards, misleading accounting, phishing emails, etc. Due to small cases in significant populations, fraud detection has become more and more challenging. \n",
" <br>\n",
" <br>\n",
" With ClearScape Analytics, data scientists can use their preferred language, tools and platform to develop models to identify this fraud. Even in large scale operations, users have the guarantee that Vantage can scale to their needs and reduce fraud.</p>\n",
" \n",
"<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Business Values</b></p>\n",
"<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
"<p style = 'font-size:18px;font-family:Arial'><b>Business Values</b></p>\n",
"<ul style = 'font-size:16px;font-family:Arial'>\n",
" <li>Identification of financial fraud in multiple accounts</li>\n",
" <li>Pattern recognition of fraudulent versus normal transactions</li>\n",
" <li>Reduction of money lost due to recovering fraudulent charges</li>\n",
" <li>Improved customer satisfaction and reduction of customer churn</li>\n",
"</ul>\n",
"\n",
"<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Why Vantage?</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>To maximize the business value of advanced analytic techniques including Machine Learning and Artificial Intelligence, it is estimated that organizations must scale their model development and deployment pipelines to 100s or 1000s of times greater amounts of data, models, or both.\n",
"<p style = 'font-size:18px;font-family:Arial'><b>Why Vantage?</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial'>To maximize the business value of advanced analytic techniques including Machine Learning and Artificial Intelligence, it is estimated that organizations must scale their model development and deployment pipelines to 100s or 1000s of times greater amounts of data, models, or both.\n",
" <br>\n",
" <br>\n",
" ClearScape Analytics provides powerful, flexible end-to-end data connectivity, feature engineering, model training, evaluation, and operational functions that can be deployed at scale as enterprise data assets; treating the products of ML and AI as first-class analytic processes in the enterprise.</p>"
Expand All @@ -46,20 +46,20 @@
"id": "5d505907-9eab-4150-b1ca-05a967405181",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
"<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Approach</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
"<hr style=\"height:2px;border:none;\">\n",
"<p style = 'font-size:20px;font-family:Arial'><b>Approach</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial'>\n",
" With ClearScape Analytics, data scientists can use various approaches to develop models and identify this fraud. Even in large scale operations, users have the guarantee that Vantage can scale to their needs and reduce fraud. Here, we are showcasing two different approaches using Vantage</p>\n",
"<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
"<ul style = 'font-size:16px;font-family:Arial'>\n",
" <li><b>Traditional Approach</b></li>\n",
" <li><b>AutoML Approach</b></li>\n",
"</ul></p>\n",
"\n",
"<center><img src=\"images/TraditionalvsAutoml.png\" alt=\"efs\" width=500 height=500></center>\n",
"\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>ClearScape Analytics provides powerful, flexible end-to-end data connectivity, feature engineering, model training, evaluation, and operational functions that can be deployed at scale as enterprise data assets; treating the products of ML and AI as first-class analytic processes in the enterprise.</p>\n",
"<p style = 'font-size:16px;font-family:Arial'>ClearScape Analytics provides powerful, flexible end-to-end data connectivity, feature engineering, model training, evaluation, and operational functions that can be deployed at scale as enterprise data assets; treating the products of ML and AI as first-class analytic processes in the enterprise.</p>\n",
"\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We have created two different notebooks for these two approaches. Based on the choice we can click one of the buttons below, at the end of the table, and execute the steps involved in that approach.</p> "
"<p style = 'font-size:16px;font-family:Arial'>We have created two different notebooks for these two approaches. Based on the choice we can click one of the buttons below, at the end of the table, and execute the steps involved in that approach.</p> "
]
},
{
Expand All @@ -69,27 +69,27 @@
"source": [
"<table style = 'width:100%;table-layout:fixed; border: 1px solid black; border-collapse: collapse;'>\n",
" <tr style = 'border: 1px solid black; border-collapse: collapse;'>\n",
" <th style = 'font-size:24px;font-family:Arial;color:#00233C;border: 1px solid black; border-collapse: collapse;'>Traditional Approach</th>\n",
" <th style = 'font-size:24px;font-family:Arial;color:#00233C'>AutoML Approach</th>\n",
" <th style = 'font-size:24px;font-family:Arial;border: 1px solid black; border-collapse: collapse;'>Traditional Approach</th>\n",
" <th style = 'font-size:24px;font-family:Arial'>AutoML Approach</th>\n",
" </tr>\n",
" <tr style = 'border: 1px solid black; border-collapse: collapse;'>\n",
" <td style = 'border: 1px solid black; border-collapse: collapse;'><img src = 'images/TraditionalApproach4.png' ></td>\n",
" <td style = 'border: 1px solid black; border-collapse: collapse;'><img src = 'images/AutoMLApproach3.png'></td>\n",
" </tr>\n",
" <tr style = 'border: 1px solid black; border-collapse: collapse;'>\n",
" <td style = 'vertical-align:top;width:50%;border: 1px solid black;border-collapse: collapse;'>\n",
" <p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>ClearScape Analytics</b> has more than <code>150+</code> in-database functions, which can be used by business analysts as well as data scientists. These include functions for <code>feature exploration</code>, <code>feature engineering</code>, <code>data preparation</code>, <code>model training</code> and <code>model evaluation.</code> These functions help to solve large-scale complex use-cases such as dynamic pricing, forecasting, fraud detection, predictive maintenance and much more. Because these functions are built into the database, they are highly performant and require limited data movement. This can represent significant cost and friction reduction for our customers. </p></td> \n",
" <p style = 'font-size:16px;font-family:Arial'><b>ClearScape Analytics</b> has more than <code>150+</code> in-database functions, which can be used by business analysts as well as data scientists. These include functions for <code>feature exploration</code>, <code>feature engineering</code>, <code>data preparation</code>, <code>model training</code> and <code>model evaluation.</code> These functions help to solve large-scale complex use-cases such as dynamic pricing, forecasting, fraud detection, predictive maintenance and much more. Because these functions are built into the database, they are highly performant and require limited data movement. This can represent significant cost and friction reduction for our customers. </p></td> \n",
" <td style = 'vertical-align:top;width:50%;border: 1px solid black;border-collapse: collapse;'>\n",
" <p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>AutoML</b>(<b><u>Auto</u></b>mated <b><u>M</u></b>achine <b><u>L</u></b>earning) is a comprehensive approach that automates the end-to-end process of applying machine learning to real-world problems. This functionality automates the entire process of developing a predictive model. It will perform <code>feature exploration</code>, <code>feature engineering</code>, <code>data preparation</code>, <code>model training</code> and <code>model evaluation</code> on dataset in <b>auto run</b> and at end we will get <code>leaderboard</code> containined different models along with their performance. Model will also have <code>rank</code> associated with them which indicates which is <code>best performing model</code> for given data followed by other models.</p></td>\n",
" <p style = 'font-size:16px;font-family:Arial'><b>AutoML</b>(<b><u>Auto</u></b>mated <b><u>M</u></b>achine <b><u>L</u></b>earning) is a comprehensive approach that automates the end-to-end process of applying machine learning to real-world problems. This functionality automates the entire process of developing a predictive model. It will perform <code>feature exploration</code>, <code>feature engineering</code>, <code>data preparation</code>, <code>model training</code> and <code>model evaluation</code> on dataset in <b>auto run</b> and at end we will get <code>leaderboard</code> containined different models along with their performance. Model will also have <code>rank</code> associated with them which indicates which is <code>best performing model</code> for given data followed by other models.</p></td>\n",
" </tr>\n",
" <tr style = 'border: 1px solid black; border-collapse: collapse;'>\n",
" <td style = 'vertical-align:top;width:50%;border: 1px solid black;border-collapse: collapse;'>\n",
" <p style = 'font-size:16px;font-family:Arial;color:#00233C'>Launch Traditional Approach from this button below:</p>&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;<a href=\"Financial_Fraud_Detection_Traditional_Approach.ipynb\" style=\"display: inline-flex; align-items: center; justify-content: center; background-color: #017373; color: #FFFFFF; font-family: Arial, sans-serif; font-size: 16px; font-weight: bold; text-decoration: none; padding: 12px 24px; border: none; border-radius: 8px; box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); cursor: pointer; transition: all 0.3s ease;\">\n",
" <p style = 'font-size:16px;font-family:Arial'>Launch Traditional Approach from this button below:</p>&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;<a href=\"Financial_Fraud_Detection_Traditional_Approach.ipynb\" style=\"display: inline-flex; align-items: center; justify-content: center; background-color: #017373; color: #FFFFFF; font-family: Arial, sans-serif; font-size: 16px; font-weight: bold; text-decoration: none; padding: 12px 24px; border: none; border-radius: 8px; box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); cursor: pointer; transition: all 0.3s ease;\">\n",
" Traditional Approach\n",
" <img src=\"https://img.icons8.com/ios-filled/50/ffffff/external-link.png\" alt=\"External Link Icon\" style=\"margin-left: 8px; width: 20px; height: 20px;\"></a> \n",
" </td>\n",
" <td style = 'vertical-align:top;width:50%;border: 1px solid black;border-collapse: collapse;'>\n",
" <p style = 'font-size:16px;font-family:Arial;color:#00233C'>Launch AutoML Approach from this button below:</p>\n",
" <p style = 'font-size:16px;font-family:Arial'>Launch AutoML Approach from this button below:</p>\n",
" &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp;<a href=\"Financial_Fraud_Detection_AutoML_Approach.ipynb\" style=\"display: inline-flex; align-items: center; justify-content: center; background-color: #017373; color: #FFFFFF; font-family: Arial, sans-serif; font-size: 16px; font-weight: bold; text-decoration: none; padding: 12px 24px; border: none; border-radius: 8px; box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1); cursor: pointer; transition: all 0.3s ease;\">\n",
" AutoML Approach\n",
" <img src=\"https://img.icons8.com/ios-filled/50/ffffff/external-link.png\" alt=\"External Link Icon\" style=\"margin-left: 8px; width: 20px; height: 20px; align: right\">\n",
Expand All @@ -104,7 +104,7 @@
"id": "ba536b37-9b21-47ef-8efa-6a3e388b9df1",
"metadata": {},
"source": [
"<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
"<footer style=\"padding-bottom:35px; border-bottom:3px solid #91A0Ab\">\n",
" <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
" <div style=\"float:right;\">\n",
" <div style=\"float:left; margin-top:14px\">\n",
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.