notebooks/learnability.html

<!DOCTYPE html>
<html lang="en">

<head>
    <title>Arghya Dutta</title>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <link rel="apple-touch-icon" sizes="180x180" href="../assets/favicon_io/apple-touch-icon.png">
    <link rel="icon" type="image/png" sizes="32x32" href="../assets/favicon_io/favicon-32x32.png">
    <link rel="icon" type="image/png" sizes="16x16" href="../assets/favicon_io/favicon-16x16.png">
    <link rel="manifest" href="../assets/favicon_io/site.webmanifest">
    <link rel="stylesheet" href="../assets/style.css">
    <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"> </script>
    <script>window.MathJax = { tex: { inlineMath: [['$', '$'], ['\\(', '\\)']] } };</script>
</head>
<em><a href="../notebooks.html">Notebooks</a></em>

<h1>Learnability in Machine Learning</h1>


<dl>
    <dt> Input : $\mathbf{x}$ (customer application)</dt>
    <dt> Output : $y$ (good/bad customer)</dt>
    <dt> Target function : $f:\mathcal{X}\rightarrow\mathcal{Y}$ (ideal credit approval formula)</dt>
    <dt> Data : $(\mathbf{x}_1, y_1),(\mathbf{x}_2, y_2),\cdots,(\mathbf{x}_N, y_N)$ (historical records of credit customers)</dt>
    <dt> Hypothesis set : $\mathcal{H}={h}$. It plays a pivotal role. It can be a linear regression, a neural network, a support vector machine…</dt>
    <dt> Hypothesis : $g:\mathcal{X}\rightarrow\mathcal{Y}$, $g \in \mathcal{H}$. <em>We hope that $g$ approximates $f$ well, that is the goal of learning.</em></dt>
    <dt> Learning algorithm: $\mathcal{A}$ (e.g. back-propagation for neural network.) It does the searching and produces $g$.</dt>
    <dt> The hypothesis set and the learning algorithm $(\mathcal{H}, \mathcal{A})$ together are known as the **learning model**.</dt>
    <dt> <em>We don't know $f$, we can only guess what it is from the data. The learning algorithm picks $g\simeq f$ from the hypothesis set $\mathcal{H}$.</em></dt>
</dl>

<h2>Online Resources</h2>

<dl>
    <dt>
        <a href="https://www.youtube.com/playlist?list=PLD63A284B7615313A">Machine learning course</a> by Yaser Abu-Mostafa.
    </dt>
    <dd>
        Especially useful for its introductory discussion on learnability and VC dimension.
    </dd>
</dl>


</body>

</html>