Machine Learning
Contents I. Introduction
Objectives
Related Fields 1. Statistics
Literature Machine Learning:
Software Programming:
Software Statistics:
Software Lab Class Setup
Software Lab Class Setup
Chapter ML:I I. Introduction
Examples of Learning Tasks Car Shopping Guide
Examples of Learning Tasks Risk Analysis for Credit Approval
Examples of Learning Tasks Risk Analysis for Credit Approval
Examples of Learning Tasks Image Analysis
Examples of Learning Tasks Image Analysis
Specification of Learning Tasks Definition 1 (Machine Learning
Remarks:
Specification of Learning Tasks Learning Paradigms
Specification of Learning Tasks Learning Paradigms
Specification of Learning Tasks Example Chess: Kinds of Experience
Specification of Learning Tasks Example Chess: Kinds of Experience
Specification of Learning Tasks Example Chess: Kinds of Experience
Specification of Learning Tasks Example Chess: Ideal Target Function γ(o)
Specification of Learning Tasks Example Chess: Ideal Target Function γ(o)
Specification of Learning Tasks Example Chess: Real World → Model World
Specification of Learning Tasks Example Chess: Real World → Model World
Specification of Learning Tasks Example Chess: Real World → Model World
Remarks:
Remarks (continued):
Specification of Learning Tasks
Specification of Learning Tasks
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Remarks:
Specification of Learning Tasks The LMS Algorithm for Fitting y(x)
Remarks:
Remarks (continued):
Chapter ML:I I. Introduction
Elements of Machine Learning I. Model Formation: Real World → Model World
Elements of Machine Learning II. Design of Supervised Learning Models
Elements of Machine Learning II. Design of Supervised Learning Models: LMS in a Nutshell
Elements of Machine Learning II. Design of Supervised Learning Models: LMS in a Nutshell
Elements of Machine Learning II. Design of Supervised Learning Models: LMS in a Nutshell
Elements of Machine Learning II. Design of Supervised Learning Models: LMS in a Nutshell
Related questions:
Elements of Machine Learning III. Structure of Feature Spaces
Elements of Machine Learning III. Structure of Feature Spaces
Elements of Machine Learning III. Structure of Feature Spaces
Remarks:
Elements of Machine Learning IV. Discriminative versus Generative Approach to Classification Problems
Elements of Machine Learning IV. Discriminative versus Generative Approach to Classification Problems
Elements of Machine Learning IV. Discriminative versus Generative Approach to Classification Problems
Remarks:
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Remarks:
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Elements of Machine Learning V. Frequentist versus Subjectivist Approach to Parameter Estimation
Remarks:
Chapter ML:I I. Introduction
Comparative Syntax Overview Concept
Classification Approaches Search in hypothesis space
Chapter ML:II II. Machine Learning Basics
Linear Regression Regression versus Classification
Linear Regression Regression versus Classification
Linear Regression The Linear Regression Model
Linear Regression The Linear Regression Model
Remarks:
Remarks (continued) :
Linear Regression One-Dimensional Feature Space
Linear Regression One-Dimensional Feature Space
Linear Regression One-Dimensional Feature Space
Linear Regression One-Dimensional Feature Space
Linear Regression One-Dimensional Feature Space
Linear Regression One-Dimensional Feature Space
Linear Regression One-Dimensional Feature Space
Linear Regression Higher-Dimensional Feature Space
Linear Regression Higher-Dimensional Feature Space
Linear Regression Higher-Dimensional Feature Space
Linear Regression Higher-Dimensional Feature Space
Linear Regression Higher-Dimensional Feature Space
Remarks:
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Remarks: (?) When used as operand in a scalar product with the weight vector, wT x, we consider the
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Linear Regression Linear Regression for Classification
Remarks:
Linear Regression The Linear Model Function: Variants
Linear Regression The Linear Model Function: Variants
Linear Regression The Linear Model Function: Variants
Linear Regression The Linear Model Function: Variants
Linear Regression The Linear Model Function: Properties of the Solution
Linear Regression The Linear Model Function: Properties of the Solution
Linear Regression
Linear Regression RSS(w)
Linear Regression RSS(w)
Remarks:
Chapter ML:II
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space An Exemplary Learning Task
Remarks:
Concept Learning: Search in Hypothesis Space Definition 1 (Concept, Hypothesis, Hypothesis Space)
Concept Learning: Search in Hypothesis Space Definition 1 (Concept, Hypothesis, Hypothesis Space)
Remarks:
Concept Learning: Search in Hypothesis Space Usually, an example set D, D = {(x1, c(x1)), . . . , (xn, c(xn))}, contains positive
Remarks:
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Remarks:
Remarks: (continued)
Remarks: (continued)
Concept Learning: Search in Hypothesis Space Inductive Learning Hypothesis
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Remarks:
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Discussion of the Find-S Algorithm
Concept Learning: Version Space Definition 5 (Version Space)
Concept Learning: Version Space Definition 5 (Version Space)
Remarks:
Concept Learning: Version Space Definition 6 (Boundary Sets of a Version Space)
Concept Learning: Version Space Definition 6 (Boundary Sets of a Version Space)
Concept Learning: Version Space Candidate Elimination Algorithm
Concept Learning: Version Space Candidate Elimination Algorithm
Remarks:
Concept Learning: Version Space Candidate Elimination Algorithm (pseudo code)
Concept Learning: Version Space Candidate Elimination Algorithm (pseudo code)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Discussion of the Candidate Elimination Algorithm
Concept Learning: Version Space Question 1: Selecting Examples from D
Concept Learning: Version Space Question 1: Selecting Examples from D
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Chapter ML:II
Evaluating Effectiveness (1) Misclassification Rate
Evaluating Effectiveness (1) Misclassification Rate
Remarks:
Evaluating Effectiveness (1) Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness (1) Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness (1) Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness (1) Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness (1) Misclassification Rate: Probabilistic Foundation
Remarks:
Evaluating Effectiveness (1) Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness (1) Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness (1) Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness (1) Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness (1) Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness (1) Illustration 1: Generic Feature and Class Distributions
Remarks:
Evaluating Effectiveness (1) Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness (1) Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness (1) Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness (1) Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness (1) Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness (1) Illustration 2: Feature Distribution given Linearly Separable Classes
Remarks:
Evaluating Effectiveness (1) Estimating Error Bounds
Evaluating Effectiveness (1) Estimating Error Bounds
Evaluating Effectiveness (1) Estimating Error Bounds
Remarks: ∗
Evaluating Effectiveness (1) Training Error
Remarks:
Evaluating Effectiveness (1) Holdout Error
Remarks:
Evaluating Effectiveness (1) k-Fold Cross-Validation
Evaluating Effectiveness (1) k-Fold Cross-Validation
Evaluating Effectiveness (1) k-Fold Cross-Validation
Remarks:
Evaluating Effectiveness (1) Comparing Model Variants
Evaluating Effectiveness (1) Model Selection (single validation set)
Evaluating Effectiveness (1) Model Selection (single validation set)
Evaluating Effectiveness (1) Model Selection (single validation set)
Evaluating Effectiveness (1) Model Selection (single validation set)
Evaluating Effectiveness (1) Model Selection (k validation sets)
Evaluating Effectiveness (1) Model Selection (k validation sets)
Evaluating Effectiveness (1) Model Selection (k validation sets)
Remarks:
Evaluating Effectiveness (1) Misclassification Costs
Remarks: ∗
Chapter ML:III III. Linear Models
Logistic Regression Binary Classification Problems
Logistic Regression Binary Classification Problems
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Remarks: (?) Recap.
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Remarks (interpretation of y(x) as probability):
Remarks (derivation of Lσ (w)):
Remarks (derivation of Lσ (w)) (continued):
Logistic Regression Recap. Linear Regression for Classification
Logistic Regression Recap. Linear Regression for Classification
Logistic Regressionn Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Remarks:
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Remarks:
Loss Computation Loss Computation in Logistic Regression
Loss Computation Loss Computation in Logistic Regression
Loss Computation Loss Computation in Logistic Regression
Loss Computation Loss Computation in Logistic Regression
Remarks:
Loss Computation The BGD Algorithm
Remarks:
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Chapter ML:III
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Definition
Overfitting Definition
Overfitting Definition
Overfitting Mitigation Strategies
Overfitting Mitigation Strategies
Regularization Motivation
Regularization Motivation
Regularization Motivation
Regularization Motivation
Regularization Motivation
Regularization Motivation
Regularization Motivation
Regularization Motivation
Regularization Motivation
Remarks:
Regularization Machine Learning as an Inverse Problem
Regularization Machine Learning as an Inverse Problem
Regularization Machine Learning as an Inverse Problem
Regularization Machine Learning as an Inverse Problem
Regularization Machine Learning as an Inverse Problem
Regularization Machine Learning as an Inverse Problem
Regularization Well-Posed vs. Ill-Posed Problems
Regularization Well-Posed vs. Ill-Posed Problems
Remarks:
Regularization Definition
Regularization Definition
Regularization Definition
Regularization Definition
Regularization Definition
Regularization Definition
Remarks:
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration: Lasso Regression
Regularization Illustration: Lasso Regression
Regularization Illustration: Lasso Regression
Regularization Illustration: Lasso Regression
Regularization Illustration: Ridge Regression
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Regularization Illustration
Remarks:
Remarks: (continued)
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression: Hyperparameter λ
Regularization Regularized Linear Regression: Hyperparameter λ
Chapter ML:IV IV. Neural Networks
Perceptron Learning Biological Inspiration
Perceptron Learning Biological Inspiration
Remarks (facts about the human brain):
Perceptron Learning History
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Remarks:
Perceptron Learning Binary Classification Problems
Perceptron Learning The PT Algorithm
Remarks: (?) Recap.
Perceptron Learning Weight Adaptation: Illustration in Input Space
Perceptron Learning Weight Adaptation: Illustration in Input Space
Remarks: ~ = (w1 , . . . , wp )T , and the
Perceptron Learning Example
Perceptron Learning Example
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Perceptron Convergence Theorem
Perceptron Learning Perceptron Convergence Theorem
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Discussion
Perceptron Learning Perceptron Convergence Theorem: Discussion
Chapter ML:IV IV. Neural Networks
Gradient Descent Motivation
Gradient Descent Motivation
Gradient Descent Motivation
Gradient Descent Principle
Gradient Descent Principle
Gradient Descent
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (1) Linear Regression + Squared Loss
Gradient Descent (1) Linear Regression + Squared Loss
Remarks (derivation of ∇L2 (w)) :
Gradient Descent The BGD Algorithm
Remarks: (?) Recap.
Gradient Descent Global Loss versus Pointwise Loss
Remarks:
Gradient Descent The IGD Algorithm
Remarks (IGD) :
Remarks (GD versus PT) (continued) :
Gradient Descent (2) Linear Regression + 0\u002f1 Loss
Gradient Descent (2) Linear Regression + 0\u002f1 Loss
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (3) Logistic Regression + Logistic Loss + Regularization
Gradient Descent (3) Logistic Regression + Logistic Loss + Regularization
Remarks: ~ = (w1 , . . . , wp )T , and the
Remarks (derivation of ∇Lσ (w)) :
Remarks (derivation of ∇Lσ (w)) (continued) : X
Chapter ML:IV
Multilayer Perceptron Definition 1 (Linear Separability)
Multilayer Perceptron Definition 1 (Linear Separability)
Multilayer Perceptron Linear Separability
Multilayer Perceptron Linear Separability
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Remarks:
Remarks (continued) :
Multilayer Perceptron Unrestricted Classification Problems
Multilayer Perceptron Unrestricted Classification Problems: Example
Multilayer Perceptron Unrestricted Classification Problems: Example
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron (1) Forward Propagation
Remarks:
Multilayer Perceptron (1) Forward Propagation
Multilayer Perceptron (1) Forward Propagation
Multilayer Perceptron (1) Forward Propagation: Batch Mode
Multilayer Perceptron (2) Backward Propagation
Multilayer Perceptron (2) Backward Propagation
(2) Backward Propagation
(2) Backward Propagation
(2) Backward Propagation
Multilayer Perceptron (2) Backward Propagation
Multilayer Perceptron (2) Backward Propagation
Multilayer Perceptron (2) Backward Propagation
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Remarks:
Remarks (derivation of ∇o L2 (w)) :
Multilayer Perceptron Momentum Term
Chapter ML:III III. Decision Trees
Decision Trees Basics Classification Problems with Nominal Features
Decision Trees Basics Decision Tree for the Concept “EnjoySport”
Decision Trees Basics Decision Tree for the Concept “EnjoySport”
Decision Trees Basics Definition 1 (Splitting)
Decision Trees Basics Definition 1 (Splitting)
Decision Trees Basics Definition 1 (Splitting)
Decision Trees Basics Definition 1 (Splitting)
Remarks:
Decision Trees Basics Definition 2 (Decision Tree)
Decision Trees Basics Definition 2 (Decision Tree)
Remarks:
Decision Trees Basics Notation
Remarks:
Decision Trees Basics Algorithm Template: Construction
Decision Trees Basics Algorithm Template: Classification
Remarks:
Decision Trees Basics When to Use Decision Trees
Decision Trees Basics On the Construction of Decision Trees
Decision Trees Basics Evaluation of Decision Trees
Decision Trees Basics Evaluation of Decision Trees
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Classification Error
Decision Trees Basics Evaluation of Decision Trees: Classification Error
Decision Trees Basics Evaluation of Decision Trees: Classification Error
Remarks:
Decision Trees Basics Evaluation of Decision Trees: Misclassification Costs
Decision Trees Basics Evaluation of Decision Trees: Misclassification Costs
Decision Trees Basics Evaluation of Decision Trees: Misclassification Costs
Remarks:
Chapter ML:III III. Decision Trees
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Definition 4 (Impurity Function ι)
Impurity Functions Definition 5 (Impurity of an Example Set ι(D))
Impurity Functions Definition 5 (Impurity of an Example Set ι(D))
Remarks:
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Definition 7 (Strict Impurity Function)
Impurity Functions Definition 7 (Strict Impurity Function)
Remarks:
Impurity Functions Impurity Functions Based on Entropy
Remarks:
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Remarks [::::::: Bayes :::
Remarks (continued) :
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Chapter ML:III III. Decision Trees
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Remarks:
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Remarks:
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Hypothesis Space
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Remarks:
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Chapter ML:III III. Decision Trees
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Remarks:
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Remarks:
Decision Tree Pruning Overfitting
Decision Tree Pruning (a) Stopping
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning Extensions
Chapter ML:IV IV. Statistical Learning
Probability Basics Area Overview
Probability Basics Definition 1 (Random Experiment, Random Observation)
Remarks:
Probability Basics Definition 2 (Sample Space, Event Space)
Probability Basics Definition 3 (Important Event Types)
Probability Basics Classical Concept Formation
Probability Basics Classical Concept Formation
Remarks:
Probability Basics Axiomatic Concept Formation
Probability Basics Axiomatic Concept Formation
Probability Basics Axiomatic Concept Formation
Probability Basics Axiomatic Concept Formation
Probability Basics Axiomatic Concept Formation
Remarks:
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Remarks:
Remarks (continued) :
Probability Basics Independence of Events
Probability Basics Independence of Events
Chapter ML:IV
Bayes Classification Single Conditional Event
Bayes Classification Single Conditional Event
Bayes Classification Single Conditional Event
Bayes Classification Combined Conditional Events
Bayes Classification Combined Conditional Events
Remarks [:::::::::::: Information::::::
Remarks (continued) :
Bayes Classification Naive Bayes
Bayes Classification Naive Bayes
Remarks:
Remarks (continued) :
Bayes Classification Naive Bayes
Bayes Classification Naive Bayes
Bayes Classification Naive Bayes
Remarks:
Bayes Classification Naive Bayes: Classifier Construction Summary
Bayes Classification Naive Bayes: Classifier Construction Summary
Remarks:
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Kapitel ML:IX IX. Ensemble and Meta
Ensemble Methods Motivation (1): Generalisierungsfähigkeit
Ensemble Methods Motivation (2): No Free Lunch Theorem
Ensemble Methods Motivation (3): Instabilität von Lernverfahren
Ensemble Methods Motivation (4): Consequences
Ensemble Methods Bootstrap Aggregating (Bagging)
Remarks:
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Boosting Weak Classifiers
Ensemble Methods Adaptive Boosting (AdaBoost)
Ensemble Methods Adaptive Boosting
Ensemble Methods Adaptive Boosting
Remarks:
Ensemble Methods Discrete AdaBoost
Ensemble Methods Cascades of Classifiers (Cascading)
Ensemble Methods Cascading
Ensemble Methods Maßzahlen für Cascading
Ensemble Methods Maßzahlen für Cascading
Ensemble Methods Cascading
Ensemble Methods Cascading for Face Detection (Main Loop)
Ensemble Methods Cascading for Face Detection (Gesamtklassifikator)
Ensemble Methods Cascading for Face Detection(Einzelklassifikator in Kaskade)
Ensemble Methods Cascading for Face Detection (Einzelklassifikator in Ensemble)
Ensemble Methods Cascading for Face Detection (alle Features)
Ensemble Methods Cascading for Face Detection (ausgewählte Features)
Ensemble Methods Cascading for Face Detection (Effizienz)
Ensemble Methods Cascading for Face Detection (Effizienz beim Training)
Ensemble Methods Trainingsmengen
Ensemble Methods Grundlage der Entscheidung
Ensemble Methods Literature