Machine Learning
Contents I. Introduction
Objectives
Related Fields 1. Statistics
Literature Machine Learning:
Software Programming:
Software Statistics:
Software Lab Class Setup
Software Lab Class Setup
Chapter ML:I I. Introduction
Examples of Learning Tasks (1) Car Shopping Guide
Examples of Learning Tasks (2) Risk Analysis for Credit Approval
Examples of Learning Tasks (2) Risk Analysis for Credit Approval
Examples of Learning Tasks (3) Image Analysis
Examples of Learning Tasks (3) Image Analysis
Specification of Learning Tasks Definition 1 (Machine Learning
Remarks:
Specification of Learning Tasks Learning Paradigms
Specification of Learning Tasks Learning Paradigms
Specification of Learning Tasks (4) Example Chess: Kinds of Experience
Specification of Learning Tasks (4) Example Chess: Kinds of Experience
Specification of Learning Tasks (4) Example Chess: Kinds of Experience
Specification of Learning Tasks (4) Example Chess: Ideal Target Function γ(o)
Specification of Learning Tasks (4) Example Chess: Ideal Target Function γ(o)
Specification of Learning Tasks (4) Example Chess: Real World → Model World
Specification of Learning Tasks (4) Example Chess: Real World → Model World
Specification of Learning Tasks (4) Example Chess: Real World → Model World
Remarks:
Remarks (continued):
Specification of Learning Tasks
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Specification of Learning Tasks Real World → Model World
Remarks:
Remarks (continued) :
Specification of Learning Tasks The LMS Algorithm for Fitting y(x)
Remarks:
Remarks (continued):
Chapter ML:I I. Introduction
Elements of Machine Learning (1) Model Formation: Real World → Model World
Elements of Machine Learning (2) Design Choices for Model Function Construction
Elements of Machine Learning (2) Design Choices for Model Function Construction: LMS in a Nutshell
Elements of Machine Learning (2) Design Choices for Model Function Construction: LMS in a Nutshell
Elements of Machine Learning (2) Design Choices for Model Function Construction: LMS in a Nutshell
Elements of Machine Learning (2) Design Choices for Model Function Construction: LMS in a Nutshell
Related questions:
Elements of Machine Learning (3) Feature Space Structure
Elements of Machine Learning (3) Feature Space Structure
Elements of Machine Learning (3) Feature Space Structure
Remarks:
Elements of Machine Learning (4) Discriminative vs. Generative Approach to Classification
Elements of Machine Learning (4) Discriminative vs. Generative Approach to Classification
Elements of Machine Learning (4) Discriminative vs. Generative Approach to Classification
Remarks:
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Remarks:
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Elements of Machine Learning (5) Frequentist vs. Subjectivist Approach to Parameter Estimation
Remarks:
Chapter ML:I I. Introduction
Comparative Syntax Overview Bishop 2006
Functions Overview Function name
Algorithms Overview Algorithm name
Classification Approaches Overview Search in hypothesis space
Chapter ML:II
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space An Exemplary Learning Task
Remarks:
Concept Learning: Search in Hypothesis Space Definition 1 (Concept, Hypothesis, Hypothesis Space)
Concept Learning: Search in Hypothesis Space Definition 1 (Concept, Hypothesis, Hypothesis Space)
Remarks:
Concept Learning: Search in Hypothesis Space The example set D, D = {(x1, c(x1)), . . . , (xn, c(xn))}, contains usually both positive
Remarks:
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Simple Classification Problems
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Concept Learning: Search in Hypothesis Space Order of Hypotheses
Remarks:
Remarks: (continued)
Remarks: (continued)
Concept Learning: Search in Hypothesis Space Inductive Learning Hypothesis
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Remarks:
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Find-S Algorithm
Concept Learning: Search in Hypothesis Space Discussion of the Find-S Algorithm
Concept Learning: Version Space Definition 5 (Version Space)
Concept Learning: Version Space Definition 5 (Version Space)
Remarks:
Concept Learning: Version Space Definition 6 (Boundary Sets of a Version Space)
Concept Learning: Version Space Theorem 7 (Version Space Representation)
Concept Learning: Version Space Candidate Elimination Algorithm
Concept Learning: Version Space Candidate Elimination Algorithm
Remarks:
Concept Learning: Version Space Candidate Elimination Algorithm (pseudo code)
Concept Learning: Version Space Candidate Elimination Algorithm (pseudo code)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Candidate Elimination Algorithm (illustration)
Concept Learning: Version Space Discussion of the Candidate Elimination Algorithm
Concept Learning: Version Space Question 1: Selecting Examples from D
Concept Learning: Version Space Question 1: Selecting Examples from D
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 2: Partially Learned Concepts
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Concept Learning: Version Space Question 3: Inductive Bias
Chapter ML:II II. Machine Learning Basics
From Regression to Classification Regression versus Classification
From Regression to Classification Regression versus Classification
From Regression to Classification Regression versus Classification
From Regression to Classification The Linear Regression Model
From Regression to Classification The Linear Regression Model
Remarks (residuals):
Remarks (randomness and distributions):
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification One-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
From Regression to Classification Higher-Dimensional Feature Space
Remarks:
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
Remarks: (?) We consider the feature vector x in its extended form when used as operand in a scalar
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
From Regression to Classification Linear Regression for Classification
Remarks:
From Regression to Classification Linear Model Function: Variants
From Regression to Classification Linear Model Function: Variants
From Regression to Classification Linear Model Function: Variants
From Regression to Classification Linear Model Function: Variants
From Regression to Classification Linear Model Function: Properties of the Solution
From Regression to Classification Linear Model Function: Properties of the Solution
Remarks:
From Regression to Classification
From Regression to Classification RSS(w)
From Regression to Classification RSS(w)
Remarks:
Chapter ML:II
Evaluating Effectiveness Misclassification Rate
Evaluating Effectiveness Misclassification Rate
Remarks:
Evaluating Effectiveness Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness Misclassification Rate: Probabilistic Foundation
Evaluating Effectiveness Misclassification Rate: Probabilistic Foundation
Remarks:
Remarks: (continued)
Evaluating Effectiveness Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness Illustration 1: Generic Feature and Class Distributions
Evaluating Effectiveness Illustration 1: Generic Feature and Class Distributions
Remarks:
Evaluating Effectiveness Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness Illustration 2: Feature Distribution given Linearly Separable Classes
Evaluating Effectiveness Illustration 2: Feature Distribution given Linearly Separable Classes
Remarks:
Evaluating Effectiveness Estimating Error Bounds
Evaluating Effectiveness Estimating Error Bounds
Evaluating Effectiveness Estimating Error Bounds
Remarks: ∗
Evaluating Effectiveness Training Error
Remarks:
Evaluating Effectiveness Holdout Error
Remarks:
Evaluating Effectiveness k-Fold Cross-Validation
Evaluating Effectiveness k-Fold Cross-Validation
Evaluating Effectiveness k-Fold Cross-Validation
Remarks:
Evaluating Effectiveness Comparing Model Variants
Remarks:
Evaluating Effectiveness Model Selection (single validation set)
Evaluating Effectiveness Model Selection (single validation set)
Evaluating Effectiveness Model Selection (single validation set)
Evaluating Effectiveness Model Selection (single validation set)
Evaluating Effectiveness Model Selection (k validation sets)
Evaluating Effectiveness Model Selection (k validation sets)
Evaluating Effectiveness Model Selection (k validation sets)
Remarks:
Evaluating Effectiveness Misclassification Costs
Remarks: ∗
Chapter ML:III III. Linear Models
Logistic Regression Binary Classification Problems
Logistic Regression Binary Classification Problems
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Logistic Regression Linear Regression
Remarks: (?) Recap.
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Sigmoid (Logistic) Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Logistic Regression Interpretation of the Logistic Model Function
Remarks (interpretation of y(x) as probability):
Remarks (derivation of Lσ (w)):
Remarks (derivation of Lσ (w)): (continued) = argmax
Logistic Regression Recap. Linear Regression for Classification
Logistic Regression Recap. Linear Regression for Classification
Logistic Regressionn Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Logistic Regression for Classification
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Logistic Regression Non-Linear Decision Boundaries
Remarks:
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Loss Computation Loss Computation in Linear Regression
Remarks:
Loss Computation Loss Computation in Logistic Regression
Loss Computation Loss Computation in Logistic Regression
Loss Computation Loss Computation in Logistic Regression
Loss Computation Loss Computation in Logistic Regression
Remarks:
Loss Computation The BGDσ Algorithm
Remarks:
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Loss Computation Logistic Regression in a Nutshell
Chapter ML:III
Overfitting Definition 9 (Overfitting)
Overfitting Definition 9 (Overfitting)
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Example: Linear Regression
Overfitting Definition 9 (Overfitting
Overfitting Mitigation Strategies
Overfitting Mitigation Strategies
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Regularization Bound the Absolute Values of the Weights w
Remarks:
Remarks (continued) :
Regularization The Vector Norm as Regularization Function
Remarks:
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Remarks:
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Regularization The Vector Norm as Regularization Function
Remarks:
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Regularization Regularized Linear Regression
Chapter ML:IV IV. Neural Networks
Perceptron Learning Biological Inspiration
Perceptron Learning Biological Inspiration
Remarks (facts about the human brain):
Perceptron Learning History
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Perceptron Learning The Perceptron of Rosenblatt
Remarks:
Perceptron Learning Binary Classification Problems
Perceptron Learning The PT Algorithm
Remarks: (?) Recap.
Perceptron Learning Weight Adaptation: Illustration in Input Space
Perceptron Learning Weight Adaptation: Illustration in Input Space
Remarks: ~ = (w1 , . . . , wp )T , and the
Perceptron Learning Example
Perceptron Learning Example
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Example: Illustration in Input Space
Perceptron Learning Perceptron Convergence Theorem
Perceptron Learning Perceptron Convergence Theorem
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Proof
Perceptron Learning Perceptron Convergence Theorem: Discussion
Perceptron Learning Perceptron Convergence Theorem: Discussion
Chapter ML:IV
Gradient Descent Motivation
Gradient Descent Motivation
Gradient Descent Motivation
Gradient Descent Principle
Gradient Descent Principle
Remarks:
Gradient Descent
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (1) Linear Regression + Squared Loss
Gradient Descent (1) Linear Regression + Squared Loss
Remarks (how to derive ∇L2 (w)) :
Gradient Descent The BGD Algorithm
Remarks: (?) Recap.
Gradient Descent Global Loss versus Pointwise Loss
Remarks:
Gradient Descent The IGD Algorithm
Remarks (IGD) :
Remarks (GD versus PT) :
Gradient Descent (2) Linear Regression + 0\u002f1 Loss
Gradient Descent (2) Linear Regression + 0\u002f1 Loss
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (continued)
Gradient Descent (3) Logistic Regression + Logistic Loss + Regularization
Gradient Descent (3) Logistic Regression + Logistic Loss + Regularization
Remarks: ~ = (w1 , . . . , wp )T , and the
Remarks (how to derive ∇Lσ (w)) :
Remarks (how to derive ∇Lσ (w)) (continued) : (3)
Chapter ML:IV
Multilayer Perceptron Definition 1 (Linear Separability)
Multilayer Perceptron Definition 1 (Linear Separability)
Multilayer Perceptron Linear Separability
Multilayer Perceptron Linear Separability
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Multilayer Perceptron Overcoming the Linear Separability Restriction
Remarks:
Remarks (continued) :
Multilayer Perceptron Unrestricted Classification Problems
Multilayer Perceptron Unrestricted Classification Problems: Example
Multilayer Perceptron Unrestricted Classification Problems: Example
Multilayer Perceptron Sigmoid Function
Multilayer Perceptron Sigmoid Function
Remarks:
Remarks (continued) :
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron Network Architecture
Multilayer Perceptron (1) Forward Propagation
Remarks:
Multilayer Perceptron (1) Forward Propagation
Multilayer Perceptron (1) Forward Propagation
Multilayer Perceptron (1) Forward Propagation: Batch Mode
Multilayer Perceptron (2) Backpropagation
Multilayer Perceptron (2) Backpropagation
Multilayer Perceptron (continued)
Multilayer Perceptron (continued)
Multilayer Perceptron (continued)
Remarks:
Multilayer Perceptron (2) Backpropagation
Multilayer Perceptron (2) Backpropagation
Multilayer Perceptron (2) Backpropagation
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Multilayer Perceptron The IGD Algorithm for Multilayer Perceptrons
Remarks:
Multilayer Perceptron Network Architecture with Arbitrary Depth
Multilayer Perceptron (1) Forward Propagation at Arbitrary Depth
Multilayer Perceptron (2) Backpropagation at Arbitrary Depth
Multilayer Perceptron (2) Backpropagation at Arbitrary Depth (continued)
Multilayer Perceptron (2) Backpropagation at Arbitrary Depth (continued) [one hidden layer]
Multilayer Perceptron (2) Backpropagation at Arbitrary Depth (continued) [one hidden layer]
Multilayer Perceptron (2) Backpropagation at Arbitrary Depth
Multilayer Perceptron The IGD Algorithm for MLP of Arbitrary Depth
Multilayer Perceptron The IGD Algorithm for MLP of Arbitrary Depth
Multilayer Perceptron The IGD Algorithm for MLP of Arbitrary Depth
Multilayer Perceptron The IGD Algorithm for MLP of Arbitrary Depth
Remarks (derivation of ∇hs L2 (w)) :
Remarks (derivation of ∇hs L2 (w)) (continued) :
Remarks (derivation of ∇hs L2 (w)) (continued) :
Remarks (derivation of ∇hs L2 (w)) (continued) :
Remarks (derivation of ∇hs L2 (w)) (continued) : h
Remarks (derivation of ∇hs L2 (w)) (continued) : h
Remarks (derivation of ∇o L2 (w) and ∇h L2 (w) in the one hidden layer MLP) :
Multilayer Perceptron Momentum Term
Chapter ML:III III. Decision Trees
Decision Trees Basics Classification Problems with Nominal Features
Decision Trees Basics Decision Tree for the Concept “EnjoySport”
Decision Trees Basics Decision Tree for the Concept “EnjoySport”
Decision Trees Basics Definition 1 (Splitting)
Decision Trees Basics Definition 1 (Splitting)
Decision Trees Basics Definition 1 (Splitting)
Decision Trees Basics Definition 1 (Splitting)
Remarks:
Decision Trees Basics Definition 2 (Decision Tree)
Decision Trees Basics Definition 2 (Decision Tree)
Remarks:
Decision Trees Basics Notation
Remarks:
Decision Trees Basics Algorithm Template: Construction
Decision Trees Basics Algorithm Template: Classification
Remarks:
Decision Trees Basics When to Use Decision Trees
Decision Trees Basics On the Construction of Decision Trees
Decision Trees Basics Evaluation of Decision Trees
Decision Trees Basics Evaluation of Decision Trees
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Size
Decision Trees Basics Evaluation of Decision Trees: Classification Error
Decision Trees Basics Evaluation of Decision Trees: Classification Error
Decision Trees Basics Evaluation of Decision Trees: Classification Error
Remarks:
Decision Trees Basics Evaluation of Decision Trees: Misclassification Costs
Decision Trees Basics Evaluation of Decision Trees: Misclassification Costs
Decision Trees Basics Evaluation of Decision Trees: Misclassification Costs
Remarks:
Chapter ML:III III. Decision Trees
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Splitting
Impurity Functions Definition 4 (Impurity Function ι)
Impurity Functions Definition 5 (Impurity of an Example Set ι(D))
Impurity Functions Definition 5 (Impurity of an Example Set ι(D))
Remarks:
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Impurity Functions Based on the Misclassification Rate
Impurity Functions Definition 7 (Strict Impurity Function)
Impurity Functions Definition 7 (Strict Impurity Function)
Remarks:
Impurity Functions Impurity Functions Based on Entropy
Remarks:
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Remarks [::::::: Bayes :::
Remarks (continued) :
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on Entropy
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Impurity Functions Impurity Functions Based on the Gini Index
Chapter ML:III III. Decision Trees
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Decision Tree Algorithms ID3 Algorithm (pseudo code)
Remarks:
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Remarks:
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Example
Decision Tree Algorithms ID3 Algorithm: Hypothesis Space
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Decision Tree Algorithms ID3 Algorithm: Inductive Bias
Remarks:
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Decision Tree Algorithms CART Algorithm
Chapter ML:III III. Decision Trees
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Remarks:
Decision Tree Pruning Overfitting
Decision Tree Pruning Overfitting
Remarks:
Decision Tree Pruning Overfitting
Decision Tree Pruning (a) Stopping
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning (b) Pruning: Reduced Error Pruning
Decision Tree Pruning Extensions
Chapter ML:IV IV. Statistical Learning
Probability Basics Area Overview
Probability Basics Area Overview
Probability Basics Definition 1 (Random Experiment, Random Observation)
Probability Basics Definition 1 (Random Experiment, Random Observation)
Remarks:
Probability Basics Definition 2 (Sample Space, Event Space)
Probability Basics Definition 2 (Sample Space, Event Space)
Probability Basics Definition 2 (Sample Space, Event Space)
Probability Basics Definition 2 (Sample Space, Event Space)
Probability Basics Definition 3 (Important Event Types)
Probability Basics Definition 3 (Important Event Types)
Remarks:
Probability Basics Approaches to Capture the Nature of Probability
Probability Basics Approaches to Capture the Nature of Probability
Remarks:
Probability Basics Approaches to Capture the Nature of Probability
Remarks:
Probability Basics Approaches to Capture the Nature of Probability
Probability Basics Approaches to Capture the Nature of Probability
Probability Basics Approaches to Capture the Nature of Probability
Remarks:
Probability Basics Approaches to Capture the Nature of Probability
Probability Basics Approaches to Capture the Nature of Probability
Probability Basics Axiomatic Approach to Probability
Probability Basics Axiomatic Approach to Probability
Probability Basics Axiomatic Approach to Probability
Probability Basics Axiomatic Approach to Probability
Probability Basics Axiomatic Approach to Probability
Remarks:
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Remarks:
Remarks (continued) :
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Probability Basics Conditional Probability
Remarks:
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Probability Basics Independence of Events
Chapter ML:IV
Bayes Classification Generative Approach to Classification Problems
Bayes Classification Bayes Theorem
Bayes Classification Bayes Theorem
Bayes Classification Bayes Theorem
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Example: Reasoning About a Disease
Bayes Classification Combined Conditional Events: P (Ai | B1, . . . , Bp)
Bayes Classification Combined Conditional Events: P (Ai | B1, . . . , Bp)
Bayes Classification Combined Conditional Events: P (Ai | B1, . . . , Bp)
Remarks [:::::::::::: information:::::
Remarks (continued) :
Bayes Classification Naive Bayes
Bayes Classification Naive Bayes
Remarks:
Remarks (continued) :
Bayes Classification Naive Bayes
Bayes Classification Naive Bayes
Bayes Classification Naive Bayes
Remarks:
Bayes Classification Naive Bayes: Classifier Construction Summary
Bayes Classification Naive Bayes: Classifier Construction Summary
Remarks:
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Bayes Classification Naive Bayes: Example
Chapter ML:IV
MAP versus ML Hypotheses The Frequentist Setting to Parameter Estimation
MAP versus ML Hypotheses The Frequentist Setting to Parameter Estimation
MAP versus ML Hypotheses The Frequentist Setting to Parameter Estimation
Remarks:
Kapitel ML:IX IX. Ensemble and Meta
Ensemble Methods Motivation (1): Generalisierungsfähigkeit
Ensemble Methods Motivation (2): No Free Lunch Theorem
Ensemble Methods Motivation (3): Instabilität von Lernverfahren
Ensemble Methods Motivation (4): Consequences
Ensemble Methods Bootstrap Aggregating (Bagging)
Remarks:
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Bootstrap Aggregating
Ensemble Methods Boosting Weak Classifiers
Ensemble Methods Adaptive Boosting (AdaBoost)
Ensemble Methods Adaptive Boosting
Ensemble Methods Adaptive Boosting
Remarks:
Ensemble Methods Discrete AdaBoost
Ensemble Methods Cascades of Classifiers (Cascading)
Ensemble Methods Cascading
Ensemble Methods Maßzahlen für Cascading
Ensemble Methods Maßzahlen für Cascading
Ensemble Methods Cascading
Ensemble Methods Cascading for Face Detection (Main Loop)
Ensemble Methods Cascading for Face Detection (Gesamtklassifikator)
Ensemble Methods Cascading for Face Detection(Einzelklassifikator in Kaskade)
Ensemble Methods Cascading for Face Detection (Einzelklassifikator in Ensemble)
Ensemble Methods Cascading for Face Detection (alle Features)
Ensemble Methods Cascading for Face Detection (ausgewählte Features)
Ensemble Methods Cascading for Face Detection (Effizienz)
Ensemble Methods Cascading for Face Detection (Effizienz beim Training)
Ensemble Methods Trainingsmengen
Ensemble Methods Grundlage der Entscheidung
Ensemble Methods Literature