Slide 1

Python

Origin of the Name

Intensions and Philosophy of Python

Starting Python

„Hello World“ in Python

Compiled or Interpreted?

Internal Execution if Code

Freezing the Code

Freezing Tools

Python's Keywords

Variables in C-like Languages

Variables

Variables in Python

Valid Identifier Names

Assignments

Variables in Python, 1

Variables in Python, 2

Variables in Python, 3

Casting in Python

Number Data Types and Literals

Close to all about Strings

Strings

Indexing and Slicing

Fundamentals on Strings

Escape Characters in Strings

print using Escape Character

raw strings

Strings are immutable

Operators

Lists / Sequences

Lists in Python

Immutable Lists: Tuple

Tuples Special Cases

Tuple packing und unpacking

Packing and Unpacking, 2

Splitting Strings

Examples

More on split

Disassemble String in Lines

Partition

join()

Finding Substrings

Explanatory Examples

String Substitution

Some more Methods

Stripping Strings

Examples

String Tests

Appending Elements to a List

The Optimal Way

Inserting Elements in Lists

Nested Lists

Entfernen von Elementen

Removing Values

Contained in List / Get the Index

Sequential Data Types

Inserting and Deleting with Slicing

Dictionaries

Example

Dictionaries

Operators on Dictionaries

Exercise

Solution

Converting Dictionaries into Lists

zip

Lists in Dictionaries, Python2

Dictionaries concatenating

Interactive Help

help()

Help Concerning Methods

Sets

Operations and Methods on Sets

Subset, Superset

frozenset

Writing Scripts

Grouping Statements

if statement

input(): Differences Python 2 and 3

Example

Dog Years

Solution

Calculating Leap Years

True or False?

“Abbreviated” if, Example

Python Modules

Command Line Parameters

The os Module

Custom-made Modules

Custom-made Modules, 2

Custom-made Modules, 3

Loops

While Loop: Syntax

Example: Sum from 1 to n

Exercise

Solution

Exercise

Guessing Number Game

Solution:

Alternative Solution:

The else Branch with Break

Code With a Break

For Loop

String Formatting

Formatted output: string modulo

Printing Floats

format with positional parameters

Example

format with keyword parameters

Iterate over a List

Iterate through a Dictionary

The range() Function

Sum of Numbers with for

Iterating over Lists

The pass Statement

Deep insight in Variables and Assignments

“Strange” Behaviour

Lists and Assignments

Proof with Identity

Copying with [:] / Shallow Copy

Copying with [:]

Copying with [:], part 2

The “real” deepcopy

Functions

Functions: Example

The Letter „E“

Reversing a List

The Savings Plan Formula

Exercise

Exercise: Morse Code

Solution, 1st part

Solution, 2nd part

Exercise

Solution

Exercise

Computing Square Roots

Beispiel: Heron-Verfahren

Exercise

Keyword Parameter

Variable Length Argument List

Arbitrary Keyword Parameters

* in Function Calls

Exercise

** in Function Calls

Simple Definition of Dictionaries

Parameter Passing in Detail

Call by Object

Example

Mutable Data Types as Parameter

Passing the Copy of a List

Lists as Default Parameters

The Proper Way to Do it!

Nested Functions

Global and local Variables

Namespaces and Scopes

Global or Local

Global and Local

Global Variables in Functions

Global Variables

Global Variables of Functions

Recursive Functions

Permutations

Exercise

Solution: flatten()

Alternative Solution

Iterative Solution (only lists)

Fibonacci

Exercise

Recursive Solution

Problem

Iterative Solution

Recursion with Memoization

Lucas numbers

Primality Test with Lucas Numbers

Decorators

Functions are Objects

Functions inside Functions

Another Example

Customized Functions

Polynomial Factory

Polynomial Factory, 2

A Simple Decorator I

Another Way to Decorate

Decorators - Summary

Example - Checking Parameters

Decorators with Parameters

Decorator with Parameter

Doing it Without Decorator Syntax

Using wraps from functools

Example

Manually Fixing the Problem

Better Way to Do it

Reading and Writing Files

Reading a File

Reading and Writing to a File

Reading and Writing

Reading „Line by Line“ / char by char

Exercise

Various ways of Reading a File

Using readline()

Simultaneous Reading and Writing

File Scanner

Get yourself into a pickle

A Simple Example

de“pickle“

What can be pickled and unpickled?

Pickling Class Objects

The shelve-Module

Python Modules

Applying the Previous Example

More on Modules

Importing Names Directly

Executing Modules as Scripts

Renaming a Namespace

Kinds of Modules

The Module Search Path

Directories with Modules

dir() Function

dir and help

List of built-in-Functions

Module Packages

Simple Example

Sound Package Example

Intra Package Reference

Slide 248

The map Function

Using lambda Notation

map() on Multiple Lists

Filtering of a List

reduce-Function

Example for reduce()

Exercises for reduce()

List Comprehension

List-Comprehension II

A more Challenging Example

Problem of the Previous Example

Exercises

Set Comprehension in Python 3

Generator Comprehension

„A dirty little secret“

sort

Parameter key

Defining an Ordering

sort with Key Function

Reverse Sort

Another Example

And Another One

Sorting With Indices

Alternatively with collection-Module

Output

Generators

Iterators in for Loops

Iterating manually

Simple Example

PI with Generators

Elegant method with next()

Permutations Generator

Variations without Repetitions

Obsolete Functions of itertools

Infinite Iterators: count

Infinite Iterators: cycle

Exercise

Infinite Iterators: repeat

accumulate / compress

groupby

starmap

Combinatoric Generators

Exercise

Handling Exceptions

Some Exceptions

Handling Exceptions

Functional Principle of try-except

Handling Exceptions, Example

Raising and Catching Exceptions

Raising and Catching Exceptions 2

Raising and Catching Exceptions 3

The optional else Clause

sys.exc_info()

Multiple Exceptions

except with Multiple Exceptions

Another Example

Raising Exceptions

Python2 vs. Python3

Clean-up Actions

Example: try … finally

All Together Now

The assert Statement

Using assert

Context Manager

Exercise: HTML Tags

Yield in the Context Manager

Typical Usage:

Composition of Context Managers

Unit Tests

Syntax Errors

Semantical Errors

Modul Tests using __name__

Exercise: Palindromic Square Numbers

Solution, Part1

Solution, Part2

Test-Driven Development

unittest Module

Example

Running the Test Cases

New run

Objects and Classes

Object Oriented Programming

Strawberry Tart Class

Some Simple Definitions

Public Libraries

First-class Everything

Minimal Class in Python

Old-Style vs. New Style Classes

Attributs

Attributes and Class Names

Class Attributes

Robot Class with Class Attributes

Retrieving the Values of Attributes

Functions with Attributes

Methods in Classes, 1

Methods in Classes, 2

The __init__ Method

Robot Class with __init__

Data Abstraction

Data Encapsulation: “traditional”

Robots with Getters and Setters

Exercise

Name Schema for Attributes

Example

Class with Private Attributes

Exercise

Counting Instances

__str__ and __repr__ Methods

__str__ and __repr__

Getting Rid of Getters and Setter ...

Property with del and Docstring

Varying Numbers of Parameters

Parameter Overloading

Built-in Methods of Operators

Length Class

Length Class, Implementation

Length Class, Implementation, 2

Improved __add__ and __iadd__

Example: Employee, Person

Overriding __str__

Inheritance and Attributes

Staticmethods vs. Classmethods

Now with a Classmethod

Inheriting Counter Class

Extended List Class

Defaultdict Class

Context Managers with Classes

Multiple Inheritance

Calendar Class

Calendar Class, 2

CalendarClock Class

Cooperative Classes

Illustrative Example

Example, Continuation

Another Problem

“Solution” for the problem

Traditional Approach

Method Resolution Order

super()

Slots and Metaclasses

Avoiding dynamically created attributes

Classes, that does not work

Slots

Slots II

Class creation: type

type()

Example

type, __new__ and __init__

Next example

Next example, II

On the road to metaclasses

Bad solution

Without redundant code

Decision at runtime

„Optional“ methods

Continue...

Manager function

augment_answer, I

augment_answer, II

Some improvements

Class decorateur, I

Class decorateur, II

Metaclasses

„Little Meta“

Back to the philosophers

Continue...

Principle of operation

Singleton

Singeltons with metaclasses

Singletons with Inheritance

System- Programming Threads, Pipes and Multiprocessing under Python

Some Modules

Method to end a Script

Using sys.exit()

Using os._exit()

Shell Environment Variables

Standard in- and output

IO under python

Standard Streams

Usage of the standard streams

Combined Example

I/O Redirection

Redirecting Output

Error redirecting

Error Redirection with print

Python and the Shell

Reading characters without return

General Solution

Problems in the previous example

Results of Shell Scripts

Shell outputs: line for line

subprocess Module

Work with subprocess

Popen and the Command String

Adopt output in Python

wait()

Command Line Parameters

Forking with counters

Starting independent programms

The exec*() functions

Bash script test.sh

Example with execvp()

Correct call

Examples

Adding a Path to PATH

execlpe() instead of execvpe()

Examples

Classification of the exec*() functions

Back to forks

Python scripts with exec*()

Slide 511

Multitasking vs. Parallel Processing

CPU-Bound vs. I/O-Bound Tasks

Introduction

Threads

Process with threads

Threads in Python

The threading Module

Example with class, 1

Example with class, 2

Using Functions as Threads

Exercise

Example: threading module

Threads as Demons

Shared Data

Increasing the Iterations

Race conditions

Mutual Exclusion

Solution for Previous Example

Possible Deadlock

RLock

Rlock example

Semaphores

Semaphores in threading

The GIL

The Check Interval

Raison d'etre of the GIL

Practical tasks

ping

Solution Without Threads

Solution with threads (1)

Solution with threads (2)

Pipes

Anonymous and Named Pipes

Beer-Pipe

Solution, part 1

Solution, part 2

Bidirectional Pipes

Number Gessing Game

Number Guessing Game

Number Guessing Game: Code

Number Guessing Game: Code 2

Number Guessing Game: Code 3

Improving the Guesses

Names Pipes, Fifos

Example

Multiprocessing Module

Simple Example

Exercise

Exercise, part 2

Timing the Benefit

Timing the Benefit, other example

Exchanging Values / Objects

Pool

Manager object

Manager, example

Extending and Embedding Python

Embedding and Extending

Basic Procedure

C/C++ in Python

Ctypes Module

Embedding Custom Library

so-Library: create and use

Embedding Custom Library

Numerically stable Version

Proper Wrapper Functions

Wrapper Function

Examples of PyArg_ParseTuple

Py_BuildValue

Module Initialization

Compiling for Dynamic Loading

Exercise

Problem and Solution

SWIG

Simple Example: example.c

SWIG Interface File

Building the Python Interface

Simple Example: Compiling C Prog

Simple Example: Using in Python

Prev. Example with Addional Function

A C++ Simple Example

Using the New Factorial Module

SWIG Transformations

Constants and enum in C

Using the „ultimate“ Module

struct Example

Internal Representation in SWIG

struct Example II

Exercise

Solution

Typemaps

Using the „example“ module

Using the Module

typemaps.i libray

typemaps.i libray, II

Pointers

Returning Lists

Converting Python list to a char **

Creating and Using the Module foo

Classes and Swig

Clock Class: Clock.h and Clock.i

Clock Class Used in Python

Clock Class with getTime()

Python in C/C++

Embedding Python in C / C++

Small Example

Using the Interpreter as a Shell

Getting Int Variables

Reading int, string and float

Slide 625

SQLITE

Starting with SQLITE

Cursor

Communicating with the Data Base

Inserting Data with execute Commit

Inquiring Data

MySQL under Python

Working with MySQL

Example: Get Server Version

Example: Server Version (Python 3)

Create Table / Insert (I)

Create Table / Insert (II)

Create Table / Insert (III)

Results of Previous Script

Select

Insert into a table, part 1

Insert into a table, part2

Selecting a Information

Importing NumPy and First Example

Speed Advantage

Lists in darrays

Creating Evenly Spaced Arrays

linspace

Looking for Something

Arrays in NumPy

Nested Lists/Tuples

Axis

Data Types

Data Types II

Casting darrays into another Type

Slicing

Further Indexing Options

Views

Exercise

Solution

Arrays With Various Data Types

nonzero

Boolean Indexing

Boolean Indexing II

Result of Previous Script

Boolean Indexing III

Exercise

Solution

Indexing with an Array of integers

Reshaping Arrays

Copying Arrays

flatten

flatten() 3-dimensional

Concatenation

New Dimensions

Combine Column Vectors

Zeros and Ones

Exercise

Identity / Diagonal Matrix

Diagonal Matrices: Alternative Way

Examples

numpy.where

Watermark Image with ‘where’

Scalar Produkt (Dot Product)

Matrix Multiplication

Practical Application

Expenditure per Person

Cross Product / Vector Product

Example with Cross Product

Submodul linalg

Fibonacci with matrices

Reductions

Core Linear Algebra Tools in linalg

Mathematical Functions

Polynoms

Polynome

Previous Plot is generated by

Deviations and Integrals

Integration mit scipy

Evaluate Polynoms and Regression

Exercise

Solution

Statistical Functions

random-submodule

Multidemensional Random Numbers

random() and randint()

Randomly shuffling lists

Permutationen

Gauß and Poisson

Reading Data

Reading Example File

Transforming into Floats

Using matplot in a Notebook

matplotlib in a program

Arguments of plot

Format String of plot

Changing the plot Behaviour

A Simple Example

Plotting Discrete Values

Defining the Axis

Changing Default Range for Axis

Line Properties

Changing the Line Style

Configuring Spines

Changing Spines Example

Customizing Ticks

Setting Tick Labels

Adding a Legend

Annotations

Tick Label Rendering

Tick Label Rendering, Example

Diagram of Previous Example

subplots, horizontal

Slide 761

subplots, vertical

Diagram of Previous Example

Without Subplots

Subplots with gridspec, Part 1

Subplots with gridspec, Part 2

Slide 773

Previous Example with Graphs, 1

Previous Example with Graphs, 2

Slide 776

Working with Objects

Slide 778

Inserted Diagram

Inserted Diagrams

Saving Figures

Setting the Plot Range

Plot

Logarithmic Scale

Grid Lines

Twin Axes

Diagram of Previous Program

Creating a „meshgrid“

Calculation of the Values

Creating the Contour Plot

Refinement

Colors and Linestyle

Filled Contours

Individual Colors

Levels

Example

Plot from Previous Example

pandas

Introduction

Series

Relationship to Numpy

Using Individual Indices

Different Indices

Accessing Values of a Series

pandas.Series.apply

Example for apply

Exercise

Indexing with Boolean Arrays

Series and Dictionaries

Missing Data - NaN

Missing Data

isnull() and notnull()

Filtering out Missing Data

None Values in Dictionaries

Filtering Out Missing Data

DataFrame

DataFrame Example

Output

Changing the Index

Rearrange The Ordering of Columns

Existing column as Index

Alternative Approach

Summing up Columns

Cumulative sum: cumsum

Reassign a Column with New Values

Adding New Columns

Assigning New Values to Column

Exercise

Output of Previous Exercise

Sorting DataFrames

Nested Dictionary vs. DataFrame

Result

Transposing the Data

Reindexing with „reindex“

Randomly Populating a DataFrame

Reading csv Files

Machine Learning

Creating the Virtual Environment

Quandl.com

Data Scientist

Machine Learning

Databases to Dig for Data

Scikit - sklearn

k-Nearest-Neighbor Classifier

k-Nearest-Neighbor Mathematically

K-nearest-neighbour Classification

Dataset

The Iris Data Set in sklean

Creating a Learnset

Visualising our Learnset

Learnset Visualisation

Determining the Neighbors

Creating a List of k Neighbors

Weighted Nearest Neighbor Classifier

Exercise

Testing the Previous Solution

Exercise: Using the Real Distance

kNN in Linguistics

Solutions

General Word Misspellings

Output

Neural Networks

Neuron

Abstraction of a Biological Neuron

Perceptron / Neuron

Detailled View

Simple ANN without Hidden Layers

Simple ANN for the „AND“ Function

Using the Perceptron Class

Line Separation - Decision Boundary

Implementation, Part1

Implementation, Part2

Implementation, Part3

Decision Boundary, Calculation

Plotting the Decision Boundary

Slide 889

A New Dataset and Scatterplot of it

A New Dataset

Exercise

Visualize Decision Boundary

ANN with Hidden Layers

Single Layer with Bias

xor Problem

The XOR Solution

Neural Network with Bias Values

Program

Scatterplot of Data

Neural Network with Bias

Neural Network with Bias, Part2

Neural Network with Bias, Part3

Neural Network with Bias, Part4

Results and Plot

Activation Functions

Naive Bayes Classifier

Conditional Probability

Alien and German Swiss Speaker

Solution

Bayes Theorem

Naive Bayes Classifier: Intro Example

Plotting the Data Histogram

Histogram

Classifier for TrainConnection Prob

Classifier Based on Naive Bayes

Generalization for Multiple Features

A Boy Named Sue

Random Person Data for NB Classifier

Creating the Datasets

Filtering Firstnames and Heights

Designing Feature Class

Finally, the Classifier

Finally, the Classifier, II

The Classifier

Classifier with the firstname

What about Jessie

NB Classifier with Two Features