Numpy tutorial

Basic idea of numpy

Manipulate array of numbers very efficiently

The bad example, whithout numpy:

>>> import time
>>> import math
>>> xs = range(1000000)
>>> t11 = time.time()
>>> ys = []
>>> for x in xs:
...   ys.append(math.sqrt(x))
...
>>> t12 = time.time()
>>> print(t12-t11)
0.582495212555
>>>

The good example, with numpy:

>>> from numpy import *
>>> xs = arange(1000000)
>>> t21 = time.time()
>>> ys = sqrt(xs)
>>> t22 = time.time()
>>> print(t22-t21)
0.0222141742706

The ratio:

>>> print ((t12-t11)/(t22-t21))
26.2217809881

Warning

Avoid loops in Python when manipulating array of numbers

Numpy objects

Array objets are set of numbers of same type.

Some definitions:

name definition
size total number of elements
shape integer tuple giving the number of elements in each dimension
rank size of “shape” == number of dimensions
dtype type of elements in the array

Some types:

type name type code
bolean Bool
unsigned UInt8, UInt16, UInt32, UInt64
integer Int8, Int16, Int32 (Int), Int64
float Float32, Float64 (Float)
complex Complex32, Complex64 (Complex)

Creating a Numpy array

creating explicitely:

>>> from numpy import *
>>> x = array([1,2,3])
>>> x
array([1, 2, 3])
>>> y = array([[1,2],[3,4],[5,6]],float)
>>> y
array([[ 1.,  2.],
       [ 3.,  4.],
       [ 5.,  6.]])
>>> x.size
3
>>> x.shape
(3,)
>>> x.dtype
dtype('int64')
>>> y.size
6
>>> y.shape
(3, 2)
>>> y.dtype
dtype('float64')

creating from a python list:

>>> asarray((1,2,3))
array([1, 2, 3])

creating from building function:

>>> zeros(3)
array([ 0.,  0.,  0.])
>>> zeros((5,5))
array([[ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])
>>> ones((2,3),float)
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]])
>>> arange(10)
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> identity(10)
array([[ 1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.]])

creating from a function:

>>> def x2(x):
...   return x*x
...
>>> def xy(x,y):
...   return x*y
...
>>> fromfunction(x2,(5,))
array([  0.,   1.,   4.,   9.,  16.])
>>> fromfunction(xy,(5,5))
array([[  0.,   0.,   0.,   0.,   0.],
       [  0.,   1.,   2.,   3.,   4.],
       [  0.,   2.,   4.,   6.,   8.],
       [  0.,   3.,   6.,   9.,  12.],
       [  0.,   4.,   8.,  12.,  16.]])

change dimension:

>>> x = array([1,2,3,4,5,6,7,8,9,10,11,12])
>>> reshape(x,(2,6))
array([[ 1,  2,  3,  4,  5,  6],
       [ 7,  8,  9, 10, 11, 12]])
>>> y = reshape(x,(3,4))
>>> y
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])
>>> reshape(y,(12,))
array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12])
>>> y.shape = (12,)
>>> y
array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12])

casting:

>>> x = array([1,2,3],float)
>>> y = array([1,2,3],int)
>>> (x+y).dtype
dtype('float64')
>>> x.astype(int)
array([1, 2, 3])

Indexing arrays

Warning

index in Fortran a[x,y], index in Numpy a[y,x],

Retreieve an element or a group of elements:

>>> a = arange(12)
>>> a[0]
0
>>> a[1]=-a[1]
>>> a
array([ 0, -1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])
>>> a[2:4]
array([2, 3])
>>> a[2:4] = [-2,-3]
>>> a[5:]
array([ 5,  6,  7,  8,  9, 10, 11])

Multidimentional arrays:

>>> a = reshape(a,(3,4))
>>> a[2,0]
8
>>> a[0]
array([ 0, -1, -2, -3])
>>> a[0,:]
array([ 0, -1, -2, -3])
>>> a[:,0]
array([0, 4, 8])
>>> a[1:,1:]
array([[ 5,  6,  7],
       [ 9, 10, 11]])
>>> a[1:,1:] = ones((2,3))

Using a list of indexes:

>>> a = arange(12)**2
>>> index = [0,4,3]
>>> a[index]
array([ 0, 16,  9])
>>> a[arange(3)]
array([0, 1, 4])

>>> a = reshape(a,(3,4))
>>> ind1 = [1,2]
>>> ind2 = [0,3]
>>> a[ind1,ind2]
array([ 16, 121])

>>> ind1 = array([[1,2],[0,2]])
>>> ind2 = array([[0,3],[1,2]])
>>> a[ind1,ind2]
array([[ 16, 121],
       [  1, 100]])

Special values

nan, inf:

>>> x = array([0,1],float)
>>> y = x/0.
__main__:1: RuntimeWarning: divide by zero encountered in divide
__main__:1: RuntimeWarning: invalid value encountered in divide
>>> y
array([ nan,  inf])

track special values:

>>> isnan(y)
array([ True, False], dtype=bool)
>>> isinf(y)
array([False,  True], dtype=bool)
>>> isfinite(y)
array([False, False], dtype=bool)
>>> isfinite(x)
array([ True,  True], dtype=bool)

replace bad values:

>>> y[isnan(y)]=0
>>> y[isinf(y)]=1e10
>>> y
array([  0.00000000e+00,   1.00000000e+10])

Array methods

Some important methods:

>>> x = array([0,4,3,2])
>>> x.min()
0
>>> x.max()
4
>>> x.mean()
2.25
>>> x.argmax()
1
>>> x.argmin()
0
>>> x.argsort()
array([0, 3, 2, 1])
>>> x.sum()
9
>>> x.conjugate()
array([0, 4, 3, 2])

Operations on arrays (ufcts)

Operations elements by elements return an array of same shape. Lists of ufcts:

add (+) substract (-) multiply (*) divide ()
remainder (%) power (**)    
abs fabs floor ceil
fmod conjugate    
maximum minimum    
cos sin arccos arcsin
cosh sinh arccosh arcsinh
tan arctan tanh arctanh
log log10 exp  
greater (>) greater_equal(>=) equal(==)  
less (<) less_equal(<=) not_equal(!=)  
logical_or logical_xor logical_not logical_and
bitwise_or (|) bitwise_xor (^) bitwise_not (~) bitwise_and (&)
rshift(>>) lshift(<<)    

Examples:

>>> x = array([1,2,3],float)
>>> y = array([4,5,6],float)
>>>
>>> x/y
array([ 0.25,  0.4 ,  0.5 ])
>>> x**2
array([ 1.,  4.,  9.])
>>>
>>> x = array([1,2,3])
>>> y = reshape(arange(12),(4,3))
>>> x+y
array([[ 1,  3,  5],
       [ 4,  6,  8],
       [ 7,  9, 11],
       [10, 12, 14]])
>>> y*= 10
>>> y+= y
>>>
>>>
>>> x = array([1,2,3])
>>> y = array([0,2,3],float)
>>> (x>y)
array([ True, False, False], dtype=bool)
>>> (x==2)
array([False,  True, False], dtype=bool)
>>> (x>y) | (x==2)
array([ True,  True, False], dtype=bool)

Array functions

General function on arrays.

Warning

Always think with vectors

Replace specific values in a vectors:

>>> x = arange(16)
>>> x = reshape(x,(4,4))
>>> y = fmod(x,3)
>>> y
array([[0, 1, 2, 0],
       [1, 2, 0, 1],
       [2, 0, 1, 2],
       [0, 1, 2, 0]])
>>>
>>> where((y==0),-1,y)
array([[-1,  1,  2, -1],
       [ 1,  2, -1,  1],
       [ 2, -1,  1,  2],
       [-1,  1,  2, -1]])

Replace values using a mask:

>>> mask = where((y==0),1,0)
>>> mask
array([[1, 0, 0, 1],
       [0, 0, 1, 0],
       [0, 1, 0, 0],
       [1, 0, 0, 1]])
>>> putmask(x,mask,y)
>>> x
array([[ 0,  1,  2,  0],
       [ 4,  5,  0,  7],
       [ 8,  0, 10, 11],
       [ 0, 13, 14,  0]])

Example: remove bad values in a vector:

>>> x = array([0,1,2,3,4,5],float)
>>> y = array([1,0,3,0,5,2],float)
>>> ly = log10(y)
__main__:1: RuntimeWarning: divide by zero encountered in log10
>>> ly
array([ 0.        ,        -inf,  0.47712125,        -inf,  0.69897   ,
        0.30103   ])
>>>
>>> c = (y!=0)
>>> x = compress(c,x)
>>> y = compress(c,y)
>>> ly = log10(y)
>>> ly
array([ 0.        ,  0.47712125,  0.69897   ,  0.30103   ])

Other very usefull functions: ravel, clip, sum, concatenate:

>>> x = identity(3)
>>> x
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]])
>>> ravel(x)
array([ 1.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  1.])

>>> x = array([1,2,3])
>>> y = array([4,5,6])
>>> z = concatenate((x,y))
>>> clip(z,2,5)
array([2, 2, 3, 4, 5, 5])
>>> sum(z)
21

Matrix operations: transpose, diagonal, trace, dot product:

>>> a = array([1,2])
>>> b = array([4,5])
>>>
>>> dot(a,b)
14

Polynomials:

>>> from numpy import poly1d
>>> p1 = poly1d([3,4,5])
>>> p2 = poly1d([1,1,1])
>>> print p1
   2
3 x + 4 x + 5
>>> print p1+p2
   2
4 x + 5 x + 6
>>> print p1*p2
   4     3      2
3 x + 7 x + 12 x + 9 x + 5
>>> print p1.deriv()

6 x + 4
>>> print p1.integ(k=6)
   3     2
1 x + 2 x + 5 x + 6