Pandas Series - Selection and Indexing
DONE

Files associated with this lesson:

btc-market-price.csv

          
      2017-04-02 00:00:00
      1099.169125
    
      0
      2017-04-03 00:00:00
      1141.813000
    
      1
      2017-04-04 00:00:00
      1141.600363
    
      2
      2017-04-05 00:00:00
      1133.079314
    
      3
      2017-04-06 00:00:00
      1196.307937
    
      4
      2017-04-07 00:00:00
      1190.454250
    
      5
      2017-04-08 00:00:00
      1181.149838
    
      6
      2017-04-09 00:00:00
      1208.800500
    
      7
      2017-04-10 00:00:00
      1207.744875
    
      8
      2017-04-11 00:00:00
      1226.617037
    
      9
      2017-04-12 00:00:00
      1218.922050

Lecture.ipynb

rmotr

Pandas Series - Selection and Indexing¶

Pandas Series object acts in many ways like a one-dimensional NumPy array, and in many ways like a standard Python dictionary. If we keep these two overlapping analogies in mind, it will help us to understand the patterns of data indexing and selection in these data structures.

purple-divider

Hands on!¶

In [1]:

import pandas as pd
import numpy as np

green-divider

The first thing we'll do is create again the Series from our previous lecture:

In [2]:

data_dic = {
    'Canada': 35.467,
    'France': 63.951,
    'Germany': 80.94,
    'Italy': 60.665,
    'Japan': 127.061,
    'United Kingdom': 64.511,
    'United States': 318.523
}

g7_pop = pd.Series(data_dic,
                   name='G7 Population in millions')

In [3]:

g7_pop

Out[3]:

Canada             35.467
France             63.951
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Name: G7 Population in millions, dtype: float64

green-divider

Indexing¶

Indexing works similarly to lists and dictionaries.

Indexing by index¶

you use the index of the element you're looking for:

In [4]:

g7_pop['Canada']

Out[4]:

35.467

In [5]:

g7_pop['Japan']

Out[5]:

127.061

In [6]:

g7_pop['United Kingdom']

Out[6]:

64.511

The following also works, but it's NOT recommended:

In [7]:

g7_pop.Japan

Out[7]:

127.061

Slicing and multi-selection¶

Slicing also works, but important, in Pandas, the upper limit is also included:

In [8]:

g7_pop['Germany': 'Japan']

Out[8]:

Germany     80.940
Italy       60.665
Japan      127.061
Name: G7 Population in millions, dtype: float64

Multi indexing also works (similarly to numpy):

In [9]:

g7_pop[['Italy', 'France', 'United States']]

Out[9]:

Italy             60.665
France            63.951
United States    318.523
Name: G7 Population in millions, dtype: float64

Indexing by sequential position¶

Indexing elements by their sequential position also works. In this case pandas evaluates the object received; if it doesn't exist as an index, it'll try by sequential position.

With sequential position the upper limit is not included.

In [10]:

g7_pop

Out[10]:

Canada             35.467
France             63.951
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Name: G7 Population in millions, dtype: float64

In [11]:

g7_pop.iloc[0] # First element

Out[11]:

35.467

In [12]:

g7_pop.iloc[-1] # Last element

Out[12]:

318.523

Other examples:

In [13]:

g7_pop.iloc[2]

Out[13]:

80.94

In [14]:

g7_pop.iloc[4]

Out[14]:

127.061

In [15]:

g7_pop.iloc[2:4]

Out[15]:

Germany    80.940
Italy      60.665
Name: G7 Population in millions, dtype: float64

In [16]:

g7_pop.iloc[[3, 1, 6]]

Out[16]:

Italy             60.665
France            63.951
United States    318.523
Name: G7 Population in millions, dtype: float64

green-divider

Adding new elements to a `Series`¶

In many cases we'll want to add new values to our Series, to do that we can just simply index our Series using the new index and then assigning a value to that index. Let's add two new records:

In [17]:

g7_pop['Brazil'] = 20.124
g7_pop['India'] = 32.235

In [18]:

g7_pop

Out[18]:

Canada             35.467
France             63.951
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Brazil             20.124
India              32.235
Name: G7 Population in millions, dtype: float64

green-divider

Modifying `Series` elements¶

In [19]:

g7_pop['Canada'] = 40.5

g7_pop

Out[19]:

Canada             40.500
France             63.951
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Brazil             20.124
India              32.235
Name: G7 Population in millions, dtype: float64

In [20]:

g7_pop['France'] = np.nan

g7_pop

Out[20]:

Canada             40.500
France                NaN
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Brazil             20.124
India              32.235
Name: G7 Population in millions, dtype: float64

green-divider

Removing elements from a `Series`¶

In [21]:

del g7_pop['Brazil']

g7_pop

Out[21]:

Canada             40.500
France                NaN
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
India              32.235
Name: G7 Population in millions, dtype: float64

In [22]:

del g7_pop['India']

g7_pop

Out[22]:

Canada             40.500
France                NaN
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Name: G7 Population in millions, dtype: float64

green-divider

Checking existance of a key (membership)¶

In [23]:

'France' in g7_pop

Out[23]:

True

In [24]:

'Brazil' in g7_pop

Out[24]:

False

green-divider

Introducing `loc` & `iloc`¶

What's the problem with the indexing we've seen? It's not explicit. Pandas receives an element to index and it tries figuring out if we meant to select an element by its key, or its sequential position. Check out the following example:

In [25]:

s = pd.Series(
    ['a', 'b', 'c'],
    index=[1, 2, 3])
s

Out[25]:

1    a
2    b
3    c
dtype: object

In [26]:

Out[26]:

1    a
2    b
3    c
dtype: object

What happens if we try indexing s[1], what should it return? a or b?

In [27]:

s[1]

Out[27]:

'a'

In this case, the returned object is worked out by the index, not by the sequential position. But again, it's not intuitive or explicit.

Enter loc and iloc:

loc is the preferred way to select elements in Series (and Dataframes) by their index
iloc is the preferred way to select by sequential position

In [28]:

s.loc[1]

Out[28]:

'a'

In [29]:

s.iloc[1]

Out[29]:

'b'

In [30]:

g7_pop

Out[30]:

Canada             40.500
France                NaN
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Name: G7 Population in millions, dtype: float64

In [31]:

g7_pop.iloc[-1]

Out[31]:

318.523

In [32]:

g7_pop.iloc[[0, 1]]

Out[32]:

Canada    40.5
France     NaN
Name: G7 Population in millions, dtype: float64

Using our previous series:

In [33]:

g7_pop

Out[33]:

Canada             40.500
France                NaN
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Name: G7 Population in millions, dtype: float64

In [34]:

g7_pop.loc['Japan']

Out[34]:

127.061

In [35]:

g7_pop.iloc[-1]

Out[35]:

318.523

In [36]:

g7_pop

Out[36]:

Canada             40.500
France                NaN
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     318.523
Name: G7 Population in millions, dtype: float64

In [37]:

g7_pop.loc['Canada']

Out[37]:

40.5

In [38]:

g7_pop.iloc[0]

Out[38]:

40.5

In [39]:

g7_pop.iloc[-1]

Out[39]:

318.523

In [40]:

g7_pop.loc[['Japan', 'Canada']]

Out[40]:

Japan     127.061
Canada     40.500
Name: G7 Population in millions, dtype: float64

In [41]:

g7_pop.iloc[[0, -1]]

Out[41]:

Canada            40.500
United States    318.523
Name: G7 Population in millions, dtype: float64

`loc` & `iloc` to modify `Series`¶

In [42]:

g7_pop.loc['United States'] = 1000

g7_pop

Out[42]:

Canada              40.500
France                 NaN
Germany             80.940
Italy               60.665
Japan              127.061
United Kingdom      64.511
United States     1000.000
Name: G7 Population in millions, dtype: float64

In [43]:

g7_pop.iloc[-1] = 500

g7_pop

Out[43]:

Canada             40.500
France                NaN
Germany            80.940
Italy              60.665
Japan             127.061
United Kingdom     64.511
United States     500.000
Name: G7 Population in millions, dtype: float64

green-divider

Introducing to Boolean arrays¶

Another way to select certain values within a Series is using boolean arrays, also known as Conditional selection.

We can index our Series using a list of boolean values:

In [44]:

g7_pop[[False, False,  True, False,  True, False,  True]]

Out[44]:

Germany           80.940
Japan            127.061
United States    500.000
Name: G7 Population in millions, dtype: float64

Or we can index our Series using another Series with boolean values:

In [45]:

condition = pd.Series([
    False, False,  True, False,  True, False,  True
], index=[
    'Canada', 'France', 'Germany', 'Italy', 'Japan', 'United Kingdom', 'United States'
])

condition

Out[45]:

Canada            False
France            False
Germany            True
Italy             False
Japan              True
United Kingdom    False
United States      True
dtype: bool

In [46]:

g7_pop[condition]

Out[46]:

Germany           80.940
Japan            127.061
United States    500.000
Name: G7 Population in millions, dtype: float64

On upcoming lectures we'll see how to use more complex conditional selections.

purple-divider

	2017-04-02 00:00:00	1099.169125
0	2017-04-03 00:00:00	1141.813000
1	2017-04-04 00:00:00	1141.600363
2	2017-04-05 00:00:00	1133.079314
3	2017-04-06 00:00:00	1196.307937
4	2017-04-07 00:00:00	1190.454250
5	2017-04-08 00:00:00	1181.149838
6	2017-04-09 00:00:00	1208.800500
7	2017-04-10 00:00:00	1207.744875
8	2017-04-11 00:00:00	1226.617037
9	2017-04-12 00:00:00	1218.922050

Pandas Series - Selection and Indexing DONE check

Files associated with this lesson:

Pandas Series - Selection and Indexing¶

Hands on!¶

Indexing¶

Indexing by index¶

Slicing and multi-selection¶

Indexing by sequential position¶

Adding new elements to a Series¶

Modifying Series elements¶

Removing elements from a Series¶

Checking existance of a key (membership)¶

Introducing loc & iloc¶

loc & iloc to modify Series¶

Introducing to Boolean arrays¶

😥 Please, login to continue

Previous attempt

Get in touch with us

Pandas Series - Selection and Indexing
DONE

Adding new elements to a `Series`¶

Modifying `Series` elements¶

Removing elements from a `Series`¶

Introducing `loc` & `iloc`¶

`loc` & `iloc` to modify `Series`¶