Pandas in Python, with Examples – Part I – Introduction

You can download the project for this post by clicking HERE.

Introduction

Pandas is a Python library for data analysis that provides flexible data structures that let you work with your data very efficiently. Pandas offers the following data structures:

• Series: One-dimensional indexed arrays (arrays with an index, or labeled arrays), similar to dictionaries. They can be created from dictionaries or from lists.
   
• DataFrame: Data structures similar to the tables of relational databases such as SQL.
   
• Panel, Panel4D and PanelND: These data structures let you work with more than two dimensions. Since working with arrays of more than two dimensions is somewhat complex and rarely used, we won’t cover panels in these introductory Pandas tutorials.

To follow the examples we are going to show in this tutorial and in the rest of the ones related to the Pandas library, you need to install this library (and the Numpy library). You can get them from the PyPI package repository (with pip) like this:

$ pip install pandas
$ pip install numpy

Next, let’s look at some basic examples with Series and DataFrame:

The first thing we have to do is import the Pandas library:

import pandas as pd

By convention (in the developer community), "pd" is used as the alias for the Pandas library.

The first example will be defining a "Series" data structure, which, as we mentioned, is a one-dimensional data array with an index. "Series" are defined like this:

serie = pd.Series(data, index=index)

That is, in the first parameter we pass the array data and in the second parameter the indexes. Let’s look at an example of how to create a "Series" structure with the members of the Spanish national football team that won the 2010 World Cup, where the ‘data’ is their names and the index is their shirt number:

spanishPlayers = pd.Series(
    ['Casillas', 'Ramos', 'Pique', 'Puyol', 'Capdevila', 'Xabi Alonso', 'Busquets', 'Xavi Hernandez', 'Pedrito',
     'Iniesta', 'Villa'], index=[1, 15, 3, 5, 11, 14, 16, 8, 18, 6, 7])
print "Spanish Football Players: \n%s" % spanishPlayers

The output of this code snippet, which prints the Series to the screen, is the following, and in it we can see what we just described:

Spanish Football Players: 
1           Casillas
15             Ramos
3              Pique
5              Puyol
11         Capdevila
14       Xabi Alonso
16          Busquets
8     Xavi Hernandez
18           Pedrito
6            Iniesta
7              Villa
dtype: object

In the next case, where we don’t specify the indexes explicitly, it will generate the indexes automatically, starting from zero:

spanishPlayers = pd.Series(
    ['Casillas', 'Ramos', 'Pique', 'Puyol', 'Capdevila', 'Xabi Alonso', 'Busquets', 'Xavi Hernandez', 'Pedrito',
     'Iniesta', 'Villa'])
print "Spanish Football Players: \n%s" % spanishPlayers

The output is the following:

Spanish Football Players: 
0           Casillas
1              Ramos
2              Pique
3              Puyol
4          Capdevila
5        Xabi Alonso
6           Busquets
7     Xavi Hernandez
8            Pedrito
9            Iniesta
10             Villa
dtype: object

We can also create a "Series" data structure from a list or from a dictionary. If we build it from a list, it will use the default indexes, and if we create it from a dictionary, it will use the keys as indexes. Next, let’s see an example of how to create a Series from a dictionary, and also how to insert a new element into that series:

dictPlayers = {1: 'Casillas', 15: 'Ramos', 3: 'Pique', 5: 'Puyol', 11: 'Capdevila', 14: 'Xabi Alonso',
               16: 'Busquets', 8: 'Xavi Hernandez', 18: 'Pedrito', 6: 'Iniesta', 7: 'Villa'}
players2series = pd.Series(dictPlayers)
# Insert new player
players2series[10] = 'Cesc'
print "Spanish Football Players through dictionary: \n%s" % players2series

The output of this code snippet is the following:

Spanish Football Players through dictionary: 
1           Casillas
3              Pique
5              Puyol
6            Iniesta
7              Villa
8     Xavi Hernandez
11         Capdevila
14       Xabi Alonso
15             Ramos
16          Busquets
18           Pedrito
10              Cesc
dtype: object

Next, let’s look at an example with the "DataFrame" data structure. As we mentioned, it is a data structure similar to a relational database table, an Excel spreadsheet, etc., and as such you can perform many operations on it, like the ones you would do with queries on database tables or in Excel (you can see these kinds of operations in the following tutorials: "DataFrame: Reading and Writing, Merging DataFrames and GroupBy" and "Pivot_table Operations with DataFrames").

A DataFrame can be built in different ways, for example from a list, a dictionary, a Series, another DataFrame, by reading an Excel table, a CSV, etc. Next, let’s see how we would build a DataFrame with data about the members of the Spanish national football team:

spanishPlayersDF = pd.DataFrame(
    {
        'name': ['Casillas', 'Ramos', 'Pique', 'Puyol', 'Capdevila', 'Xabi Alonso', 'Busquets', 'Xavi Hernandez',
                 'Pedrito', 'Iniesta', 'Villa'],
        'demarcation': ['Goalkeeper', 'Right back', 'Centre-back', 'Centre-back', 'Left back', 'Defensive midfield',
                        'Defensive midfield', 'Midfielder', 'Left winger', 'Right winger', 'Centre forward'],
        'team': ['Real Madrid', 'Real Madrid', 'FC Barcelona', 'FC Barcelona', 'Villareal', 'Real Madrid',
                 'FC Barcelona', 'FC Barcelona', 'FC Barcelona', 'FC Barcelona', 'FC Barcelona']
    }, columns=['name', 'demarcation', 'team'], index=[1, 15, 3, 5, 11, 14, 16, 8, 18, 6, 7]
)

We can see that, first, we pass a dictionary as a parameter whose keys are the column names and whose values are a list with the data each column will hold for each row. Second, we pass the column names, which match the keys of the dictionary with the data, and finally the index of each of the rows. If we print the contents of this DataFrame, we get the following:

              name         demarcation          team
1         Casillas          Goalkeeper   Real Madrid
15           Ramos          Right back   Real Madrid
3            Pique         Centre-back  FC Barcelona
5            Puyol         Centre-back  FC Barcelona
11       Capdevila           Left back     Villareal
14     Xabi Alonso  Defensive midfield   Real Madrid
16        Busquets  Defensive midfield  FC Barcelona
8   Xavi Hernandez          Midfielder  FC Barcelona
18         Pedrito         Left winger  FC Barcelona
6          Iniesta        Right winger  FC Barcelona
7            Villa      Centre forward  FC Barcelona

The result is a data structure similar to a relational database table or an Excel or CSV document. With this data structure you can perform many operations like the ones we would do in a database or in an Excel document.

Finally, let’s see how to insert a new element into this DataFrame, which we would do like this with the "loc()" method:

# Insert new player
spanishPlayersDF.loc[10] = ['Cesc', 'Forward', 'Arsenal']

We specify the index the row should have in the table (as we would with a dictionary) and then we give the column values in a list. As a result, we would have a new element in the DataFrame:

Spanish Football Players DataFrame: 
              name         demarcation          team
1         Casillas          Goalkeeper   Real Madrid
15           Ramos          Right back   Real Madrid
3            Pique         Centre-back  FC Barcelona
5            Puyol         Centre-back  FC Barcelona
11       Capdevila           Left back     Villareal
14     Xabi Alonso  Defensive midfield   Real Madrid
16        Busquets  Defensive midfield  FC Barcelona
8   Xavi Hernandez          Midfielder  FC Barcelona
18         Pedrito         Left winger  FC Barcelona
6          Iniesta        Right winger  FC Barcelona
7            Villa      Centre forward  FC Barcelona
10            Cesc             Forward       Arsenal

CONCLUSIONS:

The goal of this tutorial is to introduce the two most widely used data structures (Series and DataFrames) in the Pandas library, and as such we have only shown very basic examples. You can do a lot more; we explain some of it in the tutorials "DataFrame: Reading and Writing, Merging DataFrames and GroupBy" and "Pivot_table Operations with DataFrames". If you want to learn more about the Pandas library, you can check its extensive and detailed documentation at the following link: pandas documentation — pandas 3.0.6 documentation