Python Pandas - DataFrame

- A Data frame is a two-dimensional data structure: Features of DataFrames:
  Columns can be of different data types Size is mutable Axes (rows and columns) are labeled (rows only with positional index) Can perform arithmetic operations on rows and columns

Constructor

- A DataFrame is created using the following constructor:
       pandas.DataFrame( data, index, columns, dtype, copy)

          data:    Various forms...
                   e.g.: series, map, lists, dict and DataFrame.

          index:   a list of row labels.
                   Default is np.arange(n)

          columns: a list of column labels.
                   Default is np.arange(n)

          dtype:   the data type of each column.

          copy:    command (or whatever it is) used for copying of data,
                   Default is False.
Examples that show the effects of various parameters in next slide...

Examples of the use of the `DataFrame( )` constructor

     Data = a list of numbers

        data = [10,20,30]

        df = pd.DataFrame(data)

            0
        0  10
        1  20
        2  30

     Data = a list of pairs

        data = [ ['Alex',10], ['Bob',12], ['Clarke',13] ]

        df = pd.DataFrame(data)

                0   1
        0    Alex  10
        1     Bob  12
        2  Clarke  13

DEMO: `progs/df01.py`

Examples of the use of the `DataFrame( )` constructor

     Data = a list of lists

        data = [ ['Alex',10, 1, 1], ['Bob',12, 1, 1], ['Clarke',13, 1, 1] ]

        df = pd.DataFrame(data)

                0   1  2  3
        0    Alex  10  1  1
        1     Bob  12  1  1
        2  Clarke  13  1  1

DEMO: `progs/df01.py`

Examples of the use of the `DataFrame( )` constructor

     index specified:

        data = [ ['Alex',10], ['Bob',12], ['Clarke',13] ]

        df = pd.DataFrame(data, index=['a','b','c'])

                0   1
        a    Alex  10
        b     Bob  12
        c  Clarke  13

     columns specified:

        data = [ ['Alex',10], ['Bob',12], ['Clarke',13] ]

        df = pd.DataFrame(data, columns=['x','y'])

                x   y
        0    Alex  10
        1     Bob  12
        2  Clarke  13

DEMO: `progs/df01.py`

Create a DataFrame from Dict of Lists

     No index specified:

        data = {'Name':['Tom', 'Jack', 'Steve'], 'Age':[28,34,29]}    # Dict of lists

        df = pd.DataFrame(data)

            Name  Age
        0    Tom   28
        1   Jack   34
        2  Steve   29

     With index specified:

        data = {'Name':['Tom', 'Jack', 'Steve'], 'Age':[28,34,29]}    # Dict of lists

        df = pd.DataFrame(data, index=['x','y', 'z'])

            Name  Age
        x    Tom   28
        y   Jack   34
        z  Steve   29

DEMO: `progs/df02.py`

Create a DataFrame from Dict of Lists with a columns parameter

     The columns parameter can change the column order:

        data = {'Name':['Tom', 'Jack', 'Steve'], 'Age':[28,34,29]}    # Dict of lists

        df = pd.DataFrame(data, columns=['Age','Name'])

           Age   Name
        0   28    Tom
        1   34   Jack
        2   29  Steve

     Or select some columns:

        data = {'Name':['Tom', 'Jack', 'Steve'], 'Age':[28,34,29]}    # Dict of lists

        df = pd.DataFrame(data, columns=['Name'])

            Name
        0    Tom
        1   Jack
        2  Steve

DEMO: `progs/df02.py`

Create a DataFrame from a List of Dicts

     No index specified:

        data = [ {'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20} ]   # List of dict

        df = pd.DataFrame(data)

           a     b     c
        0  1     2   NaN
        1  5    10  20.0

     With index specified:

        data = [ {'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20} ]   # List of dict

        df = pd.DataFrame(data, index=['x','y'])

           a     b     c
        x  1     2   NaN
        y  5    10  20.0

DEMO: `progs/df03.py`

Create a DataFrame from a List of Dicts with a columns parameter

     The columns parameter can change the column order:

        data = [ {'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20} ]   # List of dict

        df = pd.DataFrame(data, columns=['c','b','a'])

              c    b    a
        0   NaN    2    1
        1  20.0   10    5

     Or select some columns:

        data = [ {'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20} ]   # List of dict

        df = pd.DataFrame(data, columns=['c','a'])

              c    a
        0   NaN    1
        1  20.0    5

DEMO: `progs/df03.py`

Selecting one or more columns

     Use: df['colName']

        data = [ ['Alex',10], ['Bob',12], ['Clarke',13] ]

        df = pd.DataFrame(data, columns=['x','y'])

        print( df['x'] )

        0      Alex
        1       Bob
        2    Clarke

     Or: df[['col1', 'col2', ..]]

        print( df[ ['y','x'] ] )

            y       x
        0  10    Alex
        1  12     Bob
        2  13  Clarke

DEMO: `progs/df04.py`

Adding and deleting a column

     Add a column:   df['newCol'] = [ x1, x2, ... ]  # Must have correct length

        data = [ ['Alex',10], ['Bob',12], ['Clarke',13] ]

        df = pd.DataFrame(data, columns=['x','y'])

        df['z'] = [10,20,30]

                x   y   z
        0    Alex  10  10
        1     Bob  12  20
        2  Clarke  13  30

     Delete a column:   del df['colName']  or  df.pop('colName')

        del df['y']

                x   z
        0    Alex  10
        1     Bob  20
        2  Clarke  30

DEMO: `progs/df04.py`

Selecting a row by label

     Use: df.loc['rowIndex']   or  df.loc[['row1', 'row2', ...]]

        data = [ ['Alex',10], ['Bob',12], ['Clarke',13] ]

        df = pd.DataFrame(data, columns=['x','y'], index=['a','b','c'])

        df:
                x   y
        a    Alex  10
        b     Bob  12
        c  Clarke  13

        print( df.loc['b'] )              print( df.loc[ ['b','a'] ] )

        df.loc['b']:                      df.loc[ ['b','a'] ]:
                    x    Bob			              x   y
                    y     12                                b   Bob  12
                                                            a  Alex  10

Selecting a row by iloc

     Use: df.iloc[n]    or   df.iloc[[n1, n2, ...]]

        data = [ ['Alex',10], ['Bob',12], ['Clarke',13] ]

        df = pd.DataFrame(data, columns=['x','y'], index=['a','b','c'])

        df:
                x   y
        a    Alex  10
        b     Bob  12
        c  Clarke  13

        print( df.iloc[1] )               print( df.iloc[ [1,0] ] )

        df.iloc[1]:                       df.iloc[ [1,0] ]:
                    x    Bob			              x   y
                    y     12                                b   Bob  12
                                                            a  Alex  10

     Rows can be sliced because it uses numeric indexes:

       df.iloc[0:2]:                     x   y
                                   a  Alex  10
    			       b   Bob  12

Adding and deleting a row

     Add a new row to a DataFrame:

        (1) Create a new DataFrame:

             a = {'x':['SY'], 'y':[99]}          # New row
             b = pd.DataFrame(a, index=['d'])    # Create DataFrame with row

        (2) Concat to existing DataFrame:

             df = pd.concat([df, b])

        df:
                x   y
        a    Alex  10
        b     Bob  12
        c  Clarke  13
        d      SY  99

     Delete a row from a DataFrame:

        (1) Use label:                df = df.drop( 'b' )

        (2) Use name at iloc[ ]:      df = df.drop( df.iloc[1].name )

❮ ❯