How to work with DataFrame in Python?

Once we data in DataFrame i.e. DataFrame is prepared, independent of source of data (csv,xls,db etc.) we can work with it, like it is a table in database, selecting element of our interest.

import pandas as pd
df1 = pd.read_csv('example.csv')
df1.head()
df1.shape

We have loaded data from csv file and created DataFrame. Now, we will see different select operation on this DataFrame.

Selecting a single column in DataFrame

column1 = df1['column_name']
column1.head()

Selecting multiple columns in DataFrame

cols = df1['column1','column2']
cols.head()

Selecting rows using indexing [] in DataFrame

rows = df1[10:20]
print (rows)

rows = df1[10:20]['column1','column2']
print (rows)

Selecting rows by lable (.loc[]) in DataFrame

df2 = df1.loc[10:20]
df3 = df1.loc[10:20,['column1','column2']]
print(df2)
print(df3)

Selecting rows by position (.iloc[]) in DataFrame

dfpos = df1.iloc[10:20,[3,4]] 

After looking at fetching required set of column we proceed to manipulation of DataFrame

How to manipulate a DataFrame

1. Transpose

import pandas as pd
df1 = pd.read_csv('example.csv')
df2 = df1[10:20]['cols1','cols2']
print("transpose : {}".format(df2.T))

2. sort_values

df2 = df1.sort_values(by='column_name') # sorting by single column
df3 = df1.sort_values(by=['col1','col2']) # sorting by multiple column
print(df2)
print(df3)

3. sort_index

df2 = df1.sort_index()
print(df2)

4. Re-indexing

import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.random.randn(3),index=['a','b','c'])
print(df1)
df2 = df1.reindex([1,2,3])
print(df2)

5. Adding a new column

import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.random.randn(10))
print(df1)
df1['col_new'] = 'a'
print(df1)

6. Remove existing column

import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.random.randn(10))
print(df1)
df1['col_new'] = 'a'
print(df1)

del df1['col_new']  # del df1[1] will work same

7. Data at particular location by label

import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.random.randn(3,5),index=[1,2,3],columns=['a','b','c','d','e'])
val1 = df1.at(1,'a')
print(val1)
df1.at(1,'a') = 0 # assign value at particular location

8. Data at particular location by position

import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.random.randn(3,5),index=[1,2,3],columns=['a','b','c','d','e'])
val1 = df1.iat[1,1]
print(val1)
df1.at[1,1] = 0 # assign value at particular location

df1[df1>0] = 2 # assign data at all location based on a condition

9. Applying a method or function in DataFrame

import pandas as pd
import numpy as np
df1 = pd.DataFrame(np.random.randn(3,5),index=[1,2,3],columns=['a','b','c','d','e'])

def add_five(number):
    return number+5

df.apply(add_five,axis=2)

There are few more functions such as dropna(),fillna() etc. which is used in manipulation of data. DataFrame also have many statiscal methods like info(),describe(),value_counts(),mean(),std() etc.

We will learn filtering and iterating in DataFrame in upcoming post.

Keep learning!

Hope it helps 🙂