Data Reading and Data Inspection Using Pandas

In the previous article, we have discussed what pandas is? It’s importance in data science, how to install it, and perform basic operations like adding and deleting index, rows, and columns in a DataFrame. Now we will dive deeper into the applications of pandas in real-time situations like Data Reading and Data Inspection Using Pandas.

As a data scientist or an analyst, you’ll probably come across many file types to import and use in your Python scripts. Some analysts use Microsoft Excel, but the application limits what you can do with large data imports. The better option is pandas. It is a powerful analysis toolkit that’s much more intuitive for a data scientist.

Data Reading and Data Inspection Using Pandas

Data reading and data inspection are the first two essential steps in any data analysis or machine learning project. Before performing calculations, creating visualizations, or building predictive models, you need to load your dataset correctly and understand its structure. The Data Reading and Data Inspection Using Pandas library in Python provides powerful functions that make these tasks simple and efficient.

Data Reading Using Pandas

Data Reading and Data Inspection Using Pandas supports reading data from multiple file formats, making it a versatile tool for data professionals. The most commonly used functions include Data Reading and Data Inspection Using Pandas:

• read_csv() – Reads data from CSV files.
• read_excel() – Imports data from Excel spreadsheets.
• read_json() – Loads data stored in JSON format.
• read_sql() – Retrieves data directly from SQL databases.
• read_html() – Extracts tables from HTML web pages.

These functions convert raw data into a DataFrame, allowing you to manipulate and analyze it using Pandas’ built-in features.

Data Inspection Using Pandas

After loading the dataset, data inspection helps you understand its contents, identify missing values, verify data types, and detect inconsistencies. This step ensures your data is clean and suitable for analysis.

Some commonly used inspection methods include Data Reading and Data Inspection Using Pandas:

• head() – Displays the first few rows of the dataset.
• tail() – Shows the last few rows.
• info() – Provides information about columns, data types, and missing values.
• describe() – Generates statistical summaries for numerical columns.
• shape – Returns the number of rows and columns.
• columns – Lists all column names.
• dtypes – Displays the data type of each column.
• isnull().sum() – Counts missing values in each column.

Using these methods, analysts can quickly evaluate data quality and determine the preprocessing steps required before further analysis.

Together, Data Reading and Data Inspection Using Pandas provide a solid foundation for data cleaning, transformation, visualization, and machine learning. By understanding the dataset early in the workflow, you can avoid errors, improve data quality, and make more accurate, data-driven decisions.

Data Reading and Data Inspection Using Pandas is crucial for ensuring data quality and integrity in your analyses.

What file formats can pandas use…?

Python can handle virtually any data file format much more than Microsoft Excel. That’s the strength of Python. It’s open-source, and there’s probably a library out there to handle it, so you get a vastly more compatible system. 

These are the most common types of Data which we will come across

• Comma-separated values (CSV)

• XLSX

• JSON

• XML

• HTML

• Images

• PDF

• DOCX

• SQL

How to read and write tabular data ?

Now we will learn to read and write Data Reading and Data Inspection Using Pandas functions. We will use pandas read_csv() and .to_csv() functions

Data Reading and Data Inspection Using Pandas. A comma-separated values (CSV) file is a plaintext file with a .csv extension that holds tabular data. This is one of the most popular file formats for storing large amounts of data. Each row of the CSV file represents a single table row. The values in the same row are by default separated with commas, but you could change the separator to a semicolon, tab, space, or some other character.

Write a CSV File

You can save your Pandas DataFrame as a CSV file with .to_csv():

df.to_csv(‘data.csv’)

That’s it! You’ve created the file data.csv in your current working directory. You can expand the code block below to see how your CSV file should look:

data.csv

Read a CSV File

Once your data is saved in a CSV file, you’ll likely want to load and use it from time to time. You can do that with the Pandas read_csv() function:

df = pd.read_csv(‘data.csv’, index_col=0)

df

COUNTRY	POP	AREA	CONT	IND_DAY
China	1398.72	9596.96	Asia	NaN
India	1351.16	3287.26	Asia	1947-08-15
US	329.74	9833.52	N.America	1776-07-04
Indonesia	268.07	1910.93	Asia	1945-08-17

Write an Excel File

You can save your Pandas DataFrame as a CSV file with .to_excel():

df.to_csv(‘data.csv’)

Read an Excel File

You can do that with the Pandas read_excel() function:

df = pd.read_excel('data.xlsx', index_col=0)

Write an Json File

You can save your Pandas DataFrame as a CSV file with .to_json():

df.to_json('data-index.json', orient='index')

Read an Json File

You can do that with the Pandas read_json() function:

df = pd.read_json('data.xlsx', index_col=0)

Write Files

Series and DataFrame objects have methods that enable writing data and labels to the clipboard or files. They’re named with the pattern .to_<file-type>(), where <file-type> is the type of the target file.

You’ve learned about .to_csv() and .to_excel(), but there are others, including:

• .to_json()

• .to_html()

• .to_sql()

• .to_pickle()

There are still more file types that you can write to, so this list is not exhaustive.

Read Files

Data Reading and Data Inspection Using Pandas: Pandas functions for reading the contents of files are named using the pattern .read_<file-type>(), where <file-type> indicates the type of the file to read. You’ve already seen the Pandas read_csv() and read_excel() functions. Here are a few others:

• read_json()

• read_html()

• read_sql()

• read_pickle()

These functions have a parameter that specifies the target file path. It can be any valid string that represents the path, either on a local machine or in a URL. Other objects are also acceptable depending on the file type.

How to view and inspect data in a DataFrame ?

For checking the Data Reading and Data Inspection Using Pandas. DataFrame and pandas. Series with many rows and columns head() and tail() methods are useful.

Now we will use Iris Data set from kaggle for this tutorial

“ https://www.kaggle.com/uciml/iris “ 

import pandas as pd 

df = sns.load_dataset("iris")

Get first n rows of DataFrame: head()

The head() method returns the first n rows.

print(df.head(5))

	sepal_length	sepal_width	petal_length	petal_width	species
0	5.1	3.5	1.4	0.2	setosa
1	4.9	3.0	1.4	0.2	setosa
2	4.7	3.2	1.3	0.2	setosa
3	4.6	3.1	1.5	0.2	setosa
4	5.0	3.6	1.4	0.2	setosa

Get first n rows of DataFrame: tail()

The tail() method returns the first n rows.

print(df.tail(5))

	sepal_length	sepal_width	petal_length	petal_width	species
145	6.7	3.0	5.2	2.3	virginica
146	6.3	2.5	5.0	1.9	virginica
147	26.5	3.0	5.2	2.0	virginica
148	6.2	3.4	5.4	2.3	virginica
159	5.9	3.0	5.1	1.8	virginica

Pandas .shape,.size and .ndim are used to return size, shape and dimensions of data frames and series.

Create a DataFrame

import pandas as pd
import numpy as np
d={‘Name’:pd.Series(['Tom','James','Ricky','Vin',    'Steve']),'Age':pd.Series([25,26,25,23,30]),'Rating':pd.Series([4.23,3.24,3.98,2.56,3.20])}
#Create a DataFrame 
df = pd.DataFrame(d)
print df

Output:

	Age	Name	Rating
0	25	Tom	4.23
1	26	James	3.24
2	25	Ricky	3.98
3	23	Vin	2.56
4	30	Steve	3.20

.shape Returns a tuple representing the dimensionality of the DataFrame. Tuple (a,b), where a represents the number of rows and b represents the number of columns.

df.shape

Output:

(5, 3) //5 Rows & 3 Columns

.size Returns the number of elements in the DataFrame.

df.size

Output: 

21 // The total number of elements in our object is:

.ndim Returns the number of dimensions of the object. By definition, DataFrame is a 2D object

df.ndim

Output: 

2 // The dimension of the object is

Pandas .info() function is used to print a concise summary of a DataFrame. This method prints information about a DataFrame including the index dtype and column dtypes, non-null values and memory usage.

Consider the following DataFrame df

	int_col	text_col	float_col
0	1	alpha	0.00
1	2	beta	0.25
2	3	gamma	0.50
3	4	delta	0.75
4	5	Epsilon	1.00

df.info()

Output:

<class ‘pandas.core.frame.DataFrame’>  
RangeIndex: 5 entries, 0 to 4          
Data columns (total 3 columns):        
float_col    5 non-null float64        
int_col      5 non-null int64          
text_col     5 non-null object         
dtypes: float64(1), int64(1), object(1)
memory usage: 192.0+ bytes             

Data Reading and Data Inspection Using Pandas: The Pandas .describe() function computes a summary of statistics pertaining to the DataFrame columns. This function gives the mean, std and IQR values. And, function excludes the character columns and given summary about numeric columns

df.describe()

Output:

	float_col	int_col
count	5.000000	5.000000
mean	0.500000	3.000000
std	0.395285	1.581139
min	0.000000	1.000000
25%	0.250000	2.000000
50%	0.500000	3.000000
75%	0.750000	4.000000
max	1.000000	5.000000

Data Reading and Data Inspection Using Pandas: Pandas .value_counts() function returns object containing counts of unique values. The resulting object will be in descending order so that the first element is the most frequently-occurring element. Excludes NA values by default.

Consider a DataFrame

	Student
0	Harry
1	Mike
2	Arther
3	Harry
4	Arther

Output:

Harry     2 
Arther    2 
Mike      1 
Nick      1 
Name: Student, dtype: int64

Data Reading and Data Inspection Using Pandas is a huge concept where we will learn all its components in a step by step manner. In the next article we will discuss data selection, data cleaning, filtering, sorting, group-by, joining and combining of the dataset.

Conclusion

Data Reading and Data Inspection Using Pandas are fundamental skills for anyone working with Python for data analysis. Reading data correctly ensures your dataset is loaded without issues, while inspecting it helps you understand its structure, identify missing values, verify data types, and spot inconsistencies before analysis begins. By Data Reading and Data Inspection Using Pandas mastering Pandas functions such as read_csv(), head(), info(), and describe(), you can prepare your data efficiently and build a strong foundation for data cleaning, visualization, and machine learning projects.

FAQs: about Data Reading and Data Inspection Using Pandas