I am putting the topics I have been working on into a series that I will tell you one by one. For this reason, I will explain the methodology and usage aspects of almost all libraries that I am actively working on. I’ll start with pandas, which allows functional operations such as data preprocessing without reading data first. With this library, we can easily perform data pre-processing steps, which are vital steps for data science, such as observing missing data and extracting that data from the data set. In addition, you can bypass data types and the front part of numerical or categorical operations that you will do on them. This provides us with significant convenience before proceeding. Each library in Python has its own specialties, but speaking for pandas, it is responsible for all of the pre-part modifications to the data to form the basis of the data science steps. Data classification processes in Pandas can be designed and activated quickly with a few functional codes. This is the most critical point in the data preprocessing stage, in the previous steps of data modeling.
We can store the data as “dataframe” or “series” and perform operations on it. The fact that Pandas library performs every operation on data in a functional, easy and fast way reduces the workload in data science processes on behalf of data scientists. In this way, it can handle steps such as the beginning and most difficult part of the process, such as data preprocessing, and focus on the last steps of the job. By reading data such as .csv, .xlsx, .json, .txt prepared in different types, it takes the data that has been entered or collected through data mining into python to process. Pandas library, which has the dataframe method, is more logical and even sustainable than other libraries in terms of making the data more functional and scalable. Those who will work in this field should work on the methodology of pandas library, which has the basic and robust structure of the python programming language, not to write code directly. Because new assignments on the data, column names, grouping variables, removing empty observations from the data or filling empty observations in a specific way (mean, 0 or median assignment) can be performed.
Data cannot be processed or analyzed before the Pandas library is known. To be clear, the pandas library can be called the heart of data science. Specially designed functions such as apply (), drop (), iloc (), dtypes () and sort_values () are the most important features that make this library exclusive. It is an indispensable library for these operations, even if it is not based here on the basis of its original starting point. In the steps to be taken, it has a structure with tremendous features and a more basic case in terms of syntax. It is possible to host the results from the loops in clusters and convert them into dataframe or series. The acceleration of the processes provides a great advantage in functional terms if the project that will emerge has a progressing process depending on time, which is generally the case. Looking at its other possibilities, it is one of the most efficient libraries among the python libraries. The fact that it is suitable for use in many areas can be considered as a great additional feature. Pandas is among the top 3 libraries in the voting among data processing libraries made by software developers using the python programming language. You can reach this situation, which I quoted with datarequest in the sources section.
The concept of “data science”, which has been developing since 2015, has brought the pandas library to the forefront and this library, which has been developing in silence for years, has come to light. After Pandas, I will explain numpy and talk about numerical and matrix operations. In general, Pandas is a library that has high-level features in basic data analysis and data processing. In addition, if you specify the topics you will talk about and the things you want me to mention, I will draw a more solid way in terms of efficiency. I hope these articles that I will publish in series will help people who will work in this field. In the future, I will add the cheatsheet style contents that I will prepare on github to the bibliography section. If you want to take advantage of such notes, I will put my github account in the resource section, and you can easily access there.
– https://www.dataquest.io/course/pandas-fundamentals/Python Data Science Libraries 1 – Pandas Methodology