Exploratory Data Analysis With R

The way I actually do EDA in R

I stopped trying to write long scripts for exploratory data analysis years ago. The habit I kept was building a small repeatable toolkit and running it on everything. It does not look like production code. It is fast, it is ugly, and it saves hours because you see the problem before you spend a week on modelling. It is a workflow, not a single technique. You load a dataset, inspect the shape, check types, look at distributions, flag missing values, and map relationships. The goal is to learn what the data is doing before you commit to any model. That shift in mindset matters more than the packages. In practice I treat the first thirty minutes as a health check. I check if the column I think is a date is actually stored as character data. I verify whether the target variable has the right distribution. I look at row counts before and after removing rows with missing values. Small details break projects later.

I do not rely on a single package. I move between base R, dplyr for manipulation, and a few visualisation tools depending on the situation. The choice changes the speed of the workflow more than anything else.

Getting started without overcomplicating it

R is free and it runs on Windows, macOS, and Linux. The standard way to get it is through CRAN. You download the base system first, then add packages as you need them. If you prefer an integrated editor, RStudio is the most common choice, but any text editor works after you are comfortable with the console. I install packages at the start of a project and keep a short list so I can reproduce the environment later. A minimal set looks like this: tidyverse for general data work
readr and readxl for reading files
janitor for cleaning column names
naniar for visualising missing data
GGally for quick relationship plots

Running the install takes less than two minutes on a normal connection. The real time savings come from writing small helper functions instead of repeating the same blocks.

A practical workflow I actually use

I start with reading the file. The first step is always checking whether the import did what I expected. Sometimes headers are shifted, sometimes blank rows appear at the top, and sometimes numeric columns are imported as character data because of a comma inside a field. I confirm types immediately. Next I check dimensions. I look at nrow and ncol to confirm the size. I run a quick str call to see the structure. I print the column names and scan them for unexpected values. This part takes about five minutes on a medium dataset. After that I inspect missing values. I do not jump straight to imputation. I want to understand the pattern first. I count missing values by column, plot them if the dataset is large enough, and decide whether the missingness is random or systematic. Missing data that correlates with the outcome often requires a different approach than missing data that is uniform across features.

Then I move to distributions. For numeric variables I check summary statistics and look at histograms or density plots. For categorical variables I check unique value counts. High-cardinality categories often create problems later, so I flag them early. Relationships come after distributions. I build a correlation matrix for numeric variables and look at scatter plots for pairs that matter. I avoid cluttering the screen with every possible combination and focus on the variables that have a theoretical link to the target or to known business drivers.

Code structure that survives real projects

Here is a straightforward example that mirrors what I run by default: The next step is usually the visual layer. I build a ggpairs call to scan relationships quickly, then drill down into specific plots. The quick view catches obvious non-linear patterns and outliers that summary tables hide. For larger datasets I switch to sampling or aggregation before plotting. Rendering fifty thousand points on a scatter plot is rarely useful. I sample ten thousand points or use geom_hex to show density instead.

Two things beginners miss

First, factor levels matter more than most people expect. If you leave factors unordered when they should be ordered, your plots will look wrong and your modelling steps may behave unexpectedly. I set levels explicitly after cleaning. It prevents subtle errors later. Second, missing-value imputation is not always the best first move. Dropping rows with missing target values is usually safe. Dropping rows with missing predictors can remove too much data or introduce selection bias. I test both approaches with a small model to see the impact before committing.

A specific edge case I ran into

I was working on a dataset where one categorical variable had thousands of unique values because it contained individual identifiers mixed into a category that should have been grouped. The column looked normal in the head output. The first fifty rows were fine. At row ten thousand there were entries like product_sku_7721 embedded in what should have been region-level codes. I found it by sorting the unique values and looking at the tail. I then created a cleaning rule that kept only values matching a known valid set and flagged the rest as unknown. The workaround was simple after detection, but the detection itself would have been easy to miss if I had only glanced at the first few rows. This is the reason I always check high-cardinality variables early. It usually reveals data-entry issues, merged columns, or encoding mistakes that destroy downstream work.

Performance limits you should know about

R handles millions of rows fine for most exploratory tasks, but memory usage grows quickly with wide tables. A dataset with thousands of columns can consume a lot of RAM during basic operations. I monitor memory with pryr::object_size when I suspect a problem, and I switch to lazyframe or data.table operations when the dataset gets large. Another bottleneck is repeated plotting in interactive sessions. Saving plots to disk with ggsave is faster than rerendering them every time you tweak the code. I save intermediate visuals during the exploratory phase so I can return to them without rebuilding the chain.

When R is not the right choice

If your dataset is very large and you only need quick previews, Python with pandas often feels faster for raw manipulation. If you are collaborating with a team that already uses Python tooling, switching to R for a small exploratory task adds overhead. In those cases I use R only for visualisation and statistical checks, then move the main pipeline elsewhere. R shines when you want reproducible analysis combined with strong statistical testing and publication-quality plots. It is slower for raw data engineering than systems built for that purpose. Accepting that boundary prevents frustration.

Common pitfalls I still see

People often skip the type check and assume numeric columns are numeric. They then run models and get confusing results because factors or characters entered the calculation. Always verify types before summarising or modelling. Another frequent mistake is treating every outlier as an error. Outliers are information. I mark them, investigate them, and decide whether to keep or transform them. I rarely remove them without documenting why. I also see people build complex visualisations too early. Simple tables and basic plots reveal more during the first pass than polished figures ever will. Clarity beats presentation until the project reaches a stable point.

Resources and links

The official R project site is r-project.org. CRAN hosts the package repository. RStudio has good documentation at posit.co/download. For focused reading on exploratory analysis practices, the R for Data Science book covers the workflow I described, though I mostly consult it for specific function behaviour rather than reading it cover to cover. If you want a quick reference for missing-data visualisation, the naniar documentation is direct. For general tidyverse conventions, the tidyverse.org site is the standard source.

Final note on the process

Exploratory Data Analysis With R works best when you treat it as an investigative loop, not a linear pipeline. You inspect, you question, you clean, you visualise, and then you inspect again. Each pass reveals something new. The speed comes from keeping the loop tight and the tools consistent. I do not claim this is the only way to do it. It is the way that has survived repeated projects with real deadlines. You will adapt the details to your own data, but the sequence stays the same. Read the data, check the shape, inspect the missingness, explore the distributions, map the relationships, and document what you find before moving on.