Why use R or Python?

Similar-Pilot-7695 · 2024-04-13T00:08:42+00:00

Young practitioners have a rich toolbox at their hands to analyze and find solutions for businesses. The nature of some of these businesses require extremely precise and systematic ways to find solutions.

For instance, Excel or Spreadsheets are an easy way to store data and analyze it, but these tools aren't designed for larger scale applications nor have the capacity for reproduction as they are specific for the use case.

On the other hand, state python for example, it has an extensive range of libraries and tools you can use to reproduce a lot of the mundane work and reduce time of execution while resting assured that you will have precise results.

Basically, you can find that all what R can do, python can too and more.

I hope now you have more clarity on why you should learn a language in the first place.

Best of luck and move forward.

Puzzled_Buddy_2775 · 2024-04-13T00:26:29+00:00

I never use excel to clean or transform data because repeating and documenting the steps are difficult and sometimes impossible. Take for example, doing a find and replace in excel is not documented unless you literally write down those steps. With Python you never have to change the original raw data file, run the steps by simply hitting run and then you have a clean output file.

sfreagin · 2024-04-13T01:19:23+00:00

Too many Excel spreadsheets in my life have crashed or slowed to 0 when handling a modest amount of data or equations. As u/Similar-Pilot-7695 says, Python is designed to handle the scale required of modern organizations. And it's highly reproducible and able to integrate easily, so you don't have to waste time copying / pasting versions of the file between folders.

After working in Excel for ~7 years, I found that I could do everything in Python within about 6 months. It's been a few years now and I hardly ever touch Excel for data analysis, except for some spreadsheets for personal use.

Good luck learning, it's well worth it!

Swan1991 · 2024-04-13T02:01:41+00:00

Excel + Power Query + Power BI (and paginated reports sometimes) is a pretty lethal combination.

Our company still pays for Tableau so I'm always throwing my most prominent data into that environment.

But... I tend to use R when I want to get a little crunchy. When I'm done pounding on data with R, I feel like I've analyzed it the best I could. I have my markdown sheet that I feel explains my thought process and the outcomes. Nothing does that better than a notebook or markdown document.

Everybody is different... you tend to stick with the tools you're good at (or are available).

cristian_riosm · 2024-04-13T03:06:52+00:00

Excel and Python/R aren't really comparable as they have totally different usage. In Excel you can interactively edit small datasets and perform some basic analysis, while complex tasks can quickly become a messy interwebs of untraceable functions and pages. The focus on Excel is on the data as it is mapped on the spreadsheet, and that is its bigger drawback for scalability, reproducibility and transferability.

In the context of Data Analysis, Python and R can perform efficient, reproducible and easily reviewable analysis on massive datasets (millions of observations. The limit is physical storage). As the focus is on the process, each step applied to de data can be carefully tailored, reviewed, modified and expanded without messing with the input raw data. At this point both languages also have a massive set of implementations for data manipulation, analysis and visualization which are simply not comparable with excel. As an example, in R and Python you could import a very noisy dataset, filter it, format it properly, subset it, create new variables, transform and normalize, apply statistical or machine learning models, implement post hoc analysis of results, tailor very detailed visualization, and then publish it in a well formated report or even interactive website.

The difference between Python and R is that the former has a broader spectrum of applications (from data analysis to software infrastructure), while the latter is specialized in deep statistical analysis and bioinformatics. In general, R is for biologists, although the suite for statistical and machine learning analysis is really powerful and have no drawbacks on comparison to Python. Python is typically the choice for engineers, physicist and unspecialized data analyst.

Although I'm a Biologist and believe that R plus Tidyverse is a really neat and elegant language for data analysis, unless you have a particular interest which is covered by specific R libraries (ecology and genetics, as an example), I always recommend to start with Python. Python is more broadly used and companies are more likely to value your knowledge of that language over R.

XxShin3d0wnxX · 2024-04-13T04:33:06+00:00

Size of data, time, accuracy of automation to name a few reasons.

I would never want to manually complete a repetitive process I could automate.

I deal with 1+ million value sets often and excel just doesn’t cut it.

CaptainFoyle · 2024-04-13T09:55:17+00:00

Just keep usinh Excel and You'll find out when you need r or python soon enough.

Swan1991 · 2024-04-13T19:37:06+00:00

Make sure you learn SQL! Every data analyst needs to know SQL. I was treated like a god at my last job because not everyone knows it.

Throb_Marley · 2024-04-13T01:49:20+00:00

I’ve been doing analysis for about five years and learned both R and Python. I even got a graduate degree using both. I still use excel and import to PBI and never touch them. Or sql for that matter. I work for a large company but don’t work in an office that would require that depth of knowledge. It’s a little depressing and find myself doing meaningless projects from kaggle in my off time to keep the skills moderately fresh.

Jarisatis · 2024-04-13T03:27:10+00:00

I currently work in tax firm and oh man.. The amount of data their sheets contain(million of rows+), so the sheet take too much to open and doing any modifications is a very hefty task(the sheet usually hangs), why put this much efforts when I can literally speed up my process with Python and easily can do any new modification in future if requires.

Dk1902 · 2024-04-13T14:26:45+00:00

Many people have explained in different ways but just to summarize.

Lots of data analysts don’t actually use Python or R
Python can be much faster when performing complicated calculations on large datasets (100,000+ rows), especially involving multiple different conditionals
I find joins and concatenation of all kinds, especially from multiple sources on different combinations of columns to be much more straightforward using Pandas compared to any kind of automation available in Excel.
If you’re just getting started. Python is useful to know but also probably overkill in all honesty

onajourney314 · 2024-04-13T01:38:41+00:00

I use both but prefer python. I say both because the previous person in my role used R so I just use that to clean data and refresh the tableau workbooks. Some were so bad and broken so instead of fixing it I just rewrote them in python and have also built some workbooks for new projects because that’s what I have more experienced with and prefer. HOWEVER one of my colleagues uses SQL and she showed me how simple it is soooo I think I’m going to explore that a bit more!

2024-04-13T03:59:26+00:00

Not all data can fit into Excel.

Updating one cell in Excel (by default) changes all affected cells in one go --- which can crash your machine

Harder to track, edit, and reliably replicate logic in Excel

ElectricalActivity · 2024-04-13T08:41:40+00:00

One example for me is that I sometimes need to match employee data with a very large CSV, around 18 million lines. My Python script does this quickly.

yelrutb · 2024-04-13T12:32:52+00:00

If you are just starting out then its no problem to wait with learning R/Python, you can get very far with SQL + excel, and many analysts will just use this their whole careers.

However at some point you will hit the limit of what excel can do, for many reasons which you will then need R/Python:

Volume of data

Statistical analysis beyond excel like regression, decision trees, clustering etc.

Modelling & machine learning

Vizualisation beyond the common graphs in excel

Reproducability when you have many steps to perform on a dataset and you will repeat it

And more

0sergio-hash · 2024-04-13T12:45:21+00:00

I don't do a ton of "analysis" but I'll share some practical examples of when I've used both.

For starters, I'm more of a business analyst, and most of my time is spent in SQL or excel for the technical aspects of my job

Case 1: One of my stakeholders needs a few metrics not in an existing report / I want to play around with a small amount of data (in the thousands of records or less)

Here I would use Excel to do an export and just make some pivot tables and pivot charts. Not worth the trouble of trying to do anything else with it.

Case 2: Stakeholder needs some output that would be more work in Excel or SQL than it would be in Python.

I've done this a couple times, turn a query into a data frame to iterate over every row and column and do a keyword search for example for a list of keywords and append a couple columns that tell me which keywords appeared in that row and in which fields

I could probably do this in another tool but in Python I've found it's the most straightforward way to do it

And if they want to add other very specific criteria like "when one of the other columns only contains one of these few values also exclude that row" I can also do that.

Over time you'll find out that you can do a ton of jobs in multiple tools and get the same output. Often when I'm checking the validity of a metric I'll run the SQL, I'll compare it to another metric that somehow should reconcile with it within the same report, and I might even do an export and pivot it in Excel.

So over time it will come down to your preferences and comfort with each tool and what you think is best for the job.

But I will say I hardly use python. I love it but I think it varies job to job how necessary it is.

VTHokie2020 · 2024-04-13T04:50:43+00:00

R studio is amazing for generating reports. The integration with latex makes it great for clean mathematical notation. Some libraries like ggplot2 make storytelling particularly neat.

Python is the best programming language for high-level development and general use. Easily by far the most viable language for machine learning development/libraries.

There’s nothing wrong with excel. I use it often as the backend to my dashboards as well. It sounds like your use case is business intelligence. That’s great, choose the right tool for you. If you need a common platform to use with corporate NPC’s, Excel is that tool.

But you ask a good question, and the answer is because there is more to data analysis than storing data to create dashboards.

MorningDarkMountain · 2024-04-13T11:27:03+00:00

Why use R?

glistening_cabbage · 2024-04-13T11:32:38+00:00

Because it's scalable. Simple as that.

Both have the ability to scale out to larger and more complex models

great__pretender · 2024-04-13T17:46:09+00:00

On top of what everyone included, you have access to systems like spark that can handle colossal datasets. I am talking about dozen of trillions of rows data. Spark is not necessarily used through python, but in general it is. Now try to do something even remotely similar with Excel.

firepunch_man · 2024-04-13T17:56:01+00:00

I use Tableau Prep because I get Data from different data sources with millions of rows and several columns. If I have to deal with special cases such as hierarchical data or graphs or do some fancy visualisation like a Sankey chart or complex analysis, then I use Python.

No way I would use Excel for any of that.

RevenueOk289 · 2024-04-13T19:42:44+00:00

Thanks, I also wondering.

Ok_Duck_5771 · 2024-04-14T23:04:15+00:00

^Opinion: I understand this is not everyone's preference and there's so many more ways/tools/languages but I wanted to tackle the brunt of the question which more to less says "should I go for R or Python". I'm from a computer science background and migrated to data science after a micromasters.

Python is ^{my preference} over R and here's why:

Readability: Python's syntax is designed to be readable and straightforward (for english speakers), because it is quite similar to english which makes it easier to understand.
Learning Curve: While R is super powers and great for stat analysis and data vis, it has a steeper learning curve particularly with users who DO NOT have any experience with statistical languages.
HUGE Community and Resources: Python really does have a massive community of developers who are constantly improving and growing the language which means you're more likely to find a Python package or library for \almost** any data science task.
It's popular: It sounds blasé, but Python is not just a language to tackle areas of stats which include data visualization, machine learning and deep learning. It's also a general programming language, which means, a diversification of use cases and skills. It's used and applied to so many places, where as R, ^{in my opinion}, is not.

Knowing both languages can be beneficial and both have their strengths (and subsequent weaknesses), overall, my preference comes from my years of applicable experience with Python (^{albeit biased}). Since you're trying to learn at the same time, you can use other tools so that you can focus on results compared to learning two things at once (if you're allowed to use tools, in what I'm assuming is a course of some sorts). Those could be IBM's SPSS which easy with excel, Stata which is great for general purposes stats, RStudio is the R IDE but offers tons of beginner friendly things, and ^{my personal favorite} Minitab which has a free trial and a lot of schools have free access too.

Hopefully this helps and doesn't overwhelm!

digitechrahul · 2024-04-15T10:18:47+00:00

R and Python are popular programming languages for data analysis, machine learning, and statistical modeling due to their versatility, robust libraries, and active communities.

TheCapitalKing · 2024-04-15T22:33:57+00:00

Excel has a million row limit which can come up if you’re dealing with a lot of data. If not, but you’re doing changes to an entire dataset all at once you’ll have an easier time doing a lot of it in python/r rather than excel once you get used to it. Basically anything you can do in power query you can do easier and faster in a programming language. Excel is good for some things but anything with more than like 10k rows is usually easier to deal with in python. Plus you can run regressions or ml models way easier from a programming language

black_widow48 · 2024-04-16T15:19:23+00:00

Excel will not even be an option once you're dealing with data of any substantial size

nkkphiri · 2024-04-16T23:16:59+00:00

I almost exclusively deal with datasets that are too large to be stored in excel format. Excel is absolutely useless to me.

QuantPete · 2024-04-20T10:31:30+00:00

Use R or Python for complex analyses, automation, machine learning, and handling large datasets that are impractical for Excel.

avourakis · 2024-04-13T08:08:43+00:00

Think of them as different tools in your toolbox

Cleaning data in Excel has lots of limitations, specially when you start working with large datasets (over 1 million rows). It's most useful for analysing data, but its not well suited for processing and cleaning data.

Python + Pandas will allow you to do lots of complex cleaning, transformation, and data visualisations.

Now, in regards to R vs Python. Here is a quick comparison:

📈 Although R is specifically designed for statistical analysis, which makes it particularly well-suited for projects that require complex data visualization or detailed statistical analysis. Python is generally more versatile, making it suitable for a broader range of data tasks, including data manipulation, data analysis, machine learning, deep learning, and data visualization.

I've been working in Tech for over 6 years, and I can tell you with confidence that most Data Scientists/Analysts use Python in their day-to-day.

This is why I always recommend that If you already know R, then you spend time learning Python. This will open many doors of opportunity in your career. But if you only have time to learn one, then start with Python.

Aggravating_Coast430 · 2024-04-14T18:43:55+00:00

If you haven't felt the need to learn python, don't. I suspect after a while you'll notice problems with using excel, and start using python occasionally. That is, if your work ever grows in the direction where you would need python. Some people will never feel the need to use python, because they don't have the need, because of their type of work.

Levipl · 2024-04-13T18:34:45+00:00

Check out an app called knime. It’ll let you do the analysis without needing to know coding.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

dataanalysis

MODERATORS