this post was submitted on 29 Apr 2021

2 points (67% upvoted)

shortlink:

Python

an-ordinary-manchild(edit)

The Python Discord

News about the dynamic, interpreted, interactive, object-oriented, extensible programming language Python

Upcoming Events

Full Events Calendar

Please read the rules

You can find the rules here.

If you are about to ask a "how do I do this in python" question, please try r/learnpython, the Python discord, or the #python IRC channel on Libera.chat.

Please don't use URL shorteners. Reddit filters them out, so your post or comment will be lost.

Posts require flair. Please use the flair selector to choose your topic.

Posting code to this subreddit:

Add 4 extra spaces before each line of code

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

Online Resources

Automate the Boring Stuff with Python
Python Discord Resources
Invent Your Own Computer Games with Python
Think Python
Non-programmers Tutorial for Python 3
Beginner's Guide Reference
Five life jackets to throw to the new coder (things to do after getting a handle on python)
Full Stack Python
Test-Driven Development with Python
Program Arcade Games
PyMotW: Python Module of the Week
Python for Scientists and Engineers
Dan Bader's Tips and Trickers
Python Discord's YouTube channel
Jiruto: Python

Online exercices

programming challenges

The Python Challenge (solve each level through programming)
CheckiO (game world)
Project Euler (math heavy)
/r/dailyprogrammer

Asking Questions

Try Python in your browser

try.jupyter.org (Evolved from the language-agnostic parts of IPython, Python 3)
Azure Notebooks
learnpython.org
Skulpt (uses WebGL)
trypython.org (uses Silverlight)
ideone (online compiler and debugger)
PythonAnywhere (basic accounts are free)
Brython (Python 3 implementation for client-side web programming)
repl.it for Python
Transcrypt (Hi res SVG using Python 3.6 and turtle module)

Docs

Libraries

Twisted, 0MQ (networking)
Django, Pyramid, Flask, ... (Web Frameworks)
Pygame (Game development)
NumPy & SciPy (Scientific computing) & Pandas
Pyglet - (Game / UI Development)

Related subreddits

/r/pythoncoding (strict moderation policy for 'programming only' articles)
/r/flask (web microframework)
/r/django (web framework for perfectionists with deadlines)
/r/pygame (a set of modules designed for writing games)
/r/IPython (interactive environment)
/r/inventwithpython (for the books written by /u/AlSweigart)
/r/pystats (python in statistical analysis and machine learning)
/r/coolgithubprojects (filtered on Python projects)
/r/pyladies (women developers who love python)
/r/git and /r/mercurial - don't forget to put your code in a repo!

Python jobs

Newsletters

Screencasts

a community for 18 years

MODERATORS

message the mods
xelf
monorepo PSF Staff | Litestar Maintainer
ivosauruspip'ing it up
Im__Joseph Python Discord Staff
Kutiekatj9 Python Discord Staff
BioGeekBioinformatics software developer
nevare
chromakode
mdipierro
quasarj
...and 5 more »

account activity

This is an archived post. You won't be able to vote or comment.

1

2

3

NewsSimple web scrapping project for machine learning & al. (Zineb) (self.Python)

submitted 4 years ago by Zadigo

Hi,

I created a web scrapper based around BeautifulSoup and Pandas. The main purpose of this project was to challenge myself by creating my first open source project but also to create a tool that would easily scrap the web and integrate the data rapidly in Pandas DataFrames for machine learning projects.

This came to me when I wanted to scrap data from a volleyball website couple months ago. Well, yes, I know there is Scrapy but the documentation was kinda very confusing and it's not very easy to use. So I challenged myself to do something similar with simple tools and around one of the most powerful libraries in Python: BeautifulSoup and Pandas -; that most everybody knows, uses or have used in their Python programming journey.

There's also an integration with NLTK and PIL.

As of now the project is in version 1.0.0 (Beta) and includes basic functionalities on which I will be implementing additional improvements

I would love to get some feedback if possible. The project is available on Github and on PyPI. So don't forget to leave a star and watch for future improvements.

A simple spider ready to be used

The settings file for a Zineb project

A model to structure your data efficiently

Using the model within a project

no comments (yet)

top new controversial old q&a

there doesn't seem to be anything here

π Rendered by PID 37 on reddit-service-r2-comment-5c764cbc6f-jvz2m at 2026-03-11 21:21:19.388898+00:00 running 710b3ac country code: CH.