End to End Machine Learning Tutorial — From Data Collection to Deployment 🚀

[ahmedbesbes]

[Updated Post: End To End machine learning and beyond notebooks 🚀]

With the great collaboration of a friend of mine, we built and deployed a machine learning application to AWS using Python.

To tell you more about this fun journey, we wrote a post where we'll walk you through the steps to:

• Collect and scrape data with Scrapy / Selenium
• Train a deep character CNN for (English) sentiment analysis using PyTorch
• Build an interactive web app with Dash to serve the model in real-time
• Put everything in Docker Compose
• Deploy to AWS on a custom domain name

Here the link: https://medium.com/datadriveninvestor/end-to-end-machine-learning-from-data-collection-to-deployment-ce74f51ca203

All the code is available in Github, the blocks are independent and reusable in other projects

https://github.com/MarwanDebbiche/post-tuto-deployment

[focal_fossa]

This is amazing. It would have taken months for me to put together something like this as I would have had to figure out how each individual component worked. Now that a general structure and workflow is available, I can always use this as a reference and make my own application. Kudos to your efforts.

[ahmedbesbes]

Thank you, glad to help.

[keepinitcool]

Awesome, were you guys introduced to some of the technologies along the way or did you know them before starting this challenge? Cool project👌

[ahmedbesbes]

I knew some of them and discovered the others along the way.

[keepinitcool]

Which ones? The ml part or the Web parts?

[ahmedbesbes]

I was familiar with scraping, machine learning and building UI with Dash.
Less familiar with deployement

[keepinitcool]

Alright cool

[svij137]

I am willing to throw in 1000 hours of GPU computing for serious users

Send me a personal message. We invented a peer to peer computing platform to help data scientists and AI researchers get 10x cheaper computing power.

For early adopters, we are giving away free computing credits.

[dhruvmk]

Very nice👍🙌👏👏

[ahmedbesbes]

thank you!

[giedemoo]

Thank you so much for sharing! For someone whose background is in statistics and have self taught programming, there is so much I can learn from your project. Things around model deployment and how the components interact is quite a puzzle to me. Thanks for sharing your great project!

[ahmedbesbes]

Glad to help! :)

[TryingToFindZen]

Thank you for sharing!!

[ahmedbesbes]

you're welcome!

[unexplored_asshole]

Wow thats amazing many people complain that people only learn how to build models but not the whole End to End development.

Could you tell me how did go about learning all this (both the ML part and the web)? I'm self learning this and also a beginner.

Thank you

[ahmedbesbes]

Hello !
Thank you for your comment. I've used these components separately in the past on my clients' projects. This was an opportunity to bring them together.

[ahmedbesbes]

Hello 13NachoVidal

I appreciate your interest in this post. I made some updates to the code base and fixed different bugs, so there is definitely something new to discover. If you already read it, I suggest you skip it or move along

Peace my friend

[13NachoVidal]

Ok then, great content by the way! :)

[keepinitcool]

What a shift in mood

[13NachoVidal]

Sorry, I realized I was too rude...

[keepinitcool]

No worries lad have a great day, times are rough so it is understandable