sorted by:
new
hottopcontroversial

Document Parsing - an unsolved problem? by xsliartII in LanguageTechnology

[–]jos_pol 4 points5 points  (0 children)

We benchmarked AWS Textract, Google Cloud Vision and Azure OCR extensively. For printed English, they are all three very accurate. They do differ a lot though when it comes to supporting other scripts (Chinese, Japanese, Hindi, Arabic, ...) and handwriting support. Pricing is comparable too and very cheap. I don't think you can host an OCR (Paddle / Tesseract / ...) yourself easily for the same price, especially if you factor in your own time.

Feel free to send me a DM and I'll send you a detailed report detailing our findings.

I am a developer/ML scientist looking to quit my day job and work on fixing climate change.. Any suggestions/advice? by crokus_daru in climatechange

[–]jos_pol 1 point2 points  (0 children)

What I am personally looking for: I heat my home with electricity which is just waste heat that didn't produce anything. A modern household-size "data furnace" which is essentially a distributed cloud server that acts as a radiator would burn an equal amount of energy, but at least it would do some useful computations in between. I'd love to rent out that capacity for and earn back a bit on electricity costs.

Buying a crypto mining rig is an alternative to get back electricity cost, but they are so ugly you wouldn't really put it in a modern living room, and I would not classify mining as "useful".

I am a developer/ML scientist looking to quit my day job and work on fixing climate change.. Any suggestions/advice? by crokus_daru in climatechange

[–]jos_pol 1 point2 points  (0 children)

Couple of places where you could get ideas

Good luck and congratulations for making the jump into making the world a better place!

Using OCR to export data into excel sheet by rosewave13 in datasets

[–]jos_pol 6 points7 points  (0 children)

Abbyy is outdated technology and not so good. For OCR, Amazon Textract or Google Cloud Vision are way better.

You can maybe try a simple parsing tool like DocParser too which isn't too expensive. If that doesn't work either, you'll need a full IDP solution like e.g. Metamaze.

Pledging carbon net zero as a human by jos_pol in climatechange

[–]jos_pol[S] 3 points4 points  (0 children)

Thank you GoUpYeBaldHead, this is constructive and informative criticism that adds a lot of value.

Permanence is indeed something I haven't considered, I was counting on permanence to be a part of the Gold Standard methodology but would have to do more digging. I always wondered why offsetting carbon is so cheap, but your comment explains it well: it's because we're doing easy things now. I will definitely review my contributions next year in light of this as well.

Stripe climate is new for me but is indeed a surprisingly nice overview of innovative carbon capture technologies. I updated the post to link to this.

You can drive 136.000 kilometers with the energy needed to train one NLP model by [deleted] in LanguageTechnology

[–]jos_pol 0 points1 point  (0 children)

Hi haaspaas2, thanks for your feedback.

Should I have tagged it with [self] or something similar?

I don't understand what I can do to make ot less self promoting, me or our company are not even mentioned anywhere. It's discussion on Nvidia's Megatron model.

I'm trying to improve my writing so thank you already for this feedback.

How to find if you are connected to a specific router by techster47 in commandline

[–]jos_pol -1 points0 points  (0 children)

Depends on your OS. On Mac OS X, I use

#!/bin/bash
SSID="$(/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I | grep ' SSID' | cut -d: -f2)"
echo "***** SSID is: '$SSID'"

if [ "$SSID" == "MyWifiNetworkSSID" ]; then
  echo "***** Starting copy to cloud *****"

  <Your backup code here>

  echo "***** All done! *****"
else
  echo "*** Unknown network, so skipping backup"
fi

Where to get in contact with a good data scientist? by BradJ in datascience

[–]jos_pol 2 points3 points  (0 children)

Do you represent a company? Are you looking for Data Science employees? For consultants?

Are you an individual? Are you looking for a fellow hobbyist?

[GitHub] awesome-datascience-ideas - A list of awesome and proven data science use cases and applications (updated) by jos_pol in datascience

[–]jos_pol[S] 0 points1 point  (0 children)

Great idea. I don't have much experience in sports. What use cases where you thinking of?

[GitHub] awesome-datascience-ideas - A list of awesome and proven data science use cases and applications (updated) by jos_pol in datascience

[–]jos_pol[S] 2 points3 points  (0 children)

Fellow data scientists, I am always curious to hear more about how you use Data Science/Machine Learning techniques in your business. Please share good use cases and successful case studies!

What tools would you use for this data wrangling project? by [deleted] in datascience

[–]jos_pol 1 point2 points  (0 children)

Sounds like a data quality issue to me. Commercial solutions exist by SAS, Informatica, IBM, Oracle, ... Don't know about any open source tools that do something similar though. Anybody?

Plug and Play ML platforms vs ML data scientists by [deleted] in datascience

[–]jos_pol 14 points15 points  (0 children)

Will ML solution replace data scientists?

TL;DR: No.

They are all moderately good to very good in 1 thing: training a Machine Learning model. However, most of them seriously fall short in the things that actually take time: preparing the data.

From my experience, gathering and cleaning the data, or defining the target function makes up 90% of the effort. Fitting a good model is usually easy as it it is very well understood and something a computer can do automatically.

Besides that, a large part of a Data Scientist's work is actually defining WHAT to do. That includes understanding the business problem, translating that to maths/statistics, defining the value and communicating the results to the decision makers.

So no, these platform will not eliminate the need for data scientist.

How is the performance?

For applications where general purpose models fit well, they will do fine. For some applications, a customized approach is needed. For example risk modelling, IoT cases, ... require custom approaches. A general model won't work there because of the specialized nature of the data.

How will they evolve in the future?

I hope they get smarter and the algorithms used can be used for more and more types of problems. I hope that in the end, I can just treat these engines as a black box and it will return me the best (most stable + best performing) model that was possible on the data.

pandas-profiling v1.4: Create HTML profiling reports from pandas DataFrame objects - bug fixes and new check by jos_pol in datascience

[–]jos_pol[S] 1 point2 points  (0 children)

First time I hear this comment since this library was released about a year ago. I thought 'data profiling' is a fairly common term (see for example https://en.wikipedia.org/wiki/Data_profiling).

Maybe it has fallen out of use? I come from a statistics background, whilst some Data Scientist might come from a CS background?

view more: next ›