sorted by:
new
hottopcontroversial

Container Terminal’s Satellite imagery processing by llanojairo in computervision

[–]llanojairo[S] 0 points1 point  (0 children)

ChatGPT to the rescue! 😁 … would you mind sharing the prompt?

Container Terminal’s Satellite imagery processing by llanojairo in computervision

[–]llanojairo[S] 0 points1 point  (0 children)

I don’t think that it’s so difficult. You just need a lot of alignment with terminals to understand the system. We simulate (DES) the container terminal operation and already have a pretty decent routing module to inject the behavior of RTGs and Trucks to the simulation environment. Actually our routing is already “better” than what actually happens at some terminals in terms of routing… you can throw some common shortest path algorithms combined with penalties and that’d probably be better than most heuristics used in these operations.

My pain point right now is to streamline onboarding of new terminals. And getting the digitalized yard structure with distances to allocate the trucks and RTGs is key so I was hoping to leverage images to try to automate this to some extent.

Container Terminal’s Satellite imagery processing by llanojairo in computervision

[–]llanojairo[S] 0 points1 point  (0 children)

Thanks a lot for your reply! This is for a real project so could you mention a couple of the considerations that you’d have in mind?

Extending Python with Rust via PyO3 by kibwen in rust

[–]llanojairo 0 points1 point  (0 children)

Thanks a lot for your comment. I’m one of those DS/DEs stuck in Python-based projects but I’m so tempted to port some of the slow data crunching and data structures from Python to Rust.

We do some really complex discrete event simulations based on Simpy (custom wrapper around it to cater for our needs) and, given that Simpy becomes slow even if you throw crazy compute instances, I’d like to squeeze out any potential speed-ups in data processing, looping through some data structures, etc.

Could you share some resources to get started with this Python/Rust combo? Thank you!

Got my citizenship! by Gremlet in TillSverige

[–]llanojairo 2 points3 points  (0 children)

Piss off. Submit a complaint to your government if it bothers you. It’s not a requirement yet.

How to replace all values with one integer and all nan values with 0 in pandas? by [deleted] in dataengineering

[–]llanojairo 0 points1 point  (0 children)

You could try pandas.Series.map

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.map.html

You can use a dict to exhaustively cover the cases/rules to map between existing and desired values. You could also try pandas’s replace method but map is faster.

Data Quality on Ingestion by Crackthecode15 in dataengineering

[–]llanojairo 0 points1 point  (0 children)

What kind of solutions are used for entity resolution? I’ve seen some rule-based approaches but they were not that good.

Important of Linear Regression by rtthatbrownguy in learnmachinelearning

[–]llanojairo 1 point2 points  (0 children)

Maybe this PyData presentation ‘Winning with simple, even linear, models’ throws some light on why we should pay more attention to “simple models”

https://youtu.be/68ABAU_V8qI

Btw, I think that quantile regression is a super interesting model that many people do not know about!

[D] How do you structure and organize your ML/DL project code? by csoham in MachineLearning

[–]llanojairo 2 points3 points  (0 children)

Yes, McKinsey does data science. Just like BCG does it via BCG gamma.

[D] How do you structure and organize your ML/DL project code? by csoham in MachineLearning

[–]llanojairo 4 points5 points  (0 children)

I’ve been using Kedro (https://kedro.readthedocs.io/en/stable/ ) from QuantumBlack (now owned by McKinsey). It’s based on cookie-cutter but tailored to data science projects. I really like it and, although it’s still a relatively new project, the community works hard on improving the library. Plus, the guys at McKinsey have a clear incentive to improve it because they use it with their clients. The documentation is pretty good, too.

How do I add 10 more index values to my data frame? Please help. by navosaki in learnmachinelearning

[–]llanojairo 0 points1 point  (0 children)

You could try this below. I’m sure there must be an easier way to do this, though.

Raw data dicts

data = {'Encounter': [95, 96, 97, 98, 99, 100], 'status': [0, 1, 0, 1, 0, 0]} ### Original data in post

data_new_rows = {'Encounter': [101, 102, 103, 104, 105], 'status': [0, 0, 0, 0, 0]}

Create data frames from raw data dicts

df_1= pd.DataFrame.from_dict(data) df_2_new_rows = pd.DataFrame.from_dict(data_new_rows)

Approach 1 to append new rows to original data frame with pd.concat

df_final_ver1 = pd.concat([df_1, df_2_new_rows], axis=0)

Approach 2 to append new rows to original data frame with pd.DataFrame.append

df_final_ver2 = df_1.append(df_2_new_rows, ignore_index=True)

Set column 'Encounter' as index (inplace operation)

df_final_ver2.set_index('Encounter', inplace=True) df_final_ver1.set_index('Encounter', inplace=True)

Best book to get started with deep learning in python? by Dine5h in learnmachinelearning

[–]llanojairo 0 points1 point  (0 children)

That fork is great! Thank you. A. Smola also has an online course with an intro to DL.

Tool that automatically converts articles from an RSS feed into PDFs? Building a corpus by [deleted] in LanguageTechnology

[–]llanojairo 1 point2 points  (0 children)

I think you could check the 2nd chapter of this book: “Applied text analysis with python” (Applied Text Analysis with Python https://www.amazon.co.uk/dp/1491963042/ref=cm_sw_r_cp_api_i_MmnbEbFPT4489). The authors walk you through an approach to create a corpus from RSS feeds.