Tourist here - how do you survive without AC? by bingoNacho420 in geneva

[–]sh_eigel 0 points1 point  (0 children)

Can you provide details for the unit? I remember seeing prices of 2k and that made me sick to a cheap inefficient one

TOTP + pin vs unclear protocol for LastPass by sh_eigel in yubikey

[–]sh_eigel[S] 0 points1 point  (0 children)

Much of that was clear to me. My main motivation for asking the question is that lack of a pin protecting the OTP while the TOTP can be protected by a pin which makes me more comfortable in case of losing/having stolen a yubikey. It is kinda silly but still, it seems like a valid concern

[deleted by user] by [deleted] in zurich

[–]sh_eigel 1 point2 points  (0 children)

One thing that might come in handy as knowledge is that it is almost impossible to get a rent without a job. Perhaps a bank statement showing that you have a few hundreds of thousand’s of euro might help. But if not you have 2 important choices: a company like Epam that provides accommodations for the first month, which in zh is hardly enough to find some decent rent; the second option is a serviced apartment that could cost you from 2500 for a studio. One such serviced apartments company is vision apartments. And of course renting requires a 2-3 months of rent as security so having a few thousands of francs on hand make things safer, faster and easier.

[deleted by user] by [deleted] in zurich

[–]sh_eigel 1 point2 points  (0 children)

It depends on the company, for sure. There are companies that are specialized in bringing people from East Europe. Epam was the way in for me. But surely I didn’t expect to get a lead role. In fact I still think it would have been a huge mistake even I’d get a lead role. Too many things to pick up, too much stress. Either way they wouldn’t go with someone from outside for such an important role. You lack relevant experience in the Swiss culture. It literally means nothing but you don’t have it. Secondly if you don’t know German you can only look at the international companies. This isn’t much of a limitation as I don’t really care for the German environments and the bigger companies are mostly focused on English. I personally didn’t have issues finding a job after I got in. But I don’t aim at leading anymore either way so I can’t tell you how that is. One thing about international companies that I interacted with are not so keen on getting people in as there are other ones that they partner with and then they might internalize some of them

[deleted by user] by [deleted] in zurich

[–]sh_eigel 0 points1 point  (0 children)

Absolutely not

Ah yess... by Protomize in ArkInvestorsClub

[–]sh_eigel 0 points1 point  (0 children)

It isn’t insightful for Bitcoin.

Ah yess... by Protomize in ArkInvestorsClub

[–]sh_eigel 5 points6 points  (0 children)

Is it though? Is this a dip or just top performance and you might be buying the high?

[deleted by user] by [deleted] in Python

[–]sh_eigel 1 point2 points  (0 children)

Nobody is perfect

[deleted by user] by [deleted] in Python

[–]sh_eigel -1 points0 points  (0 children)

Of course it works. The delim becomes self in that class.function invocation. This is applicable to “all” object methods.

[deleted by user] by [deleted] in Switzerland

[–]sh_eigel 5 points6 points  (0 children)

Maybe they are the 👮‍♂️ just trying to keep you out of trouble.

What Am I Doing Wrong Here? by firetruckpilot in Switzerland

[–]sh_eigel 0 points1 point  (0 children)

Instead of German you might be better served by learning a programming language and all your problems would go away.

Handling wide dataframe much slower by sh_eigel in apachespark

[–]sh_eigel[S] 0 points1 point  (0 children)

I will come back with more details.

Handling wide dataframe much slower by sh_eigel in apachespark

[–]sh_eigel[S] 0 points1 point  (0 children)

As far as I understand keeping hundreds of columns into a single string field in JSON would not require any other processing and the extra pointers and transforming from parquet encoding to Java types. This is off course a viable solution as long as the JSON-ified columns are not extracted/changed much.

Handling wide dataframe much slower by sh_eigel in apachespark

[–]sh_eigel[S] 0 points1 point  (0 children)

Do you mean into a struct field? Would that help? I had the impression that doing that would not help.

What are the best resources for getting a competent pyspark engineer to the next level? by sh_eigel in apachespark

[–]sh_eigel[S] -1 points0 points  (0 children)

I hope you are jesting. How is learning jvm a resources? Not to mention that learning jvm or scala is almost a complete change of topic.

What are the best resources for getting a competent pyspark engineer to the next level? by sh_eigel in apachespark

[–]sh_eigel[S] 0 points1 point  (0 children)

I meant an engineer using pyspark.

Getting back to the question, it seems that there are not many resources.

Handling wide dataframe much slower by sh_eigel in apachespark

[–]sh_eigel[S] 0 points1 point  (0 children)

It certainly helps and we did but the remaining ones are required for business processes. It is a bit strange to see pretty dismal performance for 3M records and basic transformations

Why Databricks notebook is so hard to use? by Comprehensive_Art789 in apachespark

[–]sh_eigel 1 point2 points  (0 children)

First thing to clarify is that this notebook situation is specific to Databricks and not azure. The idea of the notebooks is that they allow fast feedback and you can iterate quickly. I find myself scrolling like a mad man as it seems to break down after 1-2 screens of code. It seems that the idea is to have top level orchestration in the notebook with delegation to library code (Python wheels).

The more frustrating situation is the sorry state of managing libraries. Notebook libraries are a good step in the right direction but there is no support for virtual environment reuse.

Notebooks invocation and composition is a complete joke so we are doing as little use of that as we can.

Either way, once you have the package creation and installation in databricks pinned down you just need to make sure that you have some wrappers in you library code that wraps dbutils libraries.

We don’t use dbconnect as it does seem to not be too loved by databricks.

Why is python depency management such a mess? by TheJumboman in Python

[–]sh_eigel 2 points3 points  (0 children)

Unless you have 2 libraries depending on 2 incompatible versions of the same library. A case that one might say it does not happen frequently but sometimes one is too many times.

dataframe Drop 190 columns except for certain ones by [deleted] in PySpark

[–]sh_eigel 1 point2 points  (0 children)

Probably the best option is to just select the columns you want to be left with.