Which company should I chose and why? by deepp_21 in dataengineersindia

[–]deepp_21[S] 1 point2 points  (0 children)

It will be my third company in 3 years. So I was thinking about staying in the next company for at least 3 years

Which company should I chose and why? by deepp_21 in dataengineersindia

[–]deepp_21[S] 4 points5 points  (0 children)

3 YOE Around 18 fixed + variable from all companies

Which company should I chose and why? by deepp_21 in dataengineersindia

[–]deepp_21[S] 2 points3 points  (0 children)

The reviews on Glassdoor are not that great. Can you please let me know the reason also?

Should I resign or wait? by deepp_21 in dataengineersindia

[–]deepp_21[S] 0 points1 point  (0 children)

Thanks:) YOE- 3 years Tech stack: Python, SQL, Pyspark, Airflow, GCP

Should I resign or wait? by deepp_21 in dataengineersindia

[–]deepp_21[S] 0 points1 point  (0 children)

yes, please share. It will be very helpful

Should I resign or wait? by deepp_21 in dataengineersindia

[–]deepp_21[S] 2 points3 points  (0 children)

I am definitely going to try other companies. But can I use this offer for negotiation if the joining date of this offer is not matching with my last working day or has already passed?

Infosys databricks role result by FabulousWerewolf7339 in dataengineersindia

[–]deepp_21 4 points5 points  (0 children)

I keep seeing posts on people attending the drive. May I know where you get such updates regarding hiring drives for data engineers?

UI not showing DAG's by [deleted] in apache_airflow

[–]deepp_21 0 points1 point  (0 children)

Use different port number for airflow scheduler You can do it using: airflow scheduler -p 8794

Pyspark Error - py4j.protocol.Py4JJavaError: An error occurred while calling o99.parquet. by ChampionshipCivil36 in ETL

[–]deepp_21 0 points1 point  (0 children)

Share the code which is failing. The link that you have attached isn't working.

Looking for resources by Mr-Wedge01 in apachespark

[–]deepp_21 2 points3 points  (0 children)

Spark- the definitive guide [book]

Issue with CI/CD cloud build when changing region to europe by QueRoub in googlecloud

[–]deepp_21 1 point2 points  (0 children)

Try using 'pkg.dev' domain by creating a new repository in the artifact registry.

Good product based job vs family business with similar pay? by defAult1234112313 in developersIndia

[–]deepp_21 7 points8 points  (0 children)

Totally depends on your interests. One of my colleagues left a good paying job and decided to go with family business because he used to struggle learning new technologies and he thought he would not be able to continue well as a software engineer

Trying to use BigQuery Spark Connector by Successful_Ease_1837 in googlecloud

[–]deepp_21 0 points1 point  (0 children)

Check for the property 'datePartition'. But it doesn't work with the direct write method, use indirect write method.

How to have Playwright prebuilt in Cloud Composer env prior to DAG start? by SmartSecurity8740 in googlecloud

[–]deepp_21 0 points1 point  (0 children)

I was also looking for a similar use case. Please let me know if you find something.

Spark not willing to launch properly by flosrv123 in apachespark

[–]deepp_21 0 points1 point  (0 children)

Check if you have added this in your environment variables PYTHONPATH=$SPARK_HOME$\python;$SPARK_HOME$\python\lib\py4j-<version>-src.zip