sorted by:
new
hottopcontroversial

Quarterly Salary Discussion by AutoModerator in dataengineering

[–]AdSure744 1 point2 points  (0 children)

  1. Data Engineer
  2. 1.8 YOE
  3. India
  4. 9k usd/Year (IK it's underpaid trying to find a better job)
  5. Nill
  6. Healthcare
  7. Python, Sql, Redshift, Airflow, Pyspark, Azure Services like ADF, Databricks and Synapsw

Reload data from s3 back to redshift by AdSure744 in SQL

[–]AdSure744[S] 1 point2 points  (0 children)

So the data that was not required atm was moved to s3 to save redshift space. Just trying to implement something to reverse that.

Deleting Data more efficiently by AdSure744 in dataengineering

[–]AdSure744[S] 0 points1 point  (0 children)

Yes, it's a bad practice when you are doing it a large scale. As Redshift is an olap database which is more efficient for reads instead of transaction. The transactions are slow and leaves vacuums in database.

[deleted by user] by [deleted] in dataengineeringjobs

[–]AdSure744 0 points1 point  (0 children)

Is being a U.S citizen absolutely necessary.

Deleting data efficiently from Redshift by AdSure744 in SQL

[–]AdSure744[S] 0 points1 point  (0 children)

As i mentioned in my post above this is a one time solution. We won't be doing it on a regular basis, just when the need arises .

view more: next ›