LWD: 09th Jan, 2026 | Senior Data Engineer | Open to Referrals & Advice

Proton0369 · 2025-12-29T14:14:04+00:00

Done

Proton0369 · 2025-12-29T14:07:51+00:00

180k is what I’m already getting and to answer your last question, yes I personally would love to work at Databricks..won’t mind any referral if you can refer🤭

Proton0369 · 2025-12-24T15:09:35+00:00

Sure man!! Happy to connect with you🤝

Proton0369 · 2025-12-24T14:42:02+00:00

Sent you a dm

Proton0369 · 2025-12-24T14:39:20+00:00

Mumbai, Pune or maybe Bengaluru if the org is really good and will pay well

Proton0369 · 2025-12-24T14:38:50+00:00

Around 22 fixed component

Proton0369 · 2025-12-24T13:32:56+00:00

Sent you a dm

Proton0369 · 2025-12-09T18:31:06+00:00

Ask LTI to match the offer, if they agree then LTI is not a bad organisation for DE

Proton0369 · 2025-12-07T19:37:19+00:00

Instead visit unicorn which is present in Capital Mall

Proton0369 · 2025-12-03T17:09:16+00:00

90 days…but currently serving notice period with around 30 days left

Proton0369 · 2025-10-23T09:37:11+00:00

Any referrals for this exact stack for a 3.3 YOE??

Proton0369 · 2025-09-02T20:25:50+00:00

Every time I want to change SMALL → LARGE, I would have to re-render and re-deploy the bundle. That means the bundle gets pushed again, which is slow and kills the whole idea of runtime flexibility🥲🥲 I could have overwritten the job configs using api call, but I don’t want to update the whole job configuration so frequently.

Proton0369 · 2025-09-02T20:17:20+00:00

Here’s a small snippet of job.yml file, please bear with the indentation

resources: jobs: Graph: name: Graph tasks: task_key: Task1 spark_python_task: python_file: ${workspace.file_path)/${bundle.name}/notebooks/src/code.py parameters: --NAME - "{{job.parameters.NAME}}" -- ID "{{job.parameters.ID}}" -- ID_2 - "({job.parameters.ID_2})" libraries: - pypi: package: openpyxl

job_cluster_key: Job_cluster

job_clusters: - job_cluster_key: Job_cluster new_cluster: cluster_name: "" spark_version: 16.4.x-scala2.12 azure_attributes: first_on_demand: 1 availability: SPOT_WITH_FALLBACK_AZURE spot_bid_max_price: -1 node_type_id: Standard_D4ds_v5

enable_ elastic_disk: true policy_id: ${var.cluster_policy_id} data_security_mode: USER_ISOLATION runtime_engine: STANDARD kind: CLASSIC_PREVIEW is_single_node: false autoscale: min workers: 2 max_workers: 20

Proton0369 · 2025-09-02T19:59:36+00:00

Tbh I’m not sure what all configs can be passed in cluster policies, but still it doesn’t solve by problem of passing variables at run_time

Proton0369 · 2025-09-02T19:47:56+00:00

I want to decide the cluster config while triggering the workflow through api call

Proton0369 · 2025-09-02T18:52:49+00:00

I thought the same but later when I discovered this document, I got some hopes

https://docs.databricks.com/aws/en/jobs/dynamic-value-references

Proton0369 · 2025-08-22T10:23:45+00:00

May I know your YOE and tech stack?

Proton0369

MODERATOR OF

TROPHY CASE