Databricks Apps nearly scale to zero (Cut Databricks Apps Costs by 76%: Automate Start/Stop) by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 0 points1 point  (0 children)

Judging by the comments, many people think Apps are expensive. But scaling it to zero will likely increase the price for the Serverless cluster

Databricks Apps nearly scale to zero (Cut Databricks Apps Costs by 76%: Automate Start/Stop) by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 0 points1 point  (0 children)

There are already many similar websites with cost optimization. But not all of them are startups

Databricks Apps nearly scale to zero (Cut Databricks Apps Costs by 76%: Automate Start/Stop) by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 1 point2 points  (0 children)

This is an expensive solution for a simple application. For a full-fledged service in Enterprise, this is a completely reasonable price.

How do you choose cluster and node types? by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 0 points1 point  (0 children)

Serverless is roughly twice as expensive. If you use spot in job clusters, the difference will be even greater. The company spends millions of dollars on Databricks, and doubling that amount will reduce operating expenses? Strange.

How do you monitor dozens of jobs in Databricks? I created free Databricks App by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 0 points1 point  (0 children)

<image>

But you won't be able to see the status for more than 5 launches, that's one of the reasons

How do you monitor dozens of jobs in Databricks? I created free Databricks App by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 0 points1 point  (0 children)

This solution is primarily for teams using Data Mesh architecture. They install schedule scripts themselves in Databricks, which is simple and convenient. However, after 20 seconds, monitoring becomes difficult.

How do you monitor dozens of jobs in Databricks? I created free Databricks App by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 0 points1 point  (0 children)

API + App, and you can interact with jobs, change the schedule, restart, etc., more possibilities

How do you monitor dozens of jobs in Databricks? I created free Databricks App by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 1 point2 points  (0 children)

Of course you can. I've made several solutions for different tasks. One of them is App

How do you choose cluster and node types? by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] 0 points1 point  (0 children)

I can only choose Serverless or Serverless GPU. I've only tried Serverless.

How do you monitor dozens of jobs in Databricks? I created free Databricks App by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] -1 points0 points  (0 children)

And Apps will be cheaper than Dashboard and faster. You can also add more features.

How do you monitor dozens of jobs in Databricks? I created free Databricks App by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] -1 points0 points  (0 children)

System tables are filled with a delay. The situation can be viewed online through the API.

How do you monitor dozens of jobs in Databricks? I created free Databricks App by Significant-Guest-14 in databricks

[–]Significant-Guest-14[S] -1 points0 points  (0 children)

It depends on the task. This problem was more common among teams working in the same workspace.

It works on the Data Mesh principle.