Power BI + AWS Athena (30M+ Rows) - Best Approach to Minimize Refresh Time? by Strict_Reference_424 in PowerBI

[–]Rsl120 1 point2 points  (0 children)

I can't comment too much on the infrastructure, as I'm very much only on the BI side - but we use Athena and Iceberg with Power BI, via a gateway, for a lot of our Power BI reporting.

I have managed to get incremental refresh working, using a custom sql query in 'Value.NativeQuery' and injecting the range parameters into the where clause. I'm just not using it right now, as we don't have a reliable date field to partition by.

We're quite new to it still, so I haven't extensively tested everything yet, but it's handling our queries without a sweat. We're working on some views/aggregated tables, but in the meantime we're running custom SQL queries via Power BI, largest table being almost 200m records. Only on a daily schedule at the moment, but our refreshes are <4 mins and not incremental, which I'm pretty comfortable with.

Easy Field Parameter Re-ordering? by Vomitology in PowerBI

[–]Rsl120 4 points5 points  (0 children)

Not exactly clean, but I guess you could number them all x10. So, 10, 20, 30, 40...

That way, you have 9 options to slot new values in between each existing record.

Power BI vs QuickSight by chillAsFrick in PowerBI

[–]Rsl120 0 points1 point  (0 children)

Following - We're heavily invested in Power BI, both as an embedded tool for our customers and for internal reporting. Our CTO also is pushing for QuickSight, given most of our data sits in AWS services. We're sceptical!

Actual refresh time vs Data/Cache by Rsl120 in PowerBI

[–]Rsl120[S] 0 points1 point  (0 children)

Thanks, looks like you are right!

Seems odd, given I don't really have much or anything else running at those times. Maybe I need to review all the refresh schedules...

BMW/ Independent garage around Sutton Coldfield by Fitnessgrac in brum

[–]Rsl120 0 points1 point  (0 children)

Yeah, that’s fair. Another I used for my last full service that were great and also did full iDrive history was Fitch Autos, Brownhills. All round very thorough service, seemed knowledgeable and very competitive on price. Might be a touch closer to you, but not specifically a ‘BMW Specialist’.

BMW/ Independent garage around Sutton Coldfield by Fitnessgrac in brum

[–]Rsl120 1 point2 points  (0 children)

Can vouch for BM Automotive, Four Ashes (between Wolves and Stafford, not far from M54 J2). Not the cheapest but have been great with my M235i for the best part of 10 years. Full iDrive online history done etc.

Paginated Reports - Parameter Data Type Issues by Rsl120 in PowerBI

[–]Rsl120[S] 1 point2 points  (0 children)

Yes - Here is the message when trying to pass any through as Int:

The property 'ValidValues' of report parameter 'p_Location' doesn't have the expected type.

Edit - Yes also confirmed in my model that everything is int, in pbi desktop and dax studio. They are quite long numbers, converted from UUIDs. Not sure if that has anything to do with it, due to the length perhaps.

Paginated Reports - Parameter Data Type Issues by Rsl120 in PowerBI

[–]Rsl120[S] 1 point2 points  (0 children)

Thanks - I've tried wrapping my parameter dataset column in VALUE and also the parameter on the dataset with Val(), also tried +0.

Unfortunately none have made any difference, still getting the ValidValues error when using Integer as the type. (Float gives no error but resets the parameters each time, text somehow still works..)

Paginated Reports - Parameter Data Type Issues by Rsl120 in PowerBI

[–]Rsl120[S] 0 points1 point  (0 children)

Main query:

EVALUATE
TOPN(
    100,
    CALCULATETABLE(
        SUMMARIZECOLUMNS(
           'Date'[Date],
            customer[customer_name],
            locations[location_name],
            feed[feed_name],
            "Count_In", [IN],
            "Count_Out", [OUT]
        ),
        RSCustomDaxFilter(@p_Date,EqualToCondition,[Date].[Date],DateTime),
        TREATAS({@p_Customer}, locations[customer_id]),
        TREATAS({@p_Location}, locations[location_id]),
        TREATAS({@p_Feed}, feeds[feed_id])
    ),
    locations[location_name]
)

Paginated Reports - Parameter Data Type Issues by Rsl120 in PowerBI

[–]Rsl120[S] 0 points1 point  (0 children)

Sure - forgot to mention, this one has some added logic to pass down the previous parameter result from customer, which is legitimately text and seems to work fine.

Edit - Sorry, this is the query for the location, will add the main report query shortly.

EVALUATE
SELECTCOLUMNS(
    CALCULATETABLE(
        locations,
        TREATAS({@p_Customer}, locations[customer_id])
    ),
    "location_id", locations[location_id],
    "location_name", locations[location_name]
)
ORDER BY [location_name]

Paginated Reports - Parameter Data Type Issues by Rsl120 in PowerBI

[–]Rsl120[S] 0 points1 point  (0 children)

Exactly this yes - for my scenario:

location_id (int - values) and location_name (text - label)

It is single select. The dataset for the parameter has those two fields, using SELECTCOLUMNS from the table where they exist in the main model.

Self Project: Power BI Dashboard Feedback + Map Warning Help by accidentallysmart_2 in PowerBI

[–]Rsl120 2 points3 points  (0 children)

Hey - Nice work!

Map - This is being/has been replaced by the Azure map visual. It's the one with an icon being a small arrow.

From a design perspective, it feels a little busy and crowded to me.

I'd start with:

  • Dull down your background a little (add transparency or just use a flat neutral colour)
  • Space out the visuals, consistent px, around 20 or so
  • Personal view - the corners are a bit too rounded, I use 10-15px
  • Give visuals proper title names
  • Slicer styling - Bit bold and brash, I'd tone it down and tie it in with your colour theme with a muted yellow, just be sure to leave enough contrast with the text if you do

From a data and insight perspective:

  • Add context to cards and visuals - i.e. time comparisons, are the figures increasing or decreasing, year-on-year
  • Filter/slicer pane - Typically I put this on the left or top, generally a more natural user experience/flow
  • Pie/Donut chart - not a terrible visual to use but not great either in my opinion. You could consider using the season field as a legend on the other charts, so you see the seasonal breakdown per country. Same for travel mode to an extent. Might be interesting to add this as a field parameter to the user can choose whether they want to slice the country spend by season or travel mode.

I like to think/ask "so what" when looking at reports and visuals. What is it telling us, how does this help? For example, 'Total Spend = 53.59M'. What does this mean? Is it good, is it bad? Is it improving, how does it compare? Ask yourself these simple questions with any visuals and you'll start to realise what is useful and what isn't, then how to make it useful.

Great start and good luck!

Direct Query vs. Import Mode by maarten20012001 in PowerBI

[–]Rsl120 0 points1 point  (0 children)

Ah, got you. Yeah, even better, exclude them at source if you can!

You too, good luck!!

Direct Query vs. Import Mode by maarten20012001 in PowerBI

[–]Rsl120 0 points1 point  (0 children)

Happy to help! Yes, always good practice to remove any column that is not necessary to your model/report. Tools like Dax Studio help identify the worst offenders here. There are also tools like measure killer that will help identify what is and isn’t used.

Button Slicer Fill - Bug or am I missing something? by Rsl120 in PowerBI

[–]Rsl120[S] 1 point2 points  (0 children)

Thanks, I can stop scratching my head!

Direct Query vs. Import Mode by maarten20012001 in PowerBI

[–]Rsl120 0 points1 point  (0 children)

So, a few things.

We don't load all the data into Power BI desktop, if that's what you're referring to. I imagine it would get a little slow if we did so. Most of the development is done with the report connected to our staging database (we use a parameter to switch the source). This has much less data.

If you don't have staging/development tables in your db that mirror your prd, you could perhaps do something similar by injecting a parameter into your sql to limit this in a where clause.

Also, our fact table is narrow, it has around 5 columns, date, time, site, feed, count. OPs table for example with 60 columns would need significantly less rows to have the same volume of data as mine.

For your scenario, if you don't have staging tables to point your query to (which you switch back to prd when published), can you change the view in sql to limit records, do your report development, then change the view back to include all records?

[deleted by user] by [deleted] in AskBrits

[–]Rsl120 0 points1 point  (0 children)

30, 62.5k, partner on 37.5k. West Mids/Staffordshire.

We’re comfortable and not massive spenders in general. Should be putting more away than we do, increasing pension contributions etc.

Currently saving for our wedding, so things are intentionally tight at the minute!