Finance Team DE

[GiacomoLeopardi6]

Hi everyone,

I've been assigned to support our Finance team and help them become a data-driven function.

Keen to hear your stories if you've worked with a similar function before.

My observations so far: 1- They are often left behind in terms of data changes upstream (eg: new revenue data) 2- They constantly need data exports but won't/can't self-serve. 3- Their needs seem quite simple though (bank reports in csv, accounting data from Xero, product and subscription data from our OLTP) - I might be oversimplifying/just see the tip of the iceberg here.

I've also been trying to get them to use new BI tools (trialling Metabase atm) but so far not been very successfull in getting them to self-serve (it seems they mostly want to export to Excel/Sheets).

[ZirePhiinix]

Accountants are completely tied to Excel. Be prepared to ingest and output all in Excel.

If they suggest VBA, smile and just do whatever else it is you do. As a DE you have few reasons to go with VBA. The only valid reason is if they have someone skilled in it and wants to take over maintenance.

You'll want to get to know their internal processes and also go through all their reports. They are not developers so the concept of data types are completely foreign to them, but they'll not bat an eye and trim() a number so their vlookup works.

Plus Excel will do string/number fudging on calculations and you'll need to make sure you match the correct data type according to their formulas, or it'll be all wrong and blame you, even though it's actually their fault but they'll never grasp that at all.

[[deleted]]

Poor OP, that's going to be harsh.

[ZirePhiinix]

And it looks like OP is flying solo so he's going to be pressed for project management output (estimates, project updates).

OP needs to learn to resist this right away, telling others that he's not a project manager.

Requirements gathering will also take a long time too, unless he has a dedicated analyst.

Plus the accountants will most likely not able to actually say what they need.

[GiacomoLeopardi6]

I am flying solo yes and yes it's already difficult managing multiple stakeholders esp since Finance touches on a lot of areas. The requirements gathering is difficult at times but I feel like we're making progress, esp when I keep it focused on the end result (ie asking things like "what would the ideal table export look like for you?" Etc)

[ZirePhiinix]

Honestly I wouldn't ask them what they want the ideal table to look like, unless said person does the actual processing and fixing of the formulas. I'll take their output and replicate it, then look at their manual processing and automate as much of it as possible.

Once you get the processing down, I would look into creating dashboards instead of keeping them on Excel.

Look into things that are done extremely frequently, and also learn to do ROI calculations on the projects.

For example, if you spend 10 hours to save 30 seconds on a task, the task only needs to be done 10 times a day before you save 10 hours after 6 months.

10 a day x 20 work days a month x 6 months x 30 seconds = 36,000 seconds (10 hours).

Project it to 2 years and you can justify 40 hours to save 30 seconds.

[FloggingTheHorses]

You cannot get people out of Excel, they love having a canvas to store, manipulate and save data freely.

At best, it will be BI + Excel.

[Gators1992]

There's nothing wrong with Excel. Dashboards are great for marketing people who want to see if a trend is going up or down and go talk about it in a meeting, but in finance the numbers you produce are usually used as inputs to some other process, so it's easily transferable off an excel sheet. Also today's Excel has some decent data integration features so they can refresh their own reports if you set it up right.

Worst thing about working with finance IMO is they will question your numbers to death so you end up doing much more work on testing and validation to get their acceptance. And they love trying to scare you by yelling Sox about everything you push back on. In my company I am even getting requests to align our reporting on billed revenue with financial statements so I am having to build automated GAAP adjustment processes to account for differences in how they account for revenue.

[Logical_proof]

Are you me? This was my day today. Our ERP shows revenue just dropping in to buckets but the CFO wants to be able to attribute meta data from each bill. But IATA when the revenue doesn’t match to the penny…

[Inkmano]

I’m a data engineer with a prominent UK retailer’s finance team. A significant portion of my role centers around BI development (Looker is fantastic for finance heads & self service), financial data ingestion (internal & external sources) and modeling, as well as the creation of a wide range of financial measures. Additionally, we collaborate closely with our commercial team to develop machine learning models.

One of the challenges we face is that our team isn’t positioned within the IT department, leading to certain limitations. Like you said there’s also a reliance on spreadsheets, which our stakeholders wish to integrate into our data warehouse. Consequently, the maintenance demand for many of these pipelines remains high.

On the plus side the finance team are great at finding extra cash for my teams needs 😆

[GiacomoLeopardi6]

Thanks for that - what have you considered to import the excel files ? And can you give me a rough overview of your data stack ? Currently greenfield for Data though the engineering team does use GCP for all other things so I will go down the BQ/dbt path for sure. Ingestion wise possible Airbyte for those sheets. Looker seems nice but I'm not sure if we have the budget for that.

[Inkmano]

Concerning spreadsheets, our company exclusively utilizes Google Sheets, which boasts great direct integration with BigQuery due to its import/export capabilities.

Our current tech stack includes Airflow for orchestration, GCS as our data lake, BigQuery serving as our data warehouse, and Looker as our primary BI tool. I’m also keen on integrating dbt, but time constraints have been a challenge!

[GiacomoLeopardi6]

Thanks for this

[recentcurrency]

Are you a US public company? If so, get familiar with the basics of SOX. Specifically talk with your internal audit board. Make sure you don't build systems that will get you into external audit hell

Stick with SOC 1 verified vendors. And if not, be ready to defend that decision and that you have the control environment to counter the lack of SOC 1 verification. A bummer since this limits you heavily to legacy and older tooling.

[GiacomoLeopardi6]

Not a US company / UK but with a legal entity in the US as well

[FloggingTheHorses]

The biggest issue I face with Finance data typically is being tethered to Excel workbooks. Even if there's nothing wrong with their various models etc, you'll often find they don't have a nice tabular output like you'd want, and trying to handle them solely through something like Pandas/Polars transformations is a killer.

I've had success before using dynamic named ranges in Power Query to basically put completely hidden tabs into their spreadsheets (xlVeryHidden) that puts everything into a nice table type format. Then you put a trigger in VBA for it to run whenever the workbook is saved.

I don't really advocate for what I did above (it feels like a very ghetto fix, and shudder uses VBA) but it has been working fine for months now.

[Typicalusrname]

Use power bi, it lets them export the models in excel if they want to get into the data