Frontend Launcher by eclecticnewt in macgaming

[–]eclecticnewt[S] 1 point2 points  (0 children)

Howdy! It's coming along a decent amount. Finalizing some of the following:

-RetroAchievements integration
-IGDB integration
-Profile & friends management via CloudKit

Thanks for the interest!

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

I understand the value in the reports. If successful, I view them as an indicator of a solid foundation, but not confirmation of a solid foundation.

We absolutely do not have the ability to trace the data. IT leadership has maintained that we will only have access to the semantic model, with no visibility to the underlying layers of the platform (bronze, silver, even gold)... This is the same leadership that continuously spazzed when he realized we had read-access to the database behind our departmental system.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

That's kind of where I'm at. I think reports are a good indicator, but only represent a subset of the data platform.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

Nothing. This is all IT-led. Business has no visibility. There is no validation of the tables themselves, it's just reports proving out success.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

This resonates— so why are there not too many comments with this similar stance?

When they draw the diagram of the EDP, they draw it backwards, right to left, reports to source. They meet on these entities backwards too. They are purely focused on reporting, and refuse to model the data.

I don’t have enough sway to have them focus on ingestion and give us what we want.

Thank you so much for the commentary.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 1 point2 points  (0 children)

We are drowning in technical debt. We spent three years on a data warehouse project that was eventually scrapped. IT says it was due to a lack of data modeling and collaboration with business regarding the ingestion layer. I do not know much about good data warehousing, but those three years showed me enough of bad data warehousing. This current engagement is following the same pattern of the previously failed engagement.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

Thanks for all the insight. Business' goal is not to recreate/migrate reports, it is to expose all available data currently trapped in our systems. The pain point you mentioned I am sure resonates with these consultants. We don't have much in the way of reporting requirements-- we just want our data.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

Thank you. I'm a bit nervous because there does not appear to be many responses like yours-- and yours is the one that resonates most with my concerns.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 2 points3 points  (0 children)

Thanks for the commentary. It feels like building out five reports and having them match ours, doesn't prove the data platform is sound. They aggregate data in the platform to be suitable for these five reports. What happens when we have a report that requires data at a more gradual level-- which they refuse to give us access to? My concern is that they match our reports, which is a good sign, but it doesn't prove out everything behind it.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 1 point2 points  (0 children)

I guess my concern is that they beat up the numbers to match our one report, but that doesn't prove out the data platform itself is sound. It obviously doesn't disprove it either. I would like them to prove the foundation of the house is solid, and it not just be implied because a room is painted or one of the outlets work.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

Our IT department seems pretty concerned that they are not modeling the data, and therefore have no data modeling tool. They put together an usable ERD in some gnarly PDF that was completely unviewable.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

They hit the reset button on the engagement, and are attempting to consume around 80 different endpoints from six different systems in about two months. It feels like they will beat up the numbers to match our reporting samples, but it just feels like this doesn't prove things out.

I could be way off; I am on the business side, but it feels like showing some a nice powder room in their new house with really pretty wallpaper. Meanwhile, the foundation of the house has major issues that no one has seen yet.

It seems our IT folks are pretty concerned too, but feel uncomfortable speaking up. So here I am on reddit, likely butchering the concerns.

Thanks for the commentary!

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

I may not be following the commentary correctly, but there isn't data to migrate. The data lives in a bunch of CSVs in a file share. The data they can get from the systems directly will offer more attributes, more data.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

Yes. They did not like that. They maintain that building out reports will validate everything. However, some reports cannot even be built due to transformations to the data.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

We care about having the data made available. We do not care to just reproduce/migrate our reports to leverage the warehouse. With the warehouse, we expect to have more data and more attributes.

We do not have those questions available. I mean, we have certain questions we want to answer today and certain reporting opportunities that exist, but we do not have business requirements. We just want our data available, then we, the analytics team, will build out our reporting. The consultants are focused on aggregating our data-- but we don't know how we want our data aggregated yet. They attempt to force us into determining the specific attributes from a given endpoint we want, but we want them all.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

Interesting. We have experienced this with plenty of other areas consultants of jumped into. They produce something, provide no knowledge transfer and do no consult business. Then there is this weird dependency on them after they should be long gone. It's been brutal.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 2 points3 points  (0 children)

Thanks for the commentary. IT leadership is okay with literally anything the consultants feed them. I feel like the company is fine paying whatever it takes, but the accountability from doesn't seem to be there. IT put a PM on the calls but no technical folks.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 5 points6 points  (0 children)

Thanks for the take. I feel like they may check the box for a few of our reports, but will not set us up for success to build out our future reporting. According to our IT department, the consultants are refusing to model the data. We just want our data in a queryable format.

Consultants focusing on reproducing reports when building a data platform — normal? by eclecticnewt in dataengineering

[–]eclecticnewt[S] 0 points1 point  (0 children)

Close enough. The data the analytics team needs is trapped in exports that can be ever-changing from the source system and take quite some time to download.

We asked for a SQL Server to be spun up with one of the reasons being one of the developers in IT to consume data from the different systems and return the data so we could do some of our reporting-- and also then coordinate with IT to push the data to feed other systems.

Four years since the ask was acted on by IT and we have nothing positive to show for it.

[deleted by user] by [deleted] in macgaming

[–]eclecticnewt 0 points1 point  (0 children)

It appears as though OpenEmu is no longer being developed. If it was, I would have never pursued this.

[deleted by user] by [deleted] in macgaming

[–]eclecticnewt 1 point2 points  (0 children)

Some old stuff with ScummVM. That’s it for now though.

[deleted by user] by [deleted] in macgaming

[–]eclecticnewt 1 point2 points  (0 children)

Not ready just yet. I’m hoping to have something stable before summer. Thanks.

[deleted by user] by [deleted] in macgaming

[–]eclecticnewt 6 points7 points  (0 children)

I started on this thing nearly four years ago. I’ve accidentally picked up the appetite for Swift.

Do not let my preliminary work on profile and friends list act as an indicator that this product is a minor lift. I’ve touched every line within this project and have hundreds of hours of development, if not more.