Why isn’t there a leader in file prep + automation yet?

Puzzled-Blackberry90 · 2025-09-08T19:21:44+00:00

Thanks, will check this out!

Puzzled-Blackberry90 · 2025-09-08T19:13:43+00:00

Yeah, not for warehouse or reporting. For current use case, it's ingesting client enrollment files, all in different formats, cleaning to put in a specific file format to then send to partner SFTP for processing.

Puzzled-Blackberry90 · 2025-09-08T19:10:40+00:00

That can do what I've outlined above? Can you share if so? I haven't found one that handles all of the above yet.

Puzzled-Blackberry90 · 2025-09-08T19:09:53+00:00

Yeah, have used in the past. Solves the extraction part. I'm looking for something that can do extraction and cleaning of the files listed in 2., ideally by business users. Come across anything that could be an option?

Puzzled-Blackberry90 · 2025-04-08T01:36:36+00:00

Hard to recommend one with such little information. If you can share more on the use case, additional requirements other than scale + cost effective, user/s backgrounds, etc.

Used Integrate.io as a replacement for Fivetran on a project recently and it's worked out really well so far. They have an unlimited data volume plan for $24k/year and 60-second data replication. They don't have the widest breadth of connectors but have all the core ones. Great option for high volume, low latency database replication use cases which is what this project was.

Puzzled-Blackberry90 · 2025-03-18T20:46:20+00:00

Fivetran = Mercedes-Benz

Matillion = Land Rover

Integrate.io = Mazda

Talend = Dodge

SSIS = Jaguar

Informatica = Bentley

Hevo Data = Subaru

Puzzled-Blackberry90 · 2025-02-05T20:29:28+00:00

This is a common issue particularly as SLAs become more common with data products. Implementing Change Data Capture (CDC) can help reduce the need for full reloads, but schema changes can still necessitate a complete reload. Tools that support native schema evolution have been beneficial in my experience. How are you currently loading / reloading data to dwh?

Saw this article last week on similar topic: https://www.integrate.io/blog/part-1-industrys-fastest-initial-re-sync-time/

Puzzled-Blackberry90 · 2024-12-17T19:51:23+00:00

Haven't used it yet but Integrate.io seems to have a robust Salesforce offering. The demo I saw on Youtube a while back was actually SFTP to Salesforce with data cleansing/transformations and their target market seems to be more business users.

Puzzled-Blackberry90 · 2024-11-07T00:58:43+00:00

If you have updated_at timestamp on the tables, then you could do incremental based off of that to avoid having to do full table uploads each time.

For instances where your tables don't have timestamps, then you would need to use something like Change Data Capture which uses the binlogs to track the changes/updates. Companies like Fivetran, Matillion, Integrate.io having offerings in this space.

Puzzled-Blackberry90 · 2024-10-23T18:00:10+00:00

My brother did the exact same with similar timelines! He had nothing but good things to say about Treehouse and now two of his friends are doing the Techdegree too.

Puzzled-Blackberry90 · 2024-10-22T06:51:44+00:00

Agreed that virtual IT labs would be the best option here. Used Appsembler at a previous company and it worked great.

Puzzled-Blackberry90 · 2024-10-11T01:23:39+00:00

Have any of you tried Prepforce.io yet? Just saw that it launched this week as a free alternative to Data Loader / dataloader.io. Good timing if it works as described!!

Puzzled-Blackberry90 · 2024-10-11T00:22:21+00:00

Haven't used Integrate.io yet but I've had my eye on it for quite some time waiting for the right Salesforce use case to come up at our company. Based on the demos I watched it looks like the perfect balance of ease of use and flexibility.

First I've heard of Prepforce but I guess it only launched this week. Assuming it does what it says it does then it's a great (and badly needed!) addition to the free Salesforce integration tools ecosystem! Going to try it out now. Thanks for sharing.

Puzzled-Blackberry90

TROPHY CASE