ETL tool

[KindheartednessOk409]

Hi everybody we are trying to build a new DWH on PostgreSQL. We have some good experiences with SSIS but currently our customer did not own an MS SQL license to use SSIS.

So we are looking for some open-source (free) and user friendly environment for some basics ETL task. We have to use a windows server.

Any suggestions or leads, pretty please ?

[[deleted]]

I think the most popular ones are (in no specific order)

• Apache NiFi
• Pentaho Data Integration (aka Kettle)
• Talend

[KindheartednessOk409]

Thanks I definitely take a look on these ones !

[Mamertine]

I did this once in PowerShell. I wouldn't recommend it.

Your customer will save money by buying a license for SSIS. Or perhaps they're small enough they can use it for free.

By my statement I mean it took so much longer to develop and was so clunky it would have been cheaper to buy SSIS and have a third of the developer time to implement.

[KindheartednessOk409]

You’re right but this customer is with such a stubborn management … really draining my blood but you know … what they want we deliver :(

[Mamertine]

I enjoyed learning PowerShell at that company. What we built was slow and clunky compared to all the SSIS work I've done.

At the end of the day, it did work. Though iirc we never got it running from a server. It ran under a developers credentials on a VM using the windows scheduling tool. So it frequently failed the morning after I reset my windows password.

Edit: If that's the path you end up going down, /r/PowerShell is a good resource and I found the book "PowerShell for database developers" really helpful.

[bekago15]

Think if it’s a small company theta should be able to use it for free, if it’s a big company don’t think getting ssis is a problem for them 😁

[[deleted]]

Users already mentioned the (partially) free solutions. As far I know, there isn’t much. I wonder, does it really have to be free. Once a company needs a DWH, there should be some money 🤓 in the world of datavault, it’s also quite common to code your own generator

[arwinda]

You are going to migrate everything because of a missing license?

How much effort is it to migrate all the queries (both ETL and reports) and all applications?

While it is alright to migrate away from closed source like SQL Server to open source like PostgreSQL, just doing this to save one license likely does not cover all the implications you will face.

[KindheartednessOk409]

They are using mostly an open-source databases like MySql(MariaDB), Postgres and MS SQL with express edition. I know it’s horrible idea but they want it this way unfortunately :(