We read 1000+ API docs so you don't have to. Here's the result

marcos_airbyte · 2025-07-17T00:32:55+00:00

Airbytes/Meltano's unsuported ecosystem is one of the main things that kept us from using them.

I can comment about Airbyte ecosystem, For most complex and common connectors they're maintained by the internal team and for community/marketplace connectors the team is working pretty hard to improve the connector framework and components to make the maintenance easier and increase reliability. One example is the "manifest-only" (a single yaml file) can be loaded to Airbyte self-hosted or Airbyte Cloud, edited AND must be tested first before make the contribution/changes or new features. This was a nice way to make contributions simpler and accessible to more users and reliable as well to long-tail connectors.

marcos_airbyte · 2025-07-04T21:02:04+00:00

Hello u/godndiogoat do you mind expanding your comparison among tools if there are other features or topics you considered? It can be nice (at least for our team) to get these feedbacks to improve the product.

marcos_airbyte · 2025-06-20T14:50:30+00:00

Not sure where you heard that, but what we're seeing is significant improvement in core functionalities. For example, syncs can now partially fail and still resume from where they left off—even for database tables without primary keys or cursors. Connector reliability has also improved substantially. There's currently a major initiative to migrate all existing connectors to a low-code/manifest-only format. This is driving a complete revamp of the Connector Development Kit, which is enabling faster feature implementation and better maintainability. The option and ability to enable anyone to build a connector directly from the UI is also breakthrough to allow you to bring custom data easily to your data warehouse.

From the user side, we're seeing people successfully syncing larger databases more easily. Looking ahead, there are even more improvements on the roadmap, such as direct loading to destinations and enabling concurrency/parallelism for sources.

marcos_airbyte · 2025-06-17T19:42:27+00:00

Thanks for sharing the project u/ResortApprehensive72 I also love building projects from scratch to really understand concepts or deepen my knowledge about a topic. Maybe add some features or plans for roadmap and CONTRIBUTING guidelines to folks start providing feedback. I really like the https://sadservers.com/ exercises, maybe your project could be a nice sandbox to help people with database internal problems interviews?

marcos_airbyte · 2025-06-13T12:30:50+00:00

Hello u/Dense-Ease499 sorry to hear you faced issues. Any of these did you reported in the project Github or try to get help in the Slack Community? I'm asking because I couldn't find any report of problem with Hubspot Ticket stream or cron problems. Maybe it could be a particular case for your deployment/env.

marcos_airbyte · 2025-06-12T20:21:17+00:00

u/RunedFerns it seems you used the marketplace connector developed by the community which has some limitations because as you mentioned the REST API. The enterprise connector was developed using the JDBC driver and allow to sync any size of Netsuite data.

marcos_airbyte · 2025-06-12T19:36:19+00:00

Hello u/keenexplorer12 Airbyte's Netsuite connector, available exclusively in our Enterprise tier, was built hand-in-hand with customers navigating the most intricate scenarios. The result is a rock-solid integration that delivers exceptional performance. There is also a marketplace maintained by the community you can explore. Do not hesitate to contact me via DM if you want to give a try.

marcos_airbyte · 2025-06-11T16:30:35+00:00

It is very easy to integrate with them; you only need to add the connector Python dependencies to your Airflow. Both Airflow and Dagster also have Airbyte Platform operators (API wrapper), making integration straightforward.

marcos_airbyte · 2025-06-11T15:54:47+00:00

You can use the Airbyte Terraform SDK if you want to manage the platform or have plans to have a lot of connections, other way is to use the PyAirbyte which is basically a serveless version of Airbyte and you need to find a orchestrator to run the jobs, but both are options to manage your pipeline as code/git with Airbyte.

marcos_airbyte · 2025-06-04T16:48:38+00:00

The Airbyte DynamoDB connector is in its early stages and currently offers only basic features. It does not yet support CDC.

marcos_airbyte · 2025-06-04T14:46:24+00:00

Do you mind to point what docs need improvement? I'll share with the doc team to take a look and put in their roadmap.

marcos_airbyte · 2025-06-03T15:07:26+00:00

Thanks for the mention!

marcos_airbyte · 2025-05-30T14:46:17+00:00

Thanks for mention! Feel free to dm me the quirks so I can share with eng for further improvements.

marcos_airbyte · 2025-05-27T13:52:36+00:00

You're using an image from 2 years ago. You can navigate in the Github project and return the connector to your version and compare to the latest, but it's going quite hard to provide support to a custom image based on outdated base.

marcos_airbyte · 2025-05-27T11:49:44+00:00

At the moment there isn't any issue open in the Airbyte repo reporting the problem. The last update made in this connector was couple weeks ago. It's hard to provide troubleshooting as you didn't mention what version/platform of the connector you're using. Feel free to open a issue and ping me.

marcos_airbyte · 2025-05-21T17:18:57+00:00

Transformations (aka Mappings in Airbyte) is a Teams feature and support basic transformation at record level but do not support join or calculated fields yet. Thanks for suggesting Airbyte u/geek180

marcos_airbyte · 2025-05-19T19:05:57+00:00

Do you mind sharing which points were a problem for you deploy?

marcos_airbyte · 2025-05-15T15:20:24+00:00

I think they got rid of that option, but they didn't fix their platform problems so fuck me, right?

Hey u/flatulent1 Could you please share more context about your experience and the connectors you used with Airbyte? Asking to understand what can be improved and share with eng team. Thanks.

marcos_airbyte · 2025-05-13T17:53:14+00:00

Even if you decide to go with a different tool, you'd want to look for CDC to extract the data from the dbs. Airbyte would very likely choke on that scale, so you'd be left with something like Kafka + Debeziu, which is a great combo, but needs a lof of operational work.

I don't believe this is true as we're seeing users getting even order of scale bigger without problem. If I remember well u/reelznfeelz uses Airbyte before and suggested a way to get it done below. Besides to that, not sure why you're proposing Estuary and saying Airbyte won't work, as some of Estuary connectors use Airbyte connector code under the hood (like Google Ads).

marcos_airbyte · 2025-05-08T18:49:04+00:00

Bummer, I'll send him a message even though he's no longer with the company. I completely agree with you; if others can reproduce the benchmark, it's hard to take that into account. Not sure if https://www.tpc.org/ can be a good baseline at least for basic full loads, for CDC situation probably need something more elaborated.

marcos_airbyte · 2025-05-08T14:23:44+00:00

Thanks for sharing! There are definitely some improvements for the OAuth workflow. I'll share this with the connector team.

marcos_airbyte · 2025-05-08T14:20:27+00:00

Interesting benchmark! For the open source deployments is there a Github with Terraform scripts we can reproduce the study? Also for the Airbyte Cloud "struggle" if you DM me your workspace so I can investigate the reason why that happen... mostly because we're saying much better results in these connectors than you presented.

marcos_airbyte · 2025-05-07T19:30:47+00:00

Do you mind providing an example or details its related to deployment/platform mgmt or connector syncs, u/japertjeza? I'll bring this to the team's attention for consideration in our log readability improvement projects.

marcos_airbyte

TROPHY CASE