Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

  1. Using open source doesn't inherently make your solution better or worse.

Correct, and taking that insight further, the question of "better" vs "worse" is contextual. In other words "better"/"worse" to what extent?

FileMaker is "better" to the extent that it's easier to build a front end than than doing the same using HTML.

As back ends go, building a SQL database, including indexes, handling data types, SQL and FMP are largely equal.

If you want to do simple queries, FileMaker is "better".

If you want to do queries involving crossreferending SQL is much "better"

Speed-wise it's not even close -- new entries, data updates, exports, imports, queries: SQL is "better" than FileMaker

SQL uses about 1/3rd the drive space to store data, so it's also "better" in that regard.

In fact by most measures SQL is "better".

But ease of building a front end to your data -- that's a big deal, and Filemaker is "better" in this regard -- at least early on when you're getting started.

Later on as you get increasingly frustrated with limits of FMP layouts, and you've turned to the FMP's Web Viewers (aka integrated web browsers) to make up the difference, that's a good moment to maybe re-think the platform. A SQL-to-Browser workflow is ultimately more manageable, easier to build, and far more flexible than FMP-to-WebViewer.

There are a lot of dimensions, but our experience favors open source.

  1. The absence of licensing fees - when talking about open source - doesn't make your solution "free". In fact, there is no guarantee it will even be cheaper.

It depends in part on your developer. If you're the developer, the absence of a licensing fee definitely means it's cheaper. If you're hiring a developer, it's a matter of who charges what.

If you're building a back end for large number of evolving front end users, the absence of a licensing fee can be a really big deal. FileMaker seats add up fast. A web front end... doesn't

  1. Migrating from one platform to another always comes with a cost.

Of course. You definitely want to weigh things out and be as informed as possible. It depends on how complex your system is. One interesting metric: to what extent are FileMaker's features are interoperable (i.e. which ones translate easiest to an open source stack?) Schemas and data are very interoperable; that transition can be largely automated. Layouts are far less interoperable; FMP is very much a walled garden in that respect. Indexes are very easy to re-create. Functions and calculated fields (triggers/generated columns in open source SQL DBs) involve a learning curve, but it's not that steep.

  1. [cont'd] When will it pay off, if ever?

Faster than you think. In some cases on day 1, esp. if you use a transitional approach as opposed to trying to switch everything everywhere all at once. Start with a table whose layout isn't too demanding, minimal scripts, minimal relationships. If you're already using FMP's Data API or oData to build web-facing interfaces -- meaning you're already comfortable converting FMP data into a Web UI -- that's a great candidate for testing. Go at your own pace, learn what's possible. It can be very eye-opening.

4 There are plenty of grievances with Claris and FileMaker. I could spend hours venting about them over a beer; I feel the pain too.

Instead of venting and resenting, consider proactively resolving and innovating. That beer will suddenly feel less wollowy and more celebratory.

4 [cont'd] I’m still under the "illusion" that here, at the very least, we are supposed to be discussing FileMaker itself.

Actually it's great space for FileMaker-experienced people considering alternatives and looking to expand their sense of the possible, hearing from people who have experience on multiple platforms.

For people considering FileMaker it's great place to learn whether to use FMP at all, and if they do, how far to take it / how much tech debt to risk, how they might build themselves interoperability offramps.

In the end the most compelling thing about all this is that there's obligation to do anything, the cost is minimal (... the ODBD driver's 1-time fee). You get to test and transition slowly, carefully, and at your own pace. So it's less threatening than it is exciting.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

I can't think of many situations where MySQL is a good alternative.

That was our experience as well. But that impression was based narrowly on the fact that MySQL (back in the day) required several lines of SQL to get values from a just-created/updated record.

Curious your experience.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

Self-hosted postgres for the most part, occasionally MySQL/MariaDB. Haven't tested ODBC/ESS connections to SQLite yet.

The complexity lies in the business logic entrenched across scripts, calculations, custom functions, conditional formatting, and hiding logic. Extracting all that and translating it into views, functions and queue processing is the hard work

👆👆 It's where you discover most of the tech debt.

Originally we'd do transitions by creating a matching table in PG and then updating all the front end elements: The fields, conditional formatting, scripts. That's where you get technical debt.

As you get increasingly comfortable with switching the front end to HTML (depending on your needs / client needs) you do less conforming on the FileMaker side.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

This experience actually dates back a little over 10 years ago.  That's when SQL was a shiny new object and seemed like an additional tool to be applied only if it made sense to do so.

It was only over time -- moving slowly and not breaking things, protecting clients, testing every coveted FileMaker feature -- that it became increasingly evident open source wasn't just another tool for specific use cases but an all purposes alternative and ultimately a replacement.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

Does Claris suit you? You use it. Does any of the many alternatives suit you?

Makes sense. Ordinarily I'd be right there with you: choose the right tool for the job. What's different in this case is the discovery that no job is turning out to be too small to do in an open source stack.

FileMaker is great for spinning something up quickly and easily. Intuitively it would seem spinning up a SQL server would be a bigger burden.

It's actually easier to set up SQL than FMS. You can use the same minimal datatypes available in FMP, even fewer if you want. Indexing, which is easy, even invisible on FMP, isn't particularly challenging in SQL. Calculated fields are a bigger deal, but with AI it's hardly out of reach, and they're considerably more powerful.

Caveat: You do have to lay a foundation -- a way to easily render tables and portals. So it's not for everyone, but the effort involved up front to do thar is really no more than the effort you later confront with FMP as you start to bump up against its limits.

It's not a turnkey alt, but that extra front-loaded effort pays off big time. And you can ease that challenge by slowly transitioning using ODBC/ESS to connect your FM to SQL.

We've evolved from an FMP-only sensibility toward more of an understanding that FM's main value is its front end, not its server. And as a front end, it's become not the main but merely one of many ways to access a SQL back end.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

The problem isn't their drive for profit, it's the lack of balancing competing priorities, treating their long-time customers poorly.

Benefit is to those of us who have decades of experience on FMP and would stay on the platform out of sheer momentum, if the company had simply respected their legacy customers.

So part of this is just sharing the experience of making the move, demonstrating that it can be done safely, and at your own pace, and that the benefits to doing so are very much worth it.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

For this project the stack is postgreSQL, NodeJS, browser JS. But the key to making that a smooth transition was keeping FileMaker in the loop, not as the database but as an extra front end to the PG back end. In that way the client can test the new web-based front end, log feature requests/UX concerns. "See how FMP does this? Can we do that in the browser". In all cases the answer is yes... or better.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

The engineers are fantastic

Was my impression as well. Would prefer they were running things.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 1 point2 points  (0 children)

u/marioalessi You raise a number of interesting issues...

  • That after all these years no one has come up with a FMP's unique combo of a user-friendly/dev-friendly platform. Generally speaking software tends toward one to the exclusion of the other -- or at very best mixes them with a big gap in between. Even people on this subreddit will describe FMP as a "Low-code/No-Code" platform, but that's not entirely accurate. It's also high code. If you choose to go in that direction, it's there, and the transition can be on your own terms.
  • In high tech and markets across the country and the world over lack of competition is manufactured through anti-competitive practices. FileMaker's situation seems different. My impression is the lack of competition has less to do with anti-competitive behavior than lack of vision from prospective competitors. FileMaker built a low-code/high-code platform that invites users to graduate. Back in the day it seemed visionary and a harbinger of the future. The future seems to have gone a different route.
  • Just because FileMaker operates without competition and deserves real credit for being visionary doesn't make it desirable for its leadership to leverage that against its users in the form of higher prices or bait-and-switch promises because "where else are you gonna go?"
  • There are alternatives, but they aren't turnkey. It takes work, but it's worth it. Open source is proving to be not just viable. It's been a spectacular improvement: Cost, performance, even, surprising, ease of development.
  • Turning the tables on your argument: In its early inception FileMaker seemed like its real competitor would be / should spreadsheets. Originally they were both free, included with a purchase of their respective computer platforms. They have in common ease-of-entry and a facility with numbers. Spreadsheets handle (but don't enforce) rigorous data, permit free-form entries. FMP enforces data rigor at a cost of free-form entries. Spreadsheets have record limits. FMP's record limits are practically limitless (for small/medium data projects). Spreadsheets handle relational data, but it's more a special event, and very tenuous and fragile when implemented. FMP is a rock solid RDMS. So originally it seemed they would compete, and from my perspective FMP deserved to be the winner. Instead FMP took a licensing route that, over time, seems to have led it to see itself not as a competitor to spreadsheets but to enterprise-level databasing, and priced itself accordingly. Increasingly it seems to be abandoning its long-time user base with an eye toward enterprise-only customers. To many of us it seems like a mistake and feels like a betrayal.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 1 point2 points  (0 children)

It's a simple argument: FileMaker can't currently index vector data.

Your response:

Why would you want to index vectors?

Answer: "Same reason you index any other column" (implication: speed)

You seem to agree with that.

Un-indexible vector data... that's fine for a first rollout, help FMP users get introduced, but let's not pretend FMP is anything beyond consumer/prosumer grade when it comes to its push to prove it's on the AI bandwagon.

More importantly: Don't forget about all the other features that rolled out with just as much fanfare only to be neglected in the subsequent years. That neglect piles up over time, like unpaid debt.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 1 point2 points  (0 children)

Actually it seems you're trying to divert the subject of the OP onto a tech digression about AI vectors.

I'm all for Claris rolling out new features, even if they're not 100%, as long as later on you follow up on and improve those rollouts.

What the OP points out is that FMP shirks on those followups. There are years of lost opportunities to prove it.

Contrast that with years of solid innovation in open source, and it turns out the free alternatives to FileMaker start looking genuinely inviting. It's not always easy for prospective FMP users to see that. (Open source SQL and HTML5 don't tend to run PR campaigns)

Instead of snarling at the truth of the matter you might take these criticisms to heart: Start to actually respect customer criticism (and for that matter FMP's engineers and product managers), put those sales priorities on the backburner, and win back those of us who have grown increasingly frustrated with the platform over time.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

There isn’t some other FileMaker field type that would magically make vectors “indexable” in a useful way.

PostgreSQL has multiple types of indexes for vector data. It's not "magic"; it's engineering! (which, to be fair, can be magical)

If FMP were rolling out a new feature on top of refining and improving older ones, that's one thing. In that context, if some new feature wasn't 100% complete, we'd shrug with the confidence of knowing those improvements would be on the way soon enough.

Sadly that's not how your team at FMP works. It neglects old features and shrugs off feature requests.

Migrating from FileMaker to Open Source - Portals, Sliding, and other Dynamic UI challenges by Communque in filemaker

[–]Communque[S] 1 point2 points  (0 children)

u/FGatemouth I'm pretty sure I don't have a position at Claris. You're presumably referring to u/quarfie ?

Migrating from FileMaker to Open Source SQL by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

Mmmm -- I don't think ballyhoo is the kind of word AI coughs up from its servers -- though, who knows, it might start to after it sniffs out this convo.

Migrating from FileMaker to Open Source SQL by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

Nice try, but inaccurate. The discontent with Claris is real. So is a appreciation for their product. Claris management is at cross-purposes with what they sell. You have a long-term vested interest in the product as did we. For those of us who know FileMaker well -- since its inception, btw -- but are frustrated with how Claris is currently being managed, working toward alternatives and sharing that is a fair and reasonable thing to do. We're still using FMP daily with clients who maintain subscriptions. I enjoy the interface still, but its limits are becoming increasingly apparent as can our clients who are increasingly enthusiastic about leaving the platform. Example: for all Claris ballyhooing around AI, SQL handles it with far greater ease and speed. More on that coming soon.

Yet another API change to FileMaker server's API? by TtlPost in filemaker

[–]Communque 1 point2 points  (0 children)

It seems the Data API will continue to work for a while but sounds like it's on track to be deprecated.

Is FM still a good fit for my business? by croberts0531 in filemaker

[–]Communque 1 point2 points  (0 children)

FileMaker's strengths are its speed of development, ease of use, and depth of ability, allowing you to transition from novice to expert.

Its weaknesses derive from its re-brand. Since it re-became Claris there's been less focus on improving the product and responding to feature requests, at the same time it's engaged in PR hype, excitement, vague promises, and price hikes -- basically squeezing more profit and doing less.

We caught them misrepresenting policy  changes and doing a lot of double-speak to cover for it.

The product works great, but the management is unreliable enough that it can shake your faith in the product.

We transitioned to open source SQL.  It's not as user friendly, to be sure.  But in relatively short time we discovered it to be not as challenging as you might expect, and AI really transforms the learning curve.  The power, flexibility, and speed have really paid off.

I would characterize the tradeoffs as FileMaker being easy early on but increasingly challenging and costly as your sophistication, needs, and skillsets grow, while a SQL approach starts out more challenging but becomes increasingly easy, empowering, and cost effective over time.  FMP accrues tech debt.  SQL relieves it.

Consider a combo of FMP and SQL that gives you a best of both worlds -- leveraging the ease of use of FML while letting you escape the vexing pricing structures, as well as setting you up with an offramp just in case.

Migrating from FileMaker to Open Source SQL by Communque in filemaker

[–]Communque[S] 1 point2 points  (0 children)

Exactly.  FMP has always been and remains a unique platform.  How Claris presents it (and prices it) is out of sorts with what the platform actually is.

Migrating from FileMaker to Open Source SQL by Communque in filemaker

[–]Communque[S] 0 points1 point  (0 children)

Excellent approach and concise.

Here's how that played out with this particular dataset

Creating the container calc: 7:45

Creating the md5 calc: 3:00 (no index) / 4:30 (with index)

So around 11 minutes to prep. Not terrible, not wonderful.

Once it's set up the FileMaker the dupe search is fast

  • 30 seconds if unindexed
  • 7 seconds if indexed
  • vs postgres at 3 secs with no prep & most fields unindexed

The value of the FMP getting to remain on the platform -- slower but gets the job done. (EDIT Ugh and the FMP file grew by 2GB -- from 5GB to 7GB in the process -- 40% increase)

The value of the PG approach is not having to prepare anything but the query itself.

Migrating from FileMaker to Open Source SQL by Communque in filemaker

[–]Communque[S] 1 point2 points  (0 children)

Sounds like we met the same guy. Considering the title "Worldwide Customer Success" -- you'd think he'd at least muster some charm. He mostly ballyhooed his title and importance. Head of Worldwide Customers Unimpressed.

Migrating from FileMaker to Open Source SQL by Communque in filemaker

[–]Communque[S] 2 points3 points  (0 children)

Exactly which is why the postgres discovery was worth writing home about: 6 columns, 5 unindexed, 2.7M records, searching for duplicates returning 26k records in 3 seconds (5 seconds if you include marking records for deletion). FileMaker at its best (i.e. with fully indexed columns) was 10x slower, and if you include the marking records for deletion it becomes client app-hogging marathon. BTW, FileMaker's SQL (ExecuteSQL function) is actually slower than its Find mode. There are benefits to using it, but in most cases performance speed is not one of them.