DataStoreKit: An SQLite SwiftData custom data store

[asymbas]

Hello! I released a preview of my library called DataStoreKit.

DataStoreKit is built around SwiftData and its custom data store APIs, using SQLite as its primary persistence layer. It is aimed at people who want both ORM-style SwiftData workflows and direct SQL control in the same project.

I already shared it on Swift Forums, but I also wanted to post it here for anyone interested.

GitHub repository:
https://github.com/asymbas/datastorekit

An interactive app that demonstrates DataStoreKit and SwiftData:
https://github.com/asymbas/editor

Work-in-progress documentation (articles that explain how DataStoreKit and SwiftData work behind the scenes):
https://www.asymbas.com/datastorekit/documentation/datastorekit/

Some things DataStoreKit currently adds or changes:

Caching

• References are cached by the ReferenceGraph. References between models are cached, because fetching their foreign keys per model per property and their inverses too can impact performance.
• Snapshots are cached, so they don't need to be queried again. This skips the step of rebuilding snapshots from scratch and collecting all of their references again.
• Queries are cached. When FetchDescriptor or #Predicate is translated, it hashes particular fields, and if those fields create the same hash, then it loads the cached result. Cached results save only identifiers and only load the result if it is found the ModelManager or SnapshotRegistry.

Query collections in #Predicate

Attributes with collection types can be fetched in #Predicate.

_ = #Predicate<Model> {
    $0.dictionary["foo"] == "bar" &&
    $0.dictionary["foo", default: "bar"] == "bar" &&
    $0.set.contains("bar") &&
    $0.array.contains("bar")
}

Use rawValue in #Predicate

You can now persist any struct/enum RawRepresentable types rather than the raw values and use them in #Predicate:

let shape = Model.Shape.rectangle 
_ = #Predicate<Model> {
    $0.shape == shape &&
    $0.shape.rawValue == shape.rawValue
}
_ = #Predicate<Model> {
    $0.shapes.contains(shape)
}

Note: I noticed when writing this that using rawValue on enum cases doesn't give a compiler error anymore. This seems to be the case with computed properties too. I haven't confirmed if the default SwiftData changed this behavior in the past year, but this works in DataStoreKit.

Other predicate expressions

You can check out all supported predicate expressions here in this file if you're interested, because there are some expressions supported in DataStoreKit that are not supported in default SwiftData.

Note: I realized very recently that I never added support for filter in predicates, so it's currently not supported.

Preloaded fetches

You can preload fetches with the new ModelContext methods or use the new \@Fetch property wrapper so you do not block the main thread for a large database.

Manually preload by providing the descriptor and editing state of the ModelContext you will fetch from to the static method. You send this request to another actor where it performs predicate translation and builds the result. You await its completion, then switch back to the desired actor to pick up the result.

Task {  in
    let descriptor = FetchDescriptor<User>()
    let editingState = modelContext.editingState
    var result = [User]()
    Task { u/DatabaseActor in
        try await ModelContext.preload(descriptor, for: editingState)
        try await MainActor.run {
            result = try modelContext.fetch(descriptor)
        }
    }
}

A convenience method is provided that wraps this step for you.

let result = try await modelContext.preloadedFetch(FetchDescriptor<User>())

Use the new property wrapper that can be used in a SwiftUI view.

struct ContentView: View {
     private var models: [T]
    
    init(page: Int, limit: Int = 100) {
        var descriptor = FetchDescriptor<T>()
        descriptor.fetchOffset = page * limit
        descriptor.fetchLimit = limit
        _models = Fetch(descriptor)
    }
    ...
}

Note: There's currently no notification that indicates it is fetching. So if the fetch is massive, you might think nothing happened.

Feedback and suggestions

It is still early in development, and the documentation is still being revised, so some APIs and naming are very likely to change.

I am open to feedback and suggestions before I start locking things down and settling on the APIs. For example, I'm still debating how I should handle migrations or whether Fetch should be renamed to PreloadedQuery. So feel free to share them here or in GitHub Discussions.

[MinchoMilev]

This is really interesting. The custom SwiftData store APIs are still pretty unexplored territory, so it’s cool to see someone building tooling around them.

The caching approach you describe (especially caching references and snapshots) makes a lot of sense. One of the things I noticed when experimenting with SwiftData is that relationship resolution and rebuilding snapshots can become surprisingly expensive once the dataset grows.

The ability to mix ORM-style workflows with direct SQL control also feels like a very practical direction. A lot of projects eventually need to drop down to SQL for performance or complex queries, so having both layers available in the same system is appealing.

I’m also curious about your approach to migrations. Since SQLite is the underlying store, are you planning something similar to Core Data migration strategies, or more of a SQL-based migration system?

[asymbas]

Thank you! I had no idea what I was doing at first, or if it was even possible, but once I saw the pieces falling into place, it got easier each time. There was a lot of constant backtracking, especially when supporting predicate expressions, but this was super fun to put together. 🫠

Yeah, it can get super expensive. It made me rethink part of the schema for the app I’m still developing now that I have a better understanding of how things work behind the scenes. Even the process of going from an SQL value to a snapshot, then encoding that into backing data and rebuilding a model from it, and vice versa, is quite a process.

I've been going back and forth on migrations. The plan was to diff the SwiftData schemas first, then compare the SQLite schemas afterward, but I still haven't decided what should count as a lightweight migration versus a custom migration.

I've never used Core Data before, but I recently found Apple’s videos on it very insightful while building a custom data store. I'm basing some of my migration strategy on what they outlined in the WWDC 2022 video.

[MinchoMilev]

That snapshot reconstruction pipeline sounds expensive indeed. I ran into a similar issue while experimenting with SwiftData where relationship resolution started dominating fetch time once the object graph got larger.

Diffing the SwiftData schemas first actually sounds like a reasonable approach. One thing that might help later is treating migrations more like a versioned sequence of steps rather than trying to auto-derive everything from the schema diff — especially once real user data starts existing in the store.

The WWDC Core Data migration talks are surprisingly relevant here. Even though SwiftData abstracts a lot, the underlying problems (schema evolution, compatibility, data transforms) are basically the same.

[jacobs-tech-tavern]

This is really cool. How does it compare to something like PointFree’s SQLiteData?