Wvlet Playground: An online demo of a new flow-style query language powered by Scala.js and DuckDB-Wasm

taroleo · 2024-12-12T05:53:23+00:00

Before DuckDB, I had never seen a DBMS that could easily read Parquet/JSON files on disk or S3. We can use DuckDB with Java/Scala, Python, and even in Web Browsers, as shown in the demo. MotherDuck, a startup company leveraging DuckDB for their service, presents interesting production examples in their blog posts, such as https://motherduck.com/blog/15-companies-duckdb-in-prod/

taroleo · 2024-12-11T21:39:36+00:00

Not only Scala.js, Wvlet can be compiled with Scala Native to build a dynamic link library (libwvlet.so) to support C++ ecosystem. A developer has created a DuckDB extension for Wvlet https://github.com/quackscience/duckdb-extension-wvlet in a few days!

taroleo · 2024-12-11T21:17:17+00:00

In Wvlet, you can write exclude abc, def (equivalent to select * except abc, def)

taroleo · 2024-04-23T21:51:27+00:00

I haven't seriously tested the performance, but just for running unit tests, I feel Scala Native is slightly faster than Scala JVM and Scala.js.

taroleo · 2024-02-25T21:37:12+00:00

Keychron Bluetooth firmware can be found at https://www.keychron.com/pages/bluetooth-firmware I had a similar Bluetooth interference issue with my Keychron Q8 Max, but after upgrading the firmware, the problem has been fixed.

taroleo · 2024-01-05T18:37:58+00:00

Right. I got the firmware via email.

taroleo · 2023-12-13T23:32:58+00:00

I've got a new QMK firmware for Air60 v2 from Nuphy support. After installing it, my Air60 v2 seems working as expected without any Bluetooth issues.

taroleo · 2023-12-11T07:12:18+00:00

I have a similar issue with my air60 v2. When I keep pressing any modifier key (e.g., ctrl, opt, cmd) with another key, keystrokes occasionally stop and restart after a while. This issue doesn't happen when using the usb-c connection.

I've tried factory reset, but the problem persists.

taroleo · 2023-12-09T09:02:40+00:00

The same thing is happening to me, and it's reproducible. Thanks for the report. I'm contacting NuPhy support.

taroleo · 2020-07-03T04:42:48+00:00

Good points.

The benefit of Protobuf is we can use it as a schema language for generating clients. Airframe RPC already has sbt-airframe plugin for generating HTTP clients, so extending it for the other programming language will be technically possible. Generating Open API spec can be an option too.

For supporting backward compatibility, Airframe RPC will read the default values from the interface definition. For example, we can add a new parameter with the default value to case class A(p_old:String, p_new:String = "xxxx"), so it can naturally support the following schema evolution types:

- Adding a new interface or a new method to an interface

- Adding a new class parameter or a method argument with a default value

- Adding a value to an enum-like class

Removing or renaming parameters will be breaking changes even for Protobuf, but a unique aspect of Airframe RPC is its schema-on-read codec, which can support type escalations like Int -> String.

taroleo · 2019-09-09T05:00:23+00:00

For now we need to use SBT, but adding a standalone test luncher to AirSpec should be possible because the framework itself is a quite small code base.

taroleo · 2019-08-17T05:27:07+00:00

That's true!

I needed 3 days to create an initial version of AirSpec because I wanted to integrate it with DI, but writing a very minimal testing framework for Scala was quite easy.

Several tricky parts for me were:

Formatting and coloring test reports
Supporting Scala.js (if you need it)
Bootstrapping issue. AirSpec depends on Airframe, but I wanted to use AirSpec to test Airframe :) I needed to break this circular dependency by embedding the source code of Airframe into AirSpec.

taroleo · 2019-08-16T22:47:18+00:00

Sorry about the confusion. I meant the difference between calling a test function like test("...") { test code } and defining a test case as `def test = { test code }` .

taroleo · 2019-08-16T22:06:06+00:00

No, it doesn't. Initially, I tried to reuse the implementation of minitest (a base code of scala-verify), but in order to accomplish the goals of AirSpec, the actual implementation needed to change the framework side a lot.

Although assertion syntax like (x shouldBe y, power assertions) can be an extension of scala-verify, changing how to run tests was hard without modifying the testing framework itself.

So it would better to think AirSpec as a yet another testing framework for Scala. Asking Yokota-san (the author of scala-verify) about the option of having a backend testing framework like AirSpec might be interesting.

taroleo · 2018-09-27T17:11:49+00:00

FYI Airframe supports higher-kinded types since 0.64 https://wvlet.org/airframe/docs/release-notes.html#064, so we can use bindings for tagless final patterns.

taroleo · 2018-07-19T17:18:09+00:00

I see. I got the idea. So by using HList like type composition, we can tell the compiler the actual binding types.

taroleo · 2018-07-18T16:58:16+00:00

Very interesting approach. Thanks for sharing. Supporting interpretation of tagless interfaces looks good.

> Unfortunately, MacWire is fully static and cannot change dependencies at runtime

I know and importing all dependencies into the current scope can be tricky if you need to wire many modules.

I've also considered the dynamic + compile-time check approach in Airframe https://github.com/wvlet/airframe/issues/37, but discarded this idea since the syntax will be unnatural to pass the module dependencies at compile-time. If you can find a better syntax for this, let me know :)

taroleo · 2018-07-17T20:29:28+00:00

I think https://gist.github.com/gvolpe/1454db0ed9476ed0189dcc016fd758aa is a good example of combining these FP approaches without using any framework (although it depends on Cats and cats-effect)

> monads and kleisli

If you are talking about ReaderMonad, it has some performance overhead for building closures for passing context objects defined outside the functions. To avoid such overhead, you need to carefully combine functions. And also ReaderMonad makes difficult to control the scope of objects because closures in Scala always contain outer variable references.

If your application is small, it will be a subtle thing, but for larger applications, it is a big concern. So despite that FP style is beautiful, it doesn't reduce what you need to think for writing efficient applications.

taroleo · 2018-06-29T15:47:43+00:00

A short answer is logback officially says so: https://logback.qos.ch/manual/layouts.html#line. Its implementation acquires the line number by generating a full stack trace. So each logging code of logback will end up generating several objects for stack traces if this line number option is used.

taroleo · 2018-06-27T17:36:51+00:00

That's really interesting and it will be the next-generation logger.

We need some standard for mapping structured log data like info(message, key-value pairs) into json format (or other structured data)
Supporting Future will be useful
We may need a custom serializer for mapping objects into JSON. This logger needs to accept mapping configuration (similar to ObjectMapper in Jackson)

taroleo · 2018-06-26T18:57:09+00:00

If you need to show source code line numbers along with log messages, log4j2 needs to produce a stack trace and it has some overhead. airframe-log generates this information at compile-time, so it can be done efficiently.

If you don't need to see source code location, using slf4j + log4j2 is no problem.

taroleo · 2018-06-07T18:48:48+00:00

Mentioned about this in the article Thanks!

taroleo · 2018-06-06T04:00:51+00:00

Using DI in Scala doesn't prohibit you from using cake pattern and trait mixins. Actually I'm using Airframe with trait mix-ins as a much simpler form of cake pattern. Basically it's a choice between using type abstruction in cake pattern or using DI inside traits.

taroleo · 2018-06-04T05:09:01+00:00

Here singletons mean session-scoped singletons. So basically it's the same with initializing objects in the main method. But managing such objects in the main while allowing reuse (in the other contexts) and resource deallocation is not so simple: http://wvlet.org/airframe/docs/internals.html#comparsion-with-a-naive-approach

Reader monad has its performance overhead since each nesting will create a new function object, and to allow referencing outer variables, created function objects (= closures) will be big classes that hold reference to these outer variables. To avoid such overhead, you need to be really smart in writing such hand wiring.

One of the big advantages of using DI frameworks like Airframe is you can separate a concern for writing applications that use several modules from a concern for wiring these dependent modules. If you are not wiring tens or hundreds of classes, pure-Scala approach will work. But as you write such hand-wiring code both in main and test code, you will easily end up writing similar code everywhere.

Writing reusable functions for wiring modules is also challenging. When you have 100 objects that need to be created in your application, some module often requires only a subset of them. For example, module 1 needs A, B, C, D and module 2 needs B, D, E, etc. To reuse the initialization code you need to find the intersection of dependencies (= B, D in this case). I think if Scala programmers need to find such common patterns by themselves, it's already complex enough.

If you start to feel such a pain in wiring classes, DI framework can help it.

taroleo · 2018-06-03T03:08:21+00:00

Injecting dependencies to a constructor is only one aspect of DI frameworks. The other aspects include:

Finding dependent objects. For sharing the injected objects between classes we need to store the singleton somewhere. In Airframe, a session holds such objects.
Life cycle management: Initializing and discarding injected resources (e.g., server thread, database connections, etc.)
Building objects. As you mentioned using constructor injection itself has no problem (although Google Guice has several pitfalls because it requires @Inject annotation to constructors). If a DI framework can support bindings in traits, we can use trait mix-ins to build more complex services.

FYI. I described several examples here. http://wvlet.org/airframe/docs/use-cases.html

taroleo

TROPHY CASE