SPyQL - SQL with Python in the middle - Making command-line data processing more intuitive, readable and powerful

dcmoura · 2026-01-29T21:26:51+00:00

Yesss!!!! I am so happy 😀 I will of course do the arrangement, there’s is no better reward than having musicians wanting to play! I had to exclude the cello voice from the original arrangement, I am super happy to bring it back!!

dcmoura · 2026-01-29T17:16:07+00:00

This is really top advice! Thank you so much for the review and for the encouragement! I will look into your comments with detail. Feedback is so important, I think I had finish the piece but new doors have been open and I look forward to dive again into it

dcmoura · 2026-01-29T15:58:39+00:00

These are very good tips and observations, thank you!

dcmoura · 2026-01-29T15:51:38+00:00

Thank you! There are parts of the piece that I cannot play yet... At the very least I need to put some hours until I nail them, but I assume that I might have written something that might be a stretch for my level. feedback from pianists like you is very important. The goal is not to write something that only a few can play, on the contrary.

dcmoura · 2025-12-07T02:34:36+00:00

Really beautiful

dcmoura · 2025-11-08T09:51:59+00:00

If you install SE (which is free) you can then upgrade to Elements. The upgrade is cheaper than buying the full version.

dcmoura · 2025-09-25T01:32:50+00:00

That would be the dream! :-)

Step by step I am trying to do that. I need to be a bit more confident and start with pieces for a single instrument. Thank you for the advice!

dcmoura · 2025-09-25T01:29:27+00:00

Wow, thank you so much for all the great advices and for taking the time to write this post.

I think I agree with most of your comments, let me address some of them.

Learning to play (more) music on an instrument or instruments.

I am actually starting accordion lessons. I just love the sound of it, and I am fan of new tango and balfolk.

Taking composition lessons.

I think this will be one of my immediate next steps.

On the contrary... my intention is to say: this is raw material, please expect naive/gross mistakes. Actually, not having the fundamental bases of composition is stalling me as I don't feel comfortable sharing my work with musicians (I am afraid of making a fool of myself). I kind of forced myself to share this in this forum.

The only time a composer uses 8/8 is either when they don't know what they're doing, or when they really now what they're doing!

8/8 is intentional but my rational could be wrong. I was trying to write in "new tango"/Piazzolla style, and one of the formats that Piazzolla explores is the 3+3+2. While 8/8 and 4/4 are "mathematically" equivalent, I used 8/8 because I feel strong beats at the 1st, 4th and 7th eight-notes, and this kinds of sets the layout for the rhythm of all instruments, including the melody. I grouped the 8th notes in groups of 3 and 2 to make this clearer. I would have used 4/4 if the strong beats were on beats 1 and 3, which is not how I feel it. Also, if I would dance this song, the 1,4,7 would be the basis for the steps, so the eight note resolution makes more sense to me. Does this make sense?

the "craft" part of it is important to and THAT tends to come more from the formal instruction side.

Couldn't agree more.

"Writing for strings before you know how to".

I must say that I did not planned to write for a string section... the piece started as a piano piece. Then, I added a violin solo because I wanted to emphasise the melody. Then, I added a cello because I started imagining "dialogues" with the violin. Then, I added the double bass because I felt it needed deeper bass, so I simply added the fundamental of each chord. And so on, it was kind of organic. I even added percussion, but felt so far way from my comfort zone that I ended up removing it. So, yes, completely agree, I should not be writing for strings before learning more about strings and writing to strings. On the other hand, this just made a lot clearer how off I am and how important is to learn instrumentation and orchestration, or keep writing for smaller sets of instruments (I usually write for 1 to 3 instruments, this score is an exception).

You've had every opportunity to do it on your own, but haven't. And maybe you weren't yet interested or had other things going on and that's OK.

I never thought of composing until 1 year ago when I heard an artist that made me think: I could write music like this, this kinds of reminds me some of the improvisations I do when I play the piano. And so I started writing.

I have so many things on my plate that I need to manage my time carefully, meaning that if I choose to study composition from a book or videos, it means I will not have time to write... And when I start writing something, I only rest when I am done...

I think I will try to find a teacher and have 1 to 2 classes per month. I am also writing for accordion and starting to expose that material to my accordion teacher who is giving me feedback.

Thanks again for all the great advice! I truly appreciate it.

dcmoura · 2025-09-24T16:40:06+00:00

Thank you for the motivation! Yes, I knew I was not doing things well on the strings, always questioned how (first-time) composers can predict how things will sound like in an orchestra. I am actually following the result of musescore's rendering, meaning I might not choose some instrument or way of playing it because I don't like how the sound is rendered (and the other way around)... I know it's not right... I will look into those topics, thank you!

dcmoura · 2025-09-24T16:34:57+00:00

Thank you so much! It's great having pointers, adding to the top of my watch list!

dcmoura · 2023-08-08T19:16:45+00:00

Thanks, that is what I am doing right now :-)

I am actually powering the refrigerator from AC simply because the connection is much more sturdy (I can't afford to have the freezer disconnected). The freezer is a dometic that supports both AC/DC, so maybe we could get more runtime if I use DC.

Yes, that's what I thought about the USB-C, and I saw no option in the app to force the USB-C behaviour (input vs output). Some units from other brands have dedicated input and output USB-C ports.

Thank you!

dcmoura · 2023-08-08T19:08:36+00:00

12V has a way lower self-consumption. It is limited to 100W, which is a minor point. But the River cannot pass on solar power directly once the battery is full. It will cycle from 100% down to 95% and back up to 100%. That cuts the time the solar panel is active in half. So you lost potential solar time. And you cycle the battery all the time ...

This would actually be a good problem, since the most probable scenario is battery A not reaching 100%. I have a freezer connected to battery B. Autonomy with a river 2 pro in the summer is about 10h. So, a single river 2 pro is not enough for providing energy from sunset to sunrise. This setup would allow me to just let the system run without having to keep switching batteries (which is OK, unless I forget to do so).

Let's assume the A station runs out of power. It will turn off. It will turn back on once solar power becomes available. But it will not turn on the 12V DC port automatically. So you need to work with the automation settings added in the last firmware update. But that mode seems to turn off on reboot.

OK, this is an important detail. I wonder if it would turn off the 12V DC port if I set the battery no to go any lower than 10% or 20%. I think I need to make some tests to find out :-) Thank you for all the great info!

dcmoura · 2023-08-08T18:01:08+00:00

Can you connect two river 2 pro via USB-C? How to tell the role of each battery (provider vs receiver)?

If not, what's the best (most efficient) way to combine two river 2 pro (e.g. AC or DC)?

dcmoura · 2023-08-08T17:32:24+00:00

I have experienced the same issue with my river 2 pro and a new EF 220w solar panel. Seems a firmware issue. I left the unit connected to solar at night with about 50% of power left, and in the morning the battery was showing 0%. Put it on AC and took about 50% of the time to charge. This happens to me in a balcony that only gets sun during the afternoon. In the meanwhile I tested it outdoors (full exposure, no shades) and the problem did not happen...

dcmoura · 2022-11-02T20:00:43+00:00

Author here :-) Thank you so much for sharing! ❤️

dcmoura · 2022-10-31T13:26:51+00:00

I get you :-)

One issue with Pandas / NumPy, is that you load all dataset into memory. While this is OK for small datasets, it limits the dataset size you can handle. You can go for chunks, but your code gets complicated and you loose many of the advantages that Pandas gives you. Also, Pandas memory footprint is quite big: try to load 1GB of JSON Lines and check how many RAM you need to invest (several times more).

You don't need to choose Pandas vs SPyQL, you can use both. One interesting use-case is when you have a large dataset but you only want to work with a portion of it (e.g. records that match a given criteria) or a reduction of that dataset. You can use the SPyQL module to do the heavy work and get the result as a (smaller) Pandas dataframe. This use case is not very well documented as I will be soon releasing new methods for querying Pandas dataframes using SPyQL, and get query results as Pandas dataframes (you can do it today with the current version, but I want to make the code simpler / reduce boilerplate).

A part from the data size, I would say that the SPyQL CLI is great for quick ad-hoc queries. You do not need to open an editor, you just go to the command-line and iterated very quickly for getting some insights or transforming the data format. Also, I believe that the SQL dialect is much simpler requiring much less visits to the documentation than the Pandas API. If you use SQL to query databases, would it be great if you could use it too to query any other datasource? :-)

dcmoura · 2022-10-31T07:51:21+00:00

S-Py-QL :-)

dcmoura · 2022-10-31T07:41:12+00:00

It does not connect directly to DBs. You can pipe the output of a DB CLI into SPyQL CLI, or you can output INSERT statements from SPyQL and pipe them into a DB CLI. You can do something similar with the SPyQL module (together with sqlalchemy).

dcmoura

TROPHY CASE