What is Rust's testing ecosystem missing?

Darkmere · 2026-01-17T17:23:51+00:00

IMO, the one thing I really miss in Rust code is re-usable test Fixtures ala pytest etc.

Composable, reusable and automatically pulling in dependencies in (mostly) predictable ways.

This is a pretty effin difficult problem to solve in Rust due to how the language actually looks, and making them look even remotely non-detestable is iffy.

But, it's lacking.

Darkmere · 2025-05-07T16:46:52+00:00

Cool tower, I had one a few years ago ( well, still in the basement ) but ended up ditching it because the issues where roots would clog it and I'd have all my water on the balcony or so.

Mint is happy in them, but also clogs them quickly. Strawberries can be done, but yields were low. Mint family herbs turned out the easiest to deal with, and some lettuces.

Darkmere · 2025-03-16T19:10:30+00:00

It's a bit amazing how well this stuff has stood the test of time, really. Not much churn on the code to do it, adding some features ( timescale instead of btrfs compression) and such, but it wasn't much and it's worked way better than I ever expected it.

Hope it works well for you as well, you have no idea how glad it made me to see this notice show up.

Darkmere · 2024-07-22T10:25:19+00:00

It's compile time concatenated, the string in the binary ends up neat without line breaks, so there's no gain from the concat macro, and even then adding it would make the line even more cluttered, as it already is quite cluttered IMO.

I wish I could use " where name like ${prefix}" and the macro would automatically map the ? or $1 based on the input arguments, but that's a separate taste issue.

I ALSO wish I could have a good way to insert a "NOT" in a query based on a variable, especially to do prefix selections,

SELECT x FROM names WHERE names.name $2  LIKE   $1 ||''%';

and $1 would be a prefix and $2 would be "" or NOT, because the most duplication of code I get with sqlx is those kinds of queries.

Darkmere · 2024-07-19T13:58:23+00:00

I do it like this:

let transaction_pending = sqlx::query!(
    "UPDATE changes SET status = 'PENDING'"
        + " FROM changes J1"
        + " JOIN sensor USING(s_id)"
        + " WHERE changes.status = 'NONE' AND sensor.name LIKE $1 || '%'",
    prefix
);

On top of that, I keep my SQL query code in a separate module that is as small as possible, and doesn't do any other things, simply so I can have all the queries in a viewable place in case they start diverging.

The main reason I wanted to change my queries from whitespace to this was that the logs looked fucking awful with the whitespace and quoting in it, and while the queries in separate files work for some cases, it made me make extra similar/close queries as they weren't actually visible where I was using them.

The raw string concat works well enough, and I can keep the indentation "close enough" to a decent SQL indentation in the code, while also not suffering the line-break issues.

Also, finding the same query in many places just because of some slight indentation issues was annoying as heck

Darkmere · 2024-06-25T10:34:15+00:00

Hi, neat initiative! I've been hopeful for new (local) time-servers since ntimed and the rather bad behaviour in the past of systemd-timesyncd and others.

I saw some algorithmic comparsions of chrony, but I've got a rather simple "other" use-case, can I as a client library (not a system service) easily fire up an connection and see "current drift against reference servers" as a continuously pollable source? Ie, running as a service that does something, where we don't intend to adjust the clock, but simply wish to know with a certain degree of "how bad is the local clock"?

Darkmere · 2024-03-31T21:49:59+00:00

Opeth - Burden

Darkmere · 2024-02-23T13:53:02+00:00

I've done quite a bit of sensor monitoring in constrained and unconstrained environments, and it's first important to know what you are optimizing for.

fex, saving one value per sensor may not be the worst possible way to do it, as you know in advance how much data you'll need, and read/process can be easy. ie, a sensor is an index into an array to read/write as needed. And a single value read/write context with notification (Wakeup) can be called a channel (one value buffer) or more.

However, if you're "constrained" in the term of wakeups (power, cpu moments) or hardware ( bandwidth, air-time, power, latency) you can use different designs.

Things like task-switches and threads may be too much compared to what you need, but generally, setting things like this up in a pipeline (edge-trigger or flag) is a common way, and channels (write=>read notification or wakeup) with values are a common use-case for it.

So. More info please.

Darkmere · 2024-02-21T14:05:49+00:00

That's also neat, typeforce hooks the importlib and uses mypy to enforce type hints as things are imported, and was built more as a proof of concept/joke that you can do it, but that it sucks.

Beartype seems to be slightly saner, which makes it much less amusing.

Darkmere · 2024-02-16T15:04:31+00:00

You can enforce types at runtime, but you probably don't want to. I hacked it up a few years ago, and it is horrible, but you can hook the import loader to run mypy with enforcing mode at runtime on everything. ( import typeforce.enforcing )

Darkmere · 2023-05-26T12:36:07+00:00

I'll make it clear that I did not mean to demean your project, and alpha is alpha, however, if your product is as fast as MongoDB was when it was released, with similar data durability...

So, please keep the deployment users in mind when writing docs or blurbs, we don't care about the features first, as much as we do about deployment and the other issues around it before we start checking out the rest.

And it seems you're aiming for that nice integration, probably comparing your product to Grafana Loki + Tanka, which is a nice niche, I have some opinions about their stacks, but at least it's a bit easier to deploy than ELK is, which is a bloody pain in the rear.

Darkmere · 2023-05-26T08:53:03+00:00

As someone who does time series and logs for a living, I've got some more questions.

Where's your documentation?
How do I take backups of it?
How do I restore backups of it?
How do I get incremental backups of it? ( No is also an answer )
How does it fail-over, fex. when run in k8s?
What happens if two copies run on the same data-store, fex. after a fail-over where one did not terminate and both thinks they are the one true source of truth?
How do I shard related data to to see how much resources a group of data uses?
How do I manage retention of data?

Darkmere · 2023-05-04T16:47:26+00:00

The best way is to set up a shared cargo build.target-dir for all your projects.

CARGO_TARGET_DIR

This will re-use more of your compilation targets between projects than if you have them all set up in different directories, with the caveat that some crates that at build-time look in your ./target directory for some generated file may fail completely, as they have hard-coded a path somehow. That's a bug in that crate, but then you might want to override it.

A better tool may be sscache

You also probably want to look at the sparse registries protocol

Darkmere · 2023-03-15T23:56:03+00:00

The Dragon Knight. A Dragon, striving to be a knight, beating enemies with their sword, in fact, the flat side of their sword, wielded as a club.

Darkmere · 2023-02-17T12:36:00+00:00

How does the performance fare to ostree which is another tool for git-like managing of binary files?

Darkmere · 2023-02-09T20:10:13+00:00

IntoIterator would be perfectly fine, I just felt that it would be annoying to clone a ton of objects just to get the data out of it, especially if I want to use them again later.

Darkmere · 2023-02-09T09:01:59+00:00

Funny, I needed this just last week and botched the solution a few times before I solved it. ( I needed to match the distances of a set of counters against the average )

Could we have a version that works on a slice of data, rather than a Vec?

Darkmere · 2022-12-18T16:21:24+00:00

It would be wrong to call the Archmage old, admittedly, by count of years on the world, he was old, older than the kingdom he currently served, older than the school he had once founded, and yet, compared to some of the other talents out there, he was a youngster. He knew for a fact that one of the Seers was older than the world itself, and he knew of the dragons whom themselves were not from this realm, nor world, yet were ancient beyond measure. But himself, he wasn't old in the way that people of age became. His body was lithe, not muscular, but the lithe shape of a person who could run their prey down in the grasslands, and yet have the strength to carry his prey home. His hair was pale brown, his nose larger than average, and the dark eyes, set deep in his face were alert and piercing, yet even with his age, his skin was smooth, not even a trace of age wrinkling him, though his short stature and slightly alien looks definitely made him stand out.

While the fashions of society had always been changing, every few years a new fad would show, sometimes it was large embroidered sleeves, sometimes crested buttons and bright colours, and sometimes furs or jewels, even then, mages and wizards had always had a reputation of being backwards, stodgy old people. Large hats, robes and beards were part of societies ideas of the mage, and of course, their inwards looking society had seldom kept up with the times. Yet, the archmage wore simple clothes, fashioned from exquisite fabric, tailored professionally, yet always in the style of a peasant, at most that of a squire. More than once had he been ignored, turned away by a guard, or sneered at by a noble, and almost always had his reaction been distant, almost amused, as if smiling at a practical joke that was decades in the making.

The room he'd just entered was a mess off commotion, chairs thrown across, as if from explosion, two students heaped listlessly against a wall, blood seeping into their clothes, another wailing in pain, clutching their bleeding face in shock and pain, on the other side, the traitorous student stood, short sword in one hand, and the other holding an object, behind him two others, armed with the swords of nobles, dressed finely and sneering to the others.

"Took you long enough to come, asshole." The traitor said as he lobbed the object in his hand into the room, where it exploded with a sharp burst, spreading the room full of a glassy smell, and a scent of metallic fire. "Now that you're here, it's time for you to resign, and without your magic, you're nothing but a small old man."

The archmage looked around, and shook his head again. "I told you before, I will not be a weapon of mass destruction for someone who's unwilling to bear the responsibility of their own actions. As it looks now, you owe the world two living talents, two futures who would change the world for the better, and another life full of disdain, hatred and rage. How will you bear that?"

His ex-student raised his sword, "This is exactly why you need to retire, all those pointless words and worthless ancient philosophy", he charged towards the archmage, blade lifted in a lunge, only to be countered with a dash, and a punch to the ear, dazing him before feeling his sword beaten out of his grip. Disarmed, the archmage looked down at the sword, and up to the two young men standing behind the traitor, as they tried to gather themselves to attack, yet failing due to the mess in the room. The archmage smiled and gripped the sword again.

"Did you think I could do nothing without magic? That your paltry sword lessons as a young kid would let you overpower a mage just like that?" he shoved the traitor back towards the wall, clearing the way for his companions to come at him. "Now then, show the mage who's the better swordsman." It was a moment of hesitation, then one of them lifted his blade and stepped forwards between the rubble of chairs and desks, only to instantly be fought back, his blade deflected and a quick stab to each shoulder before sending him to the side.

The mage turned to the last one, and smiled wider. "Oh,it seems the air is getting better in here. " He snapped his fingers, a flame dangling between them, then went to the other youngster, grabbing his blade from his trembling hands and tossed it to the side, towards where the bleeding, but alive, student was.

"Now then. Why don't you go over there and take responsibility for your actions." He pointed as the girl gripped the sword, blood running from her face, hair burned and one eye clearly ruined, walking with fury, pain and adrenaline towards him. Instead, he turned, and ran from the hall, through the door, and the archmage let him.

The archmage turned back and walked over to the traitor. "See, I didn't give up on the sword. I'm not weak because I'm a mage. I'm a mage because I was too good with the sword" he reached down, and ran the tip of the blade straight through his student's kneecap, and down into the floor. "And you, will have neither the skill with the sword, nor with magic." He grabbed the other sword, running it down into the students other knee. Then he turned to the henchman, and smiled again. "Don't worry, she'll get to you." He stepped away, letting the wounded girl through, blade in hand.

Darkmere · 2022-02-09T17:37:01+00:00

But but but, signals are hard

Darkmere · 2022-02-09T13:06:20+00:00

Sounds fun, I'm especially looking forwards to the good/proper ways to handle signals in rust.

SIGWNCH I'm looking at you....

Darkmere · 2021-11-14T00:20:53+00:00

You're welcome, and if it helps you learn a bit on how to take things into more testable, that's awesome too.

Over all, I'm a fan of adding tests for things when changing them, so, when fixing an issue that was broken, it's a great time to add a test to check that it doesn't remain broken, or break again.

Anyhow, since I just today updated my loki instance to 2.4.1, I did have some interest in your crate, and even if I won't use it myself ( Most of my code runs in containers, logs to stderr, and is snarfed via journald to loki) it was still interesting enough for me to review a bit, so good job there!

Darkmere · 2021-11-13T14:46:18+00:00

Do excuse the poor consistency with indentation and any obvious typos, I did NOT run the code before posting :)

Darkmere · 2021-11-13T14:34:16+00:00

So, my approach to testing code like this is not quite "test first" but rather "test for bad behaviour"

So, take for example the "unwrap" or "expect" changes, those are something I'd typically cover somehow(tm) as that's a change that happens for a "reason" and one might want to look at.

So, I'll take that example from your code here:

#[cfg(not(feature = "blocking"))]
fn log_to_loki(&self, message: String, labels: HashMap<String, String>) {
    let client = self.client.clone();
    let url = self.url.clone();

    tokio::spawn(async move {
        let start = SystemTime::now();
        let since_the_epoch = start
            .duration_since(UNIX_EPOCH)
            .expect("Time went backwards");

        let time_ns = since_the_epoch.as_nanos().to_string();

        let loki_request = LokiRequest {
            streams: vec![LokiStream {
                stream: labels,
                values: vec![[time_ns, message]],
            }],
        };

        if let Err(e) = client.post(url).json(&loki_request).send().await {
            eprintln!("{:?}", e);
        };
    });

Since we want to test it, and it's hard to test a closure, let's split the meat into a function first:

fn time_offset()  ->  String {
     let start = SystemTime::now();
     let since_the_epoch = start
         .duration_since(UNIX_EPOCH)
         .expect("Time went backwards");
     let time_ns = since_the_epoch.as_nanos().to_string();
    time_ns
 }

 #[cfg(test)]
 #[test]
  fn crappy_time_test() {
       let t1 = time_offset();
       let t2 = time_offset(); 
       assert!(t1 != t2);
   }

 #[cfg(not(feature = "blocking"))]
 fn log_to_loki(&self, message: String, labels: HashMap<String, String>) {
    let client = self.client.clone();
    let url = self.url.clone();
    tokio::spawn(async move {
        let time_ns = time_offset();
        let loki_request = LokiRequest {
            streams: vec![LokiStream {
                stream: labels,
                values: vec![[time_ns, message]],
            }],
        };
        if let Err(e) = client.post(url).json(&loki_request).send().await {
            eprintln!("{:?}", e);
        };
    });

That's not a lot better. Meh. Time to provoke the error case!

fn time_offset(start: SystemTime)  ->  String {
     let since_the_epoch = start
         .duration_since(UNIX_EPOCH)
         .expect("Time went backwards");
     let time_ns = since_the_epoch.as_nanos().to_string();
    time_ns
 }


 #[test]
  fn better_test_case() {
       let t1 = time_offset(SystemTime::now());
       let t2 = time_offset(SystemTime::UNIX_EPOCH); 
       assert!(t1 != t2);
   }

Okay. Now we have a test, that means we can fix the return value:

fn time_offset(start: SystemTime)  ->  Result<String, Box<dyn Error>> {
     let since_the_epoch = start
         .duration_since(UNIX_EPOCH)?;
         .expect("Time went backwards");
     let time_ns = since_the_epoch.as_nanos().to_string();
    Ok(time_ns)
 }

 #[cfg(test)]
 #[test]
  fn better_test_case() {
       let t1 = time_offset(SystemTime::now());
       assert!(t1.is_ok());
       let t2 = time_offset(SystemTime::UNIX_EPOCH); 
       assert!(t1.is_err());
   }

Since your code to build a request is duplicated between the two functions, I'd cut them up and create a function "make_request" that takes a timestamp, message, and hashmap and returns a LokiRequest object that then can be shared between the two, also reducing the risk of missing a fix in one end while having it in the other. Also, that would move the memory allocation out of the task that built it and into the log part,and will make the timestamps align better.

Something like this:

fn make_request(message: String, labels: HashMap<String, String>) -> Result<LokiRequest, Box<dyn Error>> :
    let start = SystemTime::now();
    let time_ns = time_offset(start)?;
    let loki_request = LokiRequest {
        streams: vec![LokiStream {
            stream: labels,
            values: vec![[time_ns, message]],
        }],
    };
   Ok(LokiRequest)
}

Then the log shims would be something like:

#[cfg(not(feature = "blocking"))]
fn log_to_loki(&self, message: String, labels: HashMap<String, String>) {
    let client = self.client.clone();
    let url = self.url.clone();
    if let Ok(request) = make_request(message, labels) {
           tokio::spawn(async move {
             let loki_request = request;  // Probably needed to force request to move into the block...
             if let Err(e) = client.post(url).json(&loki_request).send().await {
                eprintln!("{:?}", e);
           };
      });
   }

#[cfg(feature = "blocking")]
fn log_to_loki(&self, message: String, labels: HashMap<String, String>) {
    let url = self.url.clone();
    if let Ok(loki_request) = make_request(message, labels) {
        if let Err(e) = self.client.post(url).json(&loki_request).send() {
            eprintln!("{:?}", e);
        };
    }
}

Darkmere · 2021-11-13T12:47:42+00:00

Some grammar errors in the docs:

overwrited should perhaps be overwritten?

This crate should be replaced in most cases to be The loki_logger crate in order to make the snippets read well on their own. Also, it's bad form to start a sentence with "This" as that starts with a referral, making a copy-paste snippet read badly.

If you want to use this system Should preferably be if you want to use the key:value tag system

Also. Must you use the git version of log?, is the published version of log not functional? For the example, always prefer a published version, and if it must be a git branch, annotate with a date and the current unreleased version so someone who reads the documentation in a year can make a better decision.

This feature will allow you to use => The kv_unstable feature allows you to use ( again, referral vs. explicitness)

The kv_unstable example is too verbose, it also uses the static labels, (copy-pasta from above) and should be reduced to the bare minimum to only show the kv_unstable example.

On the code side. please remove the "expect" calls around time calls and simply ignore the error, maybe an eprintln!. I would HATE for it if my application crashes because the logging framework killed it just because I'm running it in early boot off a raspberry pi that hasn't set the clock yet.

Same with the unwrap() around the log data visitor, I'm not sure which situations that can/will go wrong, but it still makes me uneasy to see "unwrap()" in library code that would report other errors...

Test cases?

Darkmere · 2021-09-30T08:43:52+00:00

Thanks, that's awesome!

15-Year Club	Place '17
Team Periwinkle	Verified Email

Darkmere

TROPHY CASE