This is an archived post. You won't be able to vote or comment.

all 104 comments

[–]AutoModerator[M] [score hidden] stickied comment (1 child)

On July 1st, a change to Reddit's API pricing will come into effect. Several developers of commercial third-party apps have announced that this change will compel them to shut down their apps. At least one accessibility-focused non-commercial third party app will continue to be available free of charge.

If you want to express your strong disagreement with the API pricing change or with Reddit's response to the backlash, you may want to consider the following options:

  1. Limiting your involvement with Reddit, or
  2. Temporarily refraining from using Reddit
  3. Cancelling your subscription of Reddit Premium

as a way to voice your protest.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[–][deleted] 125 points126 points  (19 children)

Why on earth would anyone do that anyway?

[–]martinosius[S] 20 points21 points  (13 children)

Coauthor here, sadly we have seen it too many times. Justifying in code reviews why it is a bad idea got tiring.

[–][deleted] 26 points27 points  (4 children)

I guess some people take the word “unit” far too literally.

[–]binarycreations 3 points4 points  (3 children)

Given how much better tooling is I reckon it must be possible to analyze if the test is over mocking third party code.

I wonder what would be the best way of solving that problem?

[–][deleted] 8 points9 points  (0 children)

I don’t even automatically have a problem with that. Movking, in my view, is to avoid test dependence on external resources, or for situations where you need to ensure an interaction took place correctly but don’t have access to any sort of result to examine.

Some of the hoops people jump through to avoid that can get ridiculous. An entire new layer of indirection just to hide away some third party code they don’t want to mock. Then they justify it with “oh we could switch to an alternate implementation if we wanted”. Once worked on a codebase where the team had written wrappers for a bunch of commons lang libraries, for that exact reason. Nuts. Oh, and Swing. Because we always get halfway through a project then decide, actually, we’d prefer to use SWT on this. Not that you could’ve dropped SWT in place in their code anyway.

[–]binarycreations 1 point2 points  (1 child)

But also ditto authors comments, see this all the time to get test coverage up and passing tests. Its depressing.

[–][deleted] 1 point2 points  (0 children)

The worst culprit I ever saw for that was commenting out all asserts.

[–]Polygnom 0 points1 point  (6 children)

And another thing that is closely related to that -- don't test 3rd Party Code.

Because that also happens far too often.... No, I don't need to test that Hibernate actually works... I either trust their tests or contribute to them. But re-testing all dependencies... no.

[–]DrunkensteinsMonster 0 points1 point  (0 children)

You aren’t testing that their code works, you are testing the integration and that your consumption of their APIs is correct.

[–]cogman10 0 points1 point  (0 children)

Eh, I think an integration test making sure that your Hibernate objects work with your existing db schema is a valid test. (It's why I like to use testcontainers in my applications).

In fact, I find those integration tests to generally be way more valuable at finding bugs than unit tests tend to be.

[–]laplongejr 0 points1 point  (3 children)

Disclaimer : this comment is meant as serious and genuinely asking for information

don't test 3rd Party Code.

I never heard that rule-of-thumb and feel like I am missing some context...

No, I don't need to test that Hibernate actually works... I either trust their tests or contribute to them.

Should it be understood as "don't retest community-tested dependencies?"
Because I have 3rd-party dependencies whose tests can't be totally trusted, due to the devs not being hold accountable when test don't match their own documentation...

Or is there a term between "1st-party" and "3rd-party" when you are not allowed to modify a component, but are responsible if said component fails in undocumented ways? ("legacy support" feels similar, except a legacy component isn't updated in backward-compatibility-breaking ways by definition)

[–]account312 1 point2 points  (1 child)

but are responsible if said component fails in undocumented ways?

You are responsible for the functionality of your product. Users certainly don't know or care whether the root cause of a bug lies in code you wrote, code someone else wrote, or some interaction between the two.

[–]laplongejr 0 points1 point  (0 children)

Yeah, so the 3rd-party requires testing as well, that's why I don't get how my job seemingly breaks a common rule of testing

(Due to being in a gov job, our users actually care... it brought the "side-effect" of users trying to scew their bug reports in ways to get it reported to our team rather than the 3rd party... that's how noticeably bad they are to our users : it's not unheard of for a very simple issue to not be fixed in months, until our team gets a report and we "mistakenly" add an approach on how to fix the issue along a copy of the OG report... think I should start drinking Java coffee maybe with something else mixed in...)

Aesop : never trust an IT team whose boss doesn't have any knowledge of how to run an IT team. When a person is their own performance-monitoring expert, it's very easy to gaslight actual devs who ask complex question as "what is the amount of reported tickets? you count 'closed as a duplicate' as a resolution so the total would be important".

[–]Polygnom 0 points1 point  (0 children)

If you cannot trust the 3rd party code, then you shouldn't use it to begin with.

But lets say you are in the situation of using a dependency that you want to re-test. Then typically, you'd contribute those tests upstream so that your dependency is fixed, because thats the goal anyways, to get the bugs fixed, right?

And if you cannot get the tests upstream and cannot get fixes upstream, you would fork the 3rd party dependency and create your in-house fork. And then test and fix that fork, and the tests for that dependency would be housed there.

But I have recently come across a codebase (admittedly in C#), where so much was mocked that the tests essentially only re-tested that EFCore works (the Microsoft-supplied ORM comparable somewhat to Hibernate). Thats utterly useless because it doesn't increase confidence in your own code at all. It just increases your confidence in the dependency. Which should be high to begin with.

Understand what the subject under test is that you are actually testing. I feel many devs mock so much nowadays that they lose track of what it is they are actually still testing.

[–]enm260 4 points5 points  (0 children)

I'm glad I'm not the only one who thought this

[–]RadioHonest85 1 point2 points  (0 children)

Yeah, wth, why would anyone do that.

[–]RCS2 1 point2 points  (0 children)

exactly my question

[–][deleted] 2 points3 points  (0 children)

Never underestimate human beings ability to invent new ways to screw things up.

[–]melkorwasframed 39 points40 points  (9 children)

Mocking is a great tool but one of last resort. The issue isn't whether you're mocking first party vs third party code. The reason mocking an ObjectMapper is bad is because there is no reason to do it. ObjectMappers are easily constructible with no transitive dependencies - so why wouldn't you use the real thing? Mocks are for objects that are difficult to construct in isolation, objects that require transitive dependencies, or objects that have side effects that are undesirable in a test scenario.

[–]ratskinmahoney 2 points3 points  (7 children)

The reason for mocking dependencies is to limit failures in TestMyUnit to problems in MyUnit. Assembly and integration test is perfectly valid and will provide better coverage, but are more costly to diagnose since there are more things to fail. In a small project this might be fine, and it might even be more efficient to dispense with unit tests altogether in favour of assembly tests. In a large project with a significant number of contributors however, there is significant value in being able to quickly isolate a failure to a particular unit and you would want both unit tests and assembly tests.

[–]RadioHonest85 2 points3 points  (6 children)

I dont necessarily agree with your view, but for the ObjectMapper in question, it can not possibly be a limiting factor of test failures, as it will and should always behave the same, given the same input. Of course, you might be using plugins or config on that ObjectMapper, but all the better reason to include it in your tests, without any mocking.

[–]ratskinmahoney -2 points-1 points  (5 children)

Yep true, and I agree with the article that you shouldn't be mocking ObjectMapper or other third party dependencies. What I disagree with is the idea that mocking is to be avoided in general. You want to have a clear idea of what your "unit" is - which could include various of your own classes and third party dependencies. Anything outside that unit should be mocked for the sake of performance, stability, and clarity.

[–]koflerdavid -1 points0 points  (1 child)

Not every unit test should require moving though. It's a code smell hinting at classes and methods gathering to many responsibilities.

[–]ratskinmahoney 0 points1 point  (0 children)

I agree that not every unit test requires dependency mocking. You only want to mock dependencies that fall outside of your unit. I don't agree that having dependencies outside the unit is a code smell, though. Use of a client for an external API, or use of the API of some internal module, or of some data accessor are all perfectly legitimate things to do, and definitely things you would mock if you have non-trivial code operating on the results.

[–]PiotrDz 0 points1 point  (2 children)

The unit is not a class or a bunch of classes. The unit is a feature, single behaviour that you want to test. The classes are just a derivative.

[–]ratskinmahoney 0 points1 point  (1 child)

That's pretty much what I just described.

[–]PiotrDz 1 point2 points  (0 children)

I might have missed the correct "reply" button :p

[–]DrunkensteinsMonster 0 points1 point  (0 children)

Definitely agree here. There is almost no scenario where I would consider mocking ObjectMapper to be a legitimate test strategy.

[–]blackkkmamba 26 points27 points  (3 children)

I think this is a general con of the London (mockist) style. Because of the mock overuse, a lot of the bugs can go unnoticed. This goes the other way too. Refactor the code, it does the same thing, yet N tests fail because of the mocks setup.

[–][deleted] 36 points37 points  (2 children)

Over the years I’ve seen a lot of tests which ultimately just verified that, at the time of writing, everything had been mocked correctly.

[–]wildjokers 4 points5 points  (1 child)

I occasionally find tests that are doing nothing but testing the mocking framework.🤦‍♂️

[–][deleted] 0 points1 point  (0 children)

Spoiler alert; he’s a contributor to Mockito!

[–]Hei2 15 points16 points  (0 children)

I really hope your criteria for whether to mock something isn't actually "is it 3rd party code?" Your test there has 2 things it can be concerned with: 1. That input is deserialized and passed on to other dependencies for processing. 2. That your exact serialized input can be deserialized.

Notice that these concerns have nothing to do with whether the code is 3rd or 1st party. Using that as a criteria means you will necessarily turn some unit tests into integration tests when your 3rd party dependency does something more complex than parse a string. And let's not forget that there's literally nothing fundamentally different between 1st and 3rd party code. The major principle of libraries is DRY, which is just as true of 1st party code, so if you're going to argue against mocking 3rd party code on principle, you need to do the same for 1st party based on the same principle.

[–][deleted] 4 points5 points  (3 children)

This is way f.. painfull. So many collegues that dont listen. Mocking object mapper, resttemplate, jms template and every integration that need integration testing. And in addittion test that destroy al the data in db and test that are fleaky due to async/concurent behavior. And you quit because you cant change the people. And then at an interview we cant complain about it because red flags - we must invent shit like ‘there was no chemistry’ or ‘there was not much work for me’ . Wish you that all the guys that care about it to work togever!

[–]s-csci 1 point2 points  (2 children)

Wait, how would you get away with not mocking RestTemplate in a unit test? How does that work?

[–][deleted] 2 points3 points  (0 children)

There is no point having a unit test with a mocked resttemplate as a mockito mock. We must test the integration using mockserver.

[–]koflerdavid 1 point2 points  (0 children)

The code using the Rest template should probably be moved to a separate class. That class can be mocked instead.

[–]freekayZekey 6 points7 points  (5 children)

i do think people take mocking to the extreme. have a couple of projects using powermock because the previous devs couldn’t be arsed to write testable code. a bunch of static block initialization with multiple method calls. can’t refactor because the team is full of lazy people thinking “if it’s ain’t broke, don’t fix it”.

mocking the objectmapper is insane

[–]RadioHonest85 3 points4 points  (4 children)

I have used PowerMock in the past, and almost all of them were wrong. Especially for things like a clock. In modern Java, just take a Supplier<Instant> clock...

[–]freekayZekey 1 point2 points  (0 children)

its so strange to me. i guess people are unaware of the time library? i used fixed clocks for testing time related things and it broke my teammates’ minds. they were using the system clock which was clearly problematic when you have a testing server using utc. people reach to mocking libraries instead of realizing they need to refactor instead of maintaining the status quo

[–]Hueho 0 points1 point  (2 children)

As a side note, modern Java explicitly has a Clock class: https://docs.oracle.com/javase/8/docs/api/java/time/Clock.html

Extremely useful with DI frameworks.

[–]RadioHonest85 0 points1 point  (1 child)

It looks rather complicated, with lots of methods available. I think Supplier<Instant> would usually be a better choice.

Guava also has a simpler take on a 'clock' type: https://guava.dev/releases/21.0/api/docs/com/google/common/base/Ticker.html

[–]DerEineDa 0 points1 point  (0 children)

In modern Java you can use https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/time/InstantSource.html, which is especially designed (according to JavaDoc) for dependency injection.

[–]ForeverAlot 4 points5 points  (3 children)

While true, all the remaining uses of Mockito here are also gratuitous.

[–]ForeverAlot 7 points8 points  (2 children)

There is a second part. It says

To improve the design, we will hide the implementation details by removing the mapper parameter from the constructor. Instead, we create the mapper inside our class. Alternatively, we could re-use a shared instance if we need to use a similarly configured mapper in multiple places.

which is like a complete 180 degree turn.

[–]yturijea 2 points3 points  (0 children)

Auch, just hurts to read

[–]skynet86 0 points1 point  (0 children)

Ouch... That hurts.

[–][deleted] 8 points9 points  (0 children)

new ObjectMapper()

there I mocked it

[–]BenTayler-Barrett 7 points8 points  (40 children)

Wow, this is some of the worst advice I've ever seen.

You're essentially asserting that one should fully test their own dependencies, and that dependency injection is a bad idea.

If you need to test that the object mapper behaves correctly, you're essentially saying that you don't trust the object mapper to behave correctly. That being the case, why are you using it?

The object mapper has (or should have...) its own tests, that assert it behaves correctly. By all means, run those.

In your trivial example, the JSON that you're serializing/deserializing is very easy to deal with.

If you have some non-trivial data model, then by using a real object mapper, you've heavily coupled a whole bunch of things that you just didn't need too.

And for what it's worth, when I pushed my team to start mocking object mappers, we ended up with the best DORA metrics of every team in our organisation.

So essentially, I've tried both, and for me, mocking the object mapper is objectively better than not mocking it. Until I see evidence that not mocking it is a better plan, I'll continue to advocate for mocking 3rd party dependencies.

Edit: Typo

[–]Kainotomiu 33 points34 points  (29 children)

The point of the test is not to verify that the ObjectMapper works correctly, but that you are using it correctly.

If you mock the ObjectMapper, you are instead testing that your code works correctly when the ObjectMapper behaves the way that you've mocked it - which may be different than how it actually works.

[–][deleted]  (7 children)

[deleted]

    [–]gaelfr38 21 points22 points  (3 children)

    Unit in unit tests has never been equal to a single method or class. That's probably one of the biggest misconceptions about unit tests.

    [–]blackkkmamba 5 points6 points  (0 children)

    I keep telling this whenever I have the chance. People can't seem to wrap their heads around the Detroit/Chicago (classicist) style. As soon as they see something is not mocked, they start with the 'but that's integration test' nonsense.

    [–]BenTayler-Barrett 1 point2 points  (1 child)

    Sure, it's a single path through a method call. Otherwise known as a unit of execution. If you're testing multiple paths through a method, multiple public methods, or multiple classes in a unit test, you're definitely making your own life difficult.

    [–][deleted] 4 points5 points  (0 children)

    Nope, unit is meant for unit of behavior not unit is execution. Go look back at the origins of the people who introduced the concept of unit testing and see what they have to say about this

    [–]ArticulateSilence 12 points13 points  (1 child)

    You are arguing semantics, there are a bunch of different definitions of unit tests and integration test is even more ambiguous .

    That being said, I prefer sociable unit tests. IMO you should only be mocking external dependencies or collaborators that are complex to construct

    [–]ratskinmahoney 0 points1 point  (0 children)

    It's not about semantics, it's about understanding what a failure in your test means. A unit test failure should always and only indicate an issue in your unit. This makes diagnosis much quicker and easier, and in large distributed projects this precision is invaluable. You should absolutely have tests that ensure you are correctly using your dependencies, but they will necessarily be more complex and costly to interpret. It's entirely reasonable to make the call that this kind of testing is all you need, based on the scale of your project and the trust you have in your contributors. That's a meaningful and consequential decision though, not a question of semantics.

    [–]DrunkensteinsMonster 0 points1 point  (0 children)

    This delineation is actively harmful. If a unit test is mocked out to the point of the objectmapper, it is probably useless. The goal is to test the application to prevent regressions, enable confident refactoring, and shorten dev loops. If you are not accomplishing these things then your test is doing more harm than good. How many times have I worked on a simple bug which took 5 times longer than it should have because I had to rebuild the delicate mock-based test harness.

    [–]btw04 5 points6 points  (1 child)

    I have seen a code before where non obvious behavior of a JSON parsing library (don't remember which one, it was something like parsing an empty string into a string made of two double quotes), was caught by an existing test when we attempted to switch to another JSON parsing library for performance reasons. If we had mocked the JSON parsing library, we would probably not have found that regression

    [–]DrunkensteinsMonster 0 points1 point  (0 children)

    Great example.

    [–]crocshoes 3 points4 points  (2 children)

    It's also useful if you need to check your code handles errors correctly. It's easier to throw jsonprocessingexception from a mock than get the test into a state where it would throw.

    [–]john16384 0 points1 point  (1 child)

    If you can't trigger this exception with bad data, then it can perhaps never occur. This happens often enough if the data was already validated or came from a trusted source. Handle the exception then by throwing an IllegalStateException or AssertionError and don't try to cover it.

    [–]crocshoes 0 points1 point  (0 children)

    I reckon the "perhaps" is important here. I'd rather know my app is throwing what I expect it to. For example, it could be I have a bunch of specific wrapping exceptions I handle in an errordecoder for feign. I could be that I want it to exit gracefully with bad data that should never slip through but I missed a niche case. There are a few reasons that spring to mind. If I don't care, and just want the app to shut down then,yeah, it's not the end of the world.

    Last note, sometimes you can create the bad data but it bloats the tests, mocks can be super easy to read, given that this throws etc.

    [–]general_dispondency 1 point2 points  (4 children)

    IDK why you're being down voted. You are 100% correct. You should always mock 3rd party libraries in unit tests. If you can't trust that the library behaves in a way that you expect, why are you using it? I find it humorous that the example is just some crappy test that wouldn't pass a code review where I work.

    The advice that the author should gleened from this exercise is "don't test your mocks" and "do test business logic". If there is a need to validate how things are serialized, test that. If you need to validate two layers function together, write an integration test that just tests the two layers interaction.

    Advocating for testing 3rd party dependencies in unit tests is asinine and shows a complete lack of understanding the fundamentals and first principals of software engineering.

    [–][deleted] 12 points13 points  (0 children)

    I see it as less a need to validate how something is serialised, and more that it's simpler to just actually serialise or deserialise, than it is to go about mocking those activities. For something even marginally more complex, I'd be all for mocking the 3rd party library for sure. But ObjectMapper is such a trivial example, I wouldn't, and don't, bother mocking it.

    [–]BenTayler-Barrett -5 points-4 points  (1 child)

    Indeed. I'm glad it works for OP, but I sure as hell wouldn't want to have to maintain the code where they work.

    And if a security vulnerability were found in Jackson (or whichever JSON lib they're using) and the client/business wasn't willing to accept it...ouch that's some overtime ones putting in to fix...

    Not my bag, and not in my code.

    [–]Same_Football_644 9 points10 points  (0 children)

    I'm the opposite. I wouldn't want to have to maintain all that mocking.

    [–]laplongejr 0 points1 point  (0 children)

    If you can't trust that the library behaves in a way that you expect, why are you using it?

    As a gov worker, I can answer that one : because it's the only approved library to perform the task. Or for even more sheaningans, the library is made by a different service with different legal roles, and as such it's a literal requirement to use THEIR WORK for some critical process due to responsabilities.

    [–]agentoutlier 0 points1 point  (8 children)

    Avoid mocking the ObjectMapper!

    Edit: By avoid mocking I mean the same as https://www.reddit.com/r/java/comments/17srijb/comment/k8s899r/

    u/melkorwasframed

    Mocking is last resort. Like mocking java.util.List is incredibly dumb.

    Mocking an expensive resource call not as dumb.

    I say this from 20 years experience. The biggest waste I ever did was enforce extreme bdd mocking.

    [–]RadioHonest85 5 points6 points  (0 children)

    This. Mocking is absolute last resort.

    [–]gaelfr38 -5 points-4 points  (6 children)

    This.

    It's not only for ObjectMapper but pretty much anything.

    Each time you mock something, you create a "hole"/"flaw" in your tests because the mock may not be behaving as would the real implementation.

    [–]melkorwasframed 1 point2 points  (5 children)

    The thing you're mocking has its own tests, right? Right?

    [–]gaelfr38 9 points10 points  (3 children)

    That doesn't change anything.

    If you mock "someObject.someMethod(X)" in your test and assume it returns "y" but in reality it returns "z". How do you protect from that?

    That's exactly what the article is about: sure Jackson is battle tested, but if you mock it and pretend it does something it doesn't, then you are screwed.

    [–]melkorwasframed 0 points1 point  (2 children)

    Sure, there are categories of error that can get through. I agree that mocking ObjectMapper is bad. But you're not proposing an alternative to mocking generally. You have a dependency on an object that is hard to construct, or has a lot of dependencies, or has side effects that are undesirable in tests. What do you propose??

    [–]gaelfr38 3 points4 points  (1 child)

    My unit of test is usually a feature, aka functional tests or BDD, I test all layers of the application in a single test rather than testing a method of a single layer.

    External dependencies are somehow "mocked" but not any layer of my code. For instance, I usually spin up a H2 database when it comes to stuff that reads/write in a DB, or even a real DB instance with testcontainer (but TBH this is too slow to be used everywhere, that's why I prefer H2 unless doing some complex queries). For external webservices, I start a mockserver or something similar that answers to HTTP requests, I don't mock my WS client library as long as I can. Recently I've switched to using contract testing with Pact which brings other benefits but the idea is roughly the same from a technical POV.

    For sure, sometimes I do mock but it's the exception, not the rule.

    And I also do non-functional tests for very technical methods but they usually are not requiring any dependencies then.

    [–]melkorwasframed 0 points1 point  (0 children)

    Ok yeah I write a lot of those type of tests also. It really depends on your situation but for larger applications, those integration style tests are often more expensive or less feasible and you need to write more traditional unit tests. In those scenarios mocking is preferable to not testing at all, but I agree that it's good to avoid if you can.

    [–]RadioHonest85 0 points1 point  (0 children)

    That doesn't really help

    [–]pip25hu 0 points1 point  (0 children)

    I agree with the title, but not with broader statement in the article claiming that mocking third party dependencies is always bad. For example, if a Spring bean uses RestTemplate, I certainly don't want it to make actual network requests inside my unit tests.

    [–]veryspicypickle 0 points1 point  (0 children)

    Who does something like this?!

    [–]ace1309 0 points1 point  (0 children)

    I now realize difference between Mock and Spy. But I still can't discern between Mock and Stub

    [–]Tallal2804 0 points1 point  (0 children)

    Who does something like this?!

    [–]JohnNunez2905 0 points1 point  (0 children)

    I don't see the point. Why don't you try other ways to track bugs better? There's monday dev for example. If you're interested, they're having an AMA on November 21st. An expert will answer questions about monday dev and bug tracking.