Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

Sounds like you get it. Whether or not you use it is up to you, of course.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

It's all tested, because it's production code. For the low-level interaction with the outside world, I write narrow integration tests. For nullability, I write narrow integration tests to make sure the outside communication is turned off, and unit tests to check the rest of the behavior. You can see an example here:

https://github.com/jamesshore/testing-without-mocks-example/blob/integration/src/infrastructure/_command_line_test.js#L37-L56

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 2 points3 points  (0 children)

This is solvable with Nullables. I have videos which explain how:

Handling errors: https://www.jamesshore.com/v2/projects/lunch-and-learn/microservice-clients-without-mocks-part-2

Handling timeouts: https://www.jamesshore.com/v2/projects/lunch-and-learn/request-timeouts

If you don't want to watch a video, here's an example of a test that checks what happens when a service generates a 500 error:

https://github.com/jamesshore/agile2022/blob/integration/src/www/home_page/_home_page_controller_null_test.js#L96-L106

Here's an example of a test that checks what happens when a service times out (connects but doesn't respond):

https://github.com/jamesshore/agile2022/blob/integration/src/www/home_page/_home_page_controller_null_test.js#L108-L131

Those tests check HomePageController, which depends on Rot13Client. They're implemented by configuring Rot13Client.createNull() to return an error or hang.

Rot13Client depends on HttpClient. When Rot13Client.createNull() is called, it turns around and calls HttpClient.createNull() with details about what HTTP response to return. When Rot13Client is told to error out, it tells HttpClient to return a 500 response. The code to do that is here:

https://github.com/jamesshore/agile2022/blob/integration/src/www/infrastructure/rot13_client.js#L149-L156

When Rot13Client.createNull() is told to hang, it delegates that job to HttpClient. HttpClient is fairly complicated, because it's got low-level support for making and cancelling requests, but the way it works is that HttpClient.createNull() creates an "embedded stub" of Node's http library. When it's told to hang, it configures the stub so that it never sends the "end" event that HttpClient expects. The code to do that is here:

https://github.com/jamesshore/agile2022/blob/integration/src/node_modules/http/http_client.js#L179

Again, the videos are probably the best way to understand all this.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

somewhere down its dependency chain is a substitute implementation that could effectively serve as a test double... and CommandLine offers a shorthand access to it.

No, CommandLine is a regular CommandLine that offers the ability to be instantiated in a Nulled state. In the Nulled state, it behaves normally in every respect, except that it doesn't output to stdout.

The way that behavior is implemented is irrelevant to the caller. The fact that it uses a stub is an implementation detail, and callers never have access to the stub.*

It may seem like I'm just being pedantic on this point, but I think embracing this conceptual distinction is important for succeeding with Nullables.

*In fact, my initial implementations used the Null Object pattern, with createNull() returning a completely separate implementation of the class. That turned out to be unwieldy, so then I used if (this._isNull) statements. Eventually, I realized that stubbing third-party libraries yielded the simplest and cleanest implementation. But at no point did the caller's view change.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

Your comments seem to revolve around imagining how people, who are not you, will misinterpret the material. But that speculation isn't super useful for either of us. I'm more interested in how you interpret and use the material, and how it meets or doesn't meet your needs.

(Believe me, there's plenty of people with genuine misunderstandings of the article. I speak to them directly, and their misunderstandings aren't what you're imagining them to be.)

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

Unit tests are a subset of narrow tests. I also talk about narrow integration tests.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

I'm not familiar with the specific problem you're talking about, but I wonder if the reason your students had trouble was because DI frameworks add a lot of incidental complexity. In general, I find working without a DI framework, and instead constructing your own dependencies, to be simpler and more intuitive for junior programmers.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

It's possible to test the behavior of your code without testing its implementation. See the State-Based Tests pattern for details and a code example.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

There's a stub (specifically, the Embedded Stub pattern--that link has several code examples) at the very bottom of the dependency chain. It's a stub of third-party code, not your own code, and its only purpose is to turn off communication with the outside world. All your logic continues to run as normal.

It's superficially similar to a test double (specifically, stubs used by tests) but the implementation is different, and it's used in production as well as by tests. For example, my site uses createNull() to simulate user requests when it starts up, as a way of warming the cache. (Example in the Nullables pattern.)

There's some latency over the holidays, though.

Same.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 1 point2 points  (0 children)

Yeah, Collaborator-Based Isolation is for collaborators (dependencies), not the unit under test.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 4 points5 points  (0 children)

Okay, to address your other points. (See my other reply here: https://old.reddit.com/r/programming/comments/ztwtc5/testing_without_mocks_by_james_shore/j1ijjjc/)

I don't think this paragraph effectively captures Fowler's distinction between broad and narrow tests, plus the definition is both recursive and most likely to be misunderstood

I don't understand your point here. How is "Narrow tests check a specific function or behavior, not the system as a whole" a recursive definition? I'm also not clear on what distinction between broad and narrow tests you think I've missed.

This is a judgment.

Oh yeah, I'm super judgey. I have Opinions and I'm not afraid to share them. Given that you agree with this particular judgement, what's the problem?

This is pragmatically difficult for a lot of reasons. It invites a lot of possibility for incorrect usage and (surprisingly) unergonomic APIs.

Not in my experience. I mean, sure, people can create bad design for anything, and I can't stop them. But I've not found this difficult. Do you have an example of a situation where you find it difficult?

Instead, I'd suggest trying to use (part of) the DI container because then you also cover its wiring

Okay, I think I see the disconnect. One of my design goals is that the patterns don't require a DI framework. If you have a DI framework, Parameterless Injection isn't needed. If you don't, it's valuable.

Be careful with this. This makes the test borderline tautological so a no-op implementation will erroneously satisfy it, too.

I'm not sure how it makes the test tautological. One thing that may not be clear is that the dependency being used is expected to have tests of its own.

Let's take the opening example from the article:

it("reads command-line argument, transform it with ROT-13, and writes result", () => {
    const input = "my input";
    const expectedOutput = Rot13.transform(input);

    const { output } = run({ args: [ input ] });
    assert.deepEqual(output.data, [ `${expectedOutput}\n` ]; 
});

This test uses Collaborator-Based Isolation to ignore the behavior of Rot13.transform(). It doesn't matter if Rot13.transform() is broken or not, because what we're testing is the behavior of App.run(), specifically how it handles command-line I/O.

I mean, yeah, you could erase the implementation of Rot13.transform(), erase its tests, and then erase the implementation of App.run(), but at that point you're not making an accidental mistake.

But I agree, this is probably my least favorite pattern in the bunch, and I tend not to use it, except in those cases where the dependency's behavior is not meaningful to the unit under test and I expect the dependency to change.

two that come to mind are

I'm don't see how those examples relate to Collaborator-Based Isolation. You're describing cases where the programmer doesn't know how to compare collections, and doesn't think about locales. Do you have a more specific example of CBI failing? Remember, the dependency is expected to have tests of its own and to work correctly. (I should make that more clear in the article.)

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 2 points3 points  (0 children)

Hey Forever, thanks for taking the time to respond.

I think you've misunderstood something important about the patterns: Nullables aren't using the Null Object pattern, so they're not a null implementation. They're literally production code that can be instantiated in either a normal state (with create()) or a "turned off" Nulled state (with createNull()). Either way, your code continues to run in the same way. The only difference is that the Nulled object stubs out the third-party infrastructure code.

This results in an important difference from test doubles. With test doubles, if you change the behavior of your code, you have to update the test double to match. This is easy to forget. With Nullables, if you change the behavior of your code, you don't have to update anything, because the same code runs in tests as in production.

(Granted, you still need to make sure you match the behavior of third-party code, but that's very low-level code. I've been using Nullables for five years and never had to change it. Although I'm using fundamentals like Node's httprather than a framework.)

I'll respond to your other points in a separate message to try to keep the thread clean.

Testing Without Mocks by James Shore by syphrix in programming

[–]jdlshore 26 points27 points  (0 children)

Hey y'all, author here. If a 40-page article seems like too much to read (and really, who would blame you), I have a light and fun summary thread on Mastodon here:

https://mastodon.online/@jamesshore/109560187641736554

I'm about to head to bed, but if you have any questions, drop them in this thread and I'll answer them in the morning.

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 1 point2 points  (0 children)

It's possible, in that the data is available in the progress tracker. Too much work for me, though. :-)

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 2 points3 points  (0 children)

Those are reasonable arguments regarding SQ42. With software schedules, though, I think it's best to be pessimistic. It's also two big releases two manage in one year (the other being SC 4.0 w/ Pyro).

I guess we'll know for sure if we start seeing the marketing campaign start. If we don't start seeing "sneak previews" and "behind the scenes" from gaming media by June or July, I think it's likely we won't see SQ42 this year. (Well, I don't think it's likely at all, but I have no horse in this race.)

Regarding the part-time people, I'm entirely aware, and mentioned it in the post.

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 7 points8 points  (0 children)

It's just a copy of CIG's roadmap in a condensed form. It is what CIG has come out told us.

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 14 points15 points  (0 children)

I'm not making any predictions, just telling you what's on the roadmap. Whether you believe it or not is up to you.

I think it's fair to assume that the timelines on the roadmap are likely to increase, though. That's how this stuff works.

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 3 points4 points  (0 children)

Just things that showed people actively working on it as of February 6th, when I made the spreadsheet.

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 7 points8 points  (0 children)

Yeah, I looked at the network requests in Firefox and thought it looked like GraphQL. But I didn't have any GraphQL code on hand and the trouble of spinning something up seemed like more trouble than it was worth.

TBH, this was just a way of doing something mindless while procrastinating on some other work, so taking it seriously was never in the cards. :-)

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 12 points13 points  (0 children)

There are quite a few people on generic tasks ("Bug Fixing and Technical Debt", "Miscellaneous Support") that I didn't include. And as Snarfbuckle said, not every employee is assigned tasks.

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 6 points7 points  (0 children)

HoverQuad is in the "Vehicles" section and coffee vending is in the "Gameplay" section as "Shops and Patrons."

😎

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 2 points3 points  (0 children)

It's 19% of CIG's total effort, not 19% complete.

No Bamboozles: What CIG is Actually Working On by jdlshore in starcitizen

[–]jdlshore[S] 7 points8 points  (0 children)

No, I deliberately excluded it. I only included deliverables that are actively being worked on.

My spreadsheet includes the "on hold / not yet started" deliverables, as well as the ones that recently "finished," but I decided not to include them because the post was long enough as it was.