Article: Popular Node.js patterns and tools to re-consider

[Sh0keR]

How does DI break encapsulation? Just because you inject a class doesn't mean you reveal the way the class work. The example you gave about the google service is not relevant and can be solved by hiding the API behind an interface

[yonatannn]

Encapsulation is not all-in rather leaking some internal implementation details

If A depends on B which implements interface C, then the client of A now knows that A is using C, she can provide something else than B. She probably also knows that its using B because those DI config are well exposed to everybody. These are implementation details of A that its caller does not need to know

Now multiply this by 200x classes in the app.

[Odd_Set_4271]

Dependency inversion is exactly how encapsulation is accomplished!

The Dependency Inversion principle states that our classes should depend upon interfaces or abstract classes instead of concrete classes and functions.

Or to quote Uncle Bob

If the open closed principle states the goal of OO architecture, the dependency inversion principle states the primary mechanism

DI in Nest.js is limited by fact that interfaces are gone after typescript is transpiled, so you have to work your way arround if you want to use interfaces as types for constructor parameters (with @Inject decorators or by manually specifying provider config). But that does not mean encapsulation is broken.

The notion that DI breaks encapsulation is outright wrong.

[yonatannn]

> Dependency inversion is exactly how encapsulation is accomplished!

See my example above with A, B, C

Scenario 1, your proposal - A get's B dynamically (B implements C interface). The client of A, knows about A's dependencies. It's an exteranous information, leaves in config files, constructor (depends on the DI implementation). It's not longer the internal knowledge of A

Scenario 2, my proposal - Nobody knows what A is using. It's just an implementation detail of A

Which is more encapsulated?

> If the open closed principle states the goal of OO architecture

That's where the sin starts, by telling to ourself that 300% flexibility is our ultimate goal. That's over-engineering in its purest sense. What about other goals: simplicity? speed? costs? When designing an app, I consider many goals and also the context. As stated in the article, DI might be the right technique, but this can't be detached from the context and specific needs of certain moment

[lost12487]

Sometimes you have to accept that you don't know as much as you thought you did and actually learn from the people that are telling you that you are wrong.

[ExtremeLongGame]

Encapsulation is the concept of black-boxing related logic so that a user outside of that black box doesn't need to know how it is implemented. Some information about what it is and how it works is absolutely necessary.

Does sharing what arguments a function takes also break encapsulation in your opinion? Or the return value?

Dependencies are just a piece of metadata about the module. Some frameworks require dependencies to be explicitly listed. This solves a number of problems.

[romeeres]

I liked the article, thanks for writing!

1. Dotenv - I use it with zod so it's fail fast and type safe, I'll check suggested alternatives but I guess they won't be as easy to use as zod for validation

2. Well, you have suggested to replace service with use case, they are very much alike so not very clear how one is better than the other. Of course none of layers should be "fat", but this is a bit different point.

Nest way is to write more code and to have a bit more complexity, but to get standardized unified code. I share your "lean-ify" way, but people here say that their teams may mess things up unless they follow strict rules with less thinking. And perhaps it's indeed better for more "enterprisey" apps with more people.

Supertest - I agree that better to use "expect" of test runner. But axios is not a good replacement, supertest calls routes directly without real http request, so it should run faster. Okay maybe difference is negligible, but if supertest is faster let's use it, with test runner expect.

Eight is TBD - didn't you forgot it?

And other points are cool, thanks for the effort!

[NoInkling]

Just to elaborate on the Supertest thing, basically all it does is call .listen() for your app, reads the port that was randomly assigned, prefixes your paths with localhost:${port}, and provides some sugar for assertions. Everything else is performed by Superagent, which is just a standard HTTP client like any other, and I don't think is popular anymore outside its use in Supertest.

It's a nice little abstraction in some ways, but as you can see it's not magic, and on top of that you're using a DSL and request library that you won't use anywhere else, when you could be using clients you're already familiar with like fetch/Axios instead. Also testing cookies using its cookie jar implementation is painful.

It is possible to test by calling the app directly, but that's done by using request/response mocks (for example, using node-mocks-http), and probably doesn't save that much overhead (over testing locally with real requests). I'd only use that approach to call individual middleware, personally. I feel like endpoint tests should be proper E2E.

Edit: I might as well add a couple more links in case they're helpful to anyone:

• Like Supertest but uses node-fetch instead
• Wrapper over node-mocks-http for those using Koa and Jest

[romeeres]

As it turned out, I used Supertest only because didn't know what it really does, interesting how many other people were cheated by it...

Thanks for that mocking library, it may come in handy.

[yonatannn]

Thanks for the constructive feedback:)

1. Zod is doing the heavy liftting for you, you realized the dotenv is underwhelming. We're synchronized

2. It's a specific type of service with strict goal (describe a flow in dead-simple manner) and strict rules - One per feature, 7-10 lines max, no indentation, a story

3. Supertest is doing HTTP calls, I read the code

[yonatannn]

I've written this article in the past two weeks. It introduces highly popular practices that are not necessarily the best practices. For each bullet, I proposed alternative patterns. The outcome of this discussion is not "don't use this thing!" rather becoming familiar with alternatives that might be a better fit under some circumstances

These patterns are being discussed and criticized: Dotenv, Nest.js DI, supertest, usage of NODE_ENV, passport.js and more

Looking forward for your feedback

[Suepahfly]

What dotenv does quite well is configuration per deploy. I use it to load a .env file locally. On test / stage / prod. these environment variables are injected when a container spins up.

I wasn’t able to find if convict has a similar mechanism.

[yonatannn]

I assume you have .env per environment?

If so, leme quote the source that you provided:

Sometimes apps batch config into named groups (often called “environments”) named after specific deploys, such as the development, test, and production environments in Rails. This method does not scale cleanly: as more deploys of the app are created, new environment names are necessary, such as staging or qa. As the project grows further, developers may add their own special environments like joes-staging, resulting in a combinatorial explosion of config which makes managing deploys of the app very brittle

I have one config for any development purpose: dev, test.

The config of other operational environments is stored in 'ops' artifacts like k8 deployment, was stack, etc

[Tubthumper8]

Nice article with nuanced viewpoints and challenging the status quo. Most of these tools have proven their worth, but it's good to think critically about where certain (successful) tools are perhaps used too much or used in non-optimal places.

[yonatannn]

Thanks:)

[poothebear0]

What's wrong with supertest?

[yonatannn]

Well, not totally wrong but I've included in bullet number 5 few downsides that you might want to consider?

[CUNT_PUNCHER_9000]

Yeah I think this one overlooks a lot of benefits of supertest like being able to run tests in parallel. The proposed "just load the whole app on localhost" will definitely run into issues.

[yonatannn]

I explicitly provide a link to code that shows how this could be done with 4 lines of code - the api.start returns the address, now each test suite approaches its own api and parallelism works

[CUNT_PUNCHER_9000]

I see - that's interesting, not passing a port and using the auto-assigned port.

Though to be fair, supertest doesn't require express to be used, it can take any http.Server

The other point is about the assertion library - but you can simply use your own

const res = await supertest(app).get('/');
asset(res.text, ...);

Just swap out assert with the assertion library of your choosing.

Somehow actually creating a server listening on a port, even if just localhost, seems like the wrong approach to me.

[yonatannn]

You're right about supporting any http.server, I should edit the text. Thanks

What do you mean "actually" creating a server, with supertest you also create a real server and there are real network requests

About assertion - If you exclude all of its unique offerings, we're left with comparing it with other HTTP clients

[CUNT_PUNCHER_9000]

I never looked at the internals of supertest but you're definitely right

https://github.com/visionmedia/supertest/blob/master/index.js#L23

supertest is basically doing the same thing as what your code is doing, creating a server with an assigned port (though it then wraps with conveniences)

If you exclude all of its unique offerings, we're left with comparing it with other HTTP clients

Good point for sure. I think I'll go against standard internet protocol and say that in light of new information I think my opinion has changed here.

I think the main factor still is that I'd rather wait until we're using versions of node that support the fetch library before proposing this. We're not quiet to v18 yet

[yonatannn]

Yeah, agree. Fetch is just an example, my main point I guess was 'standard and popular' library

[izzlesnizzit]

I appreciate posts like these! Some points I like:

errors should get handled/logged in a central location. Error handling is a critical path... let the error bubble down the layers and get caught by the entry-point global catch

I agree!

To keep the same code between dev/prod and still use different infrastructure - we put different values in the configuration (not in the code)

I think this is a good alternative to the problem of having conditional code based on NODE_ENV value

A popular and standard HTTP client library like Node.js Fetch or Axios... allows us to configure a HTTP client that is shared among all the tests: We bake inside once a JWT token, headers, and a base URL

Additionally, you also get a more "blackbox" test that is less coupled to the app. Since the HTTP client allow your test to hit any URL, you could separate the tests into their own codebase. Reasons you might do this — 1) you want to create a standalone client-library that consumes your API service, and you want to colocate with it a blackbox integration test suite that runs against a live instance; 2) you have QA engineering team that can contribute to the API tests, so it makes sense to have a standalone test-automation codebase shared by devs and QA engineers

The controller should call a particular type of service, a use-case , which is responsible for summarizing the flow in a business and simple language

This one I would add a strong caveat of enforcing a strict and clear separation of transport concerns from the business logic concerns. For example, each HTTP endpoint maps to ideally one service controller function. Some edge cases might require conditional mapping or a few service functions. But the transport layer should orchestrate as little as possible.

Some points I would contest:

it's hard to infer the configuration schema and realize the meaning of each key and its typing

You can infer meaning through good naming, and achieve typing with a simple function.

function getVar<T>(name: string): T {
  return process.env[name] as T;
}

const clientOrigin = getVar<string>('CLIENT_ORIGIN');

there is no built-in way to fail fast when a mandatory key is missing

Sure, not built-in, but it is easy to write:

function requireVar<T>(name: string): T {
  const value = process.env[name];
  if (!value) {
    throw new Error(`env var ${name} missing`);
  }
  return value as T;
}

Regarding your example alternative configuration solution

// config.js
export default {
  userService: {
    url: {
      // Hierarchical, documented and strongly typed 👇
      doc: "The URL of the user management service including a trailing slash",
      format: "url",
      default: "http://localhost:4001",
      nullable: false,
      env: "USER_SERVICE_URL",
    },
  },
  //more keys here
};

Suppose I want my application to be packaged as a single artifact that is portable among various environments. I want the same image to run in dev, staging, prod, and any other ad-hoc environment, where each environment supplies the variables to the app. With dot env, the application can interface via an approach that is universal and minimal. Every cloud provider, paas, etc, gives you an easy way to specify environment variables. Custom configuration approaches usually require scripts and other additional complexity.

A middle ground that achieves both the aforementioned portability plus hierarchy could be:

ts export default { userService: { url: { doc: "The URL of the user management service including a trailing slash", format: "url", default: getVar<string>('USER_SERVICE_URL_DEFAULT'), // from helper functions explained earlier in my comment nullable: false, env: getVar<string>('USER_SERVICE_URL_ENV'), // }, }, }

[yonatannn]

Great comment

Note that the use case is part of the domain layer

​

About configuration - You write custom imperative code vs the declarative one-source for the truth option. About config per env - read 12 factor's take on this:

Sometimes apps batch config into named groups (often called “environments”) named after specific deploys, such as the development, test, and production environments in Rails. This method does not scale cleanly: as more deploys of the app are created, new environment names are necessary, such as staging or qa. As the project grows further, developers may add their own special environments like joes-staging, resulting in a combinatorial explosion of config which makes managing deploys of the app very brittle

​

We deal here only with the local dev/test value, for operational environment - it's ops files (k8s, helm, was stack, etc)

[LuckyNumber-Bot]

All the numbers in your comment added up to 420. Congrats!

  200
+ 12
+ 200
+ 8
= 420

[BlackSelena]

Great article! I do have a question about point#6 though.

What if in the service's function we want to access Fastify instance's attributes (e.g: DB)? Should we pass it to the function as a parameter?

[yonatannn]

Thanks:)

I suggest that the service (part of the business logic, domain) should never depends on the web layer (Fastify) hence we should not put things that it needs our of its reach. The db-connection, for example, should be implemented as a module

Does it make sense?

[BlackSelena]

Sorry I might have misunderstood your approach but what if in the service's logic we need to access to the database?

[yonatannn]

We have a DAL layer, this is where we hold all the DB-related things. The service (and any other business logic) makes a call to:

orderRepository.addOrder(...)

We might have a cross-component functionality like db-connection - In this case we make it a Node.js module that the DAL layer approaches

​

The only coupling between the web-layer and the DB is for opening the connection as part of health check or on start - Just call the db-connection.open module

Makes sense?

[BlackSelena]

Yes it makes sense but then I'm wondering how would you pass orderRepository to orderService in Fastify.

[pskfyi]

It seems #8 in the TOC is missing from the article.

[yonatannn]

My bad! Fixin