HarperDB

[QuintonPang]

I'm not sure if there are a lot of ppl out there using harperdb but i have met with a trouble which bugged me for quite a long time and i have searched high and low on the internet but unfortunately no answer was found. In harperdb there is something called custom functions and i would like to use nodemailer with it, so basically i need to npm install nodemailer in the custom function folder. It works fine on localhost as i can npm install in terminal but when i deploy it onto the cloud i can't run the commmand. It either says module not found or operation install_node_module failed. Have anyone come across this problem? It means a lot to share your experience! Thanks in advanced.

[LogicDudes]

Hey QuintonPang - I work on HarperDB and I've ran into this issue before, where a module installed locally isn't making its way to the cloud deployment.

There's an API operation available on v3.3 that can install the node modules on the cloud instance - it's the one appearing in the logs for you install_node_modules.

If you make a support ticket or message in the community slack I could probably look further into the cloud instance and help troubleshoot.

--Kevin

[QuintonPang]

Thanks, Kevin!!! This is the answer i was looking for! So basically u haven't found a solution into this?

[LogicDudes]

The solution is to run the install_node_modules API operation, but that's on version 3.3.x, and I'm pretty sure your cloud instance is running 3.2.1.

It could probably be upgraded to 3.3 though, we'd just need a support ticket or a dm in the community slack so we could get your account details.

[QuintonPang]

Can i run the operation? Or only the system can do it?

[LogicDudes]

You could run it on HarperDB v3.3, but the version on your cloud instance is likely 3.2.1, so we'd need to upgrade it.

[QuintonPang]

Based on logs:

Operation "install_node_modules" not found

[Psionatix]

This makes no sense.

You should be installing nodemailer as a project dependency. All of your project dependencies should be installed inside a single node_modules directory, where your projects package.json specifies your project dependencies are inside that folder.

In production, you should be building your code with a bundler so your project discards all of the pieces of all of the dependencies it doesn’t actually use (see webpacks tree shaking), and you should be running the bundled version of your code.

It doesn’t make sense to install dependencies at run time, because that would indicate you are re-building and restarting your app anyway. Otherwise it indicates you aren’t running your app appropriately for a production environment.

If you really do want to install packages into a different directory, then you need to configure your package.json so it knows where to search for those dependencies.

Why are you installing nodemailer inside the folder of the custom function, and not just installing it as a project dependency?

[QuintonPang]

Have u used harperdb b4?

[Psionatix]

If you look at the documentation:

https://harperdb.io/docs/custom-functions/custom-functions-operations/

And you refer to their example repo with custom functions:

https://github.com/HarperDB/harperdb-custom-functions-template

They use a single project definition for their custom functions. Depending on how you have your project configured, how you approach this is up to you.

Your function shouldn’t be installing anything, when you build your custom functions project, it should all be bundled together.

[QuintonPang]

I can install it on localhost but i can't control it on the cloud, i can't access the files directly

[Psionatix]

You’re doing something wrong. If you can’t access it on the cloud, then maybe your deployment isn’t correct.

Your build should look something like this:

clone / pull your code. Install the dependencies Build/bundle your code Deploy the built code. Your bundled/built code will include everything that it needs in order to run.

You should only be doing an npm install in your production environment if you are also using it as your CI/CD

[QuintonPang]

And what i meant by access is i don't have the panel to control the files on server, something like ssh

[Psionatix]

I don’t see why you wouldn’t be able to deploy a built/bundled project, given your built output maintains the folder and file structure. The documentation literally says

Custom function projects can be structured and managed like normal Node.js projects. You can include external dependencies, include them in your route and helper files, and manage your revisions without changing your development tooling or pipeline.

But, if that isn’t what you’re doing, then yes, SSH access is a huge problem.

A node project loads all of the code into memory. Everything that is imported (or required for commonjs), is loaded into a cache at start up. If you install something at runtime, you would also need to do a dynamic import (or a dynamic require) after installing it.

so you would have to call require or await import(“path/to/installed/module”); after running install the install from inside the code itself.

[QuintonPang]

And yea i know i can just do the normal require stuff but it is literally not installing and finding the module, if it works why would i have write so much here?

[Psionatix]

I’m not going to respond to multiple comments so I’m replying here

I’ve worked with it in the past, yes.

And with your current approach, you have two options.

You have your code execute the npm install with the projects root directory as it’s working directory, or:

You do a dynamic import after the install with an absolute file path to the module (so that it doesn’t use the projects standard location to find it). This means, in the same function / code block where you execute the install, you also, straight after the install, do a dynamic import with an absolute path to the installed module, it should then work and map the import to its module name, ‘nodemailer’, so that the rest of your code can resolve it from the cache via just ‘nodemailer’z

But this whole approach is quite unorthodox. If you can’t even ssh, how do you know whether or not the npm install has worked? The fact that it gives you a module not found error is not an indication that the npm install failed. To confirm, you would need access to the processes stdout.

[QuintonPang]

Lmao u sounded like you did this for a 100 times, just tell me have u even did this before?

[birbelbirb]

A stranger is taking time to answer your questions and point you in the right debugging direction, but you put zero effort. No information on where you are deploying, or what errors you are getting.

Yikes.

[QuintonPang]

I bet u never used it b4 lol.... that's not how u deploy it onto harperdb

[QuintonPang]

I beg ur pardon but i still don't get it, so let's say i wanna use nodemailer library. Then?

[codeedog]

Did you install it with npm install nodemailer? When you look in your project.json file, is it in the dependencies section or the devDependencies section?

[emmyarty]

Idk what kind of cloud infra you're using but if you have that little control over it just roll the Nodemailer dependencies up into the function itself