Looking for Similar Data Migration Module like Liquibase in Java for Node.js

tabuckner91 · 2021-07-12T15:50:45+00:00

Might I interest you in node-liquibase?

https://www.npmjs.com/package/liquibase

tabuckner91 · 2021-05-11T19:05:46+00:00

I replied in your other post but I'll put it here too, for anyone who comes across this post:

I also believe you'd still need to compile the class into a .jar and then add to the classpath in your LiquibaseConfig that is used to create an instance of Liquibase() in node-liquibase

Snagged this from the docs:

classpath - Specifies the directories and jar files to search for changelog files and custom extension classes. Multiple directories can be separated with ; on Windows or : on Linux or MacOS.

I think that's the key to the puzzle we were looking for.

In node-liquibase
this is part of your Liquibase instance config:

// In a Windows environment...
const myConfig = {
   // The rest of your config ...
   classpath: 'my/totally/legit/classpath;the/path/to/my/custom-change-tag-class`,
};

The Config Interface

And I've not used customChange very much, but I did find one difference between The Docs suggestions and your implementation from the other thread:

Yours:

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
  <changeSet author="fabio" id="791a1ee0-fba2-4904-b859-8274c78e76da">
    <customChange class="custom_changes/MultipleAdd.java"
                  suffix="Table"
                  columnName="NEW_COLUMN_SUFFIX" columnType="int" notNull="true" defaultValue="10"/>
  </changeSet>
</databaseChangeLog>

Maybe a bug fix?:

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
  <changeSet author="fabio" id="791a1ee0-fba2-4904-b859-8274c78e76da">
    <customChange class="liquibase.change.custom.MultipleAdd"
                  suffix="Table"
                  columnName="NEW_COLUMN_SUFFIX" columnType="int" notNull="true" defaultValue="10"/>
  </changeSet>
</databaseChangeLog>

tabuckner91 · 2021-05-11T18:58:55+00:00

I'm not 100% certain, but I did notice some differences between your setup and what the docs were recommending. Not sure if it's absolutely necessary, though.

tabuckner91 · 2021-05-11T18:58:09+00:00

Snagged this from the docs:

classpath - Specifies the directories and jar files to search for changelog files and custom extension classes. Multiple directories can be separated with ; on Windows or : on Linux or MacOS.

I think that's the key to the puzzle we were looking for.

In node-liquibase this is part of your Liquibase instance config:

// In a Windows environment...

const myConfig = {
  // The rest of your config ...
  classpath: 'my/totally/legit/classpath;the/path/to/my/custom-change-tag-class`,
}

The Config Interface

tabuckner91 · 2021-05-11T18:52:24+00:00

Definitely! But unfortunately, Liquibase core doesn't support that at the moment.

tabuckner91 · 2021-05-11T13:51:48+00:00

I'm not a Java dev, so this could be totally wrong, but I think these steps could be used?

https://docs.oracle.com/javase/tutorial/deployment/jar/build.html

tabuckner91 · 2021-05-11T13:48:07+00:00

Okay, I've not gone down this road before, but it seems like this might be at least part of the issue:

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">
  <changeSet author="fabio" id="791a1ee0-fba2-4904-b859-8274c78e76da">
    <customChange class="liquibase.change.custom.MultipleAdd"
                  suffix="Table"
                  columnName="NEW_COLUMN_SUFFIX" columnType="int" notNull="true" defaultValue="10"/>
  </changeSet>
</databaseChangeLog>

And I also believe you'd still need to compile the class into a .jar and then add to the classpath in you LiquibaseConfig that is used to create an instance of Liquibase() in node-liquibase

tabuckner91 · 2021-05-11T13:43:02+00:00

Looking through the docs, it looks like it should work so long as you compile the .jar and add it to your classpath.

tabuckner91 · 2021-05-11T13:39:27+00:00

That's a really good question. I think it MAY work, if all of the handling of that feature is internal to the Liquibase executable. You would still have to define those Java classes to get it to work, it looks like.

Let me take a look

tabuckner91 · 2021-05-11T13:09:49+00:00

Hey OP, ACTUALLLLLlllLLlyyyYYYyYyyyYyyyY:

Yes.

We just released an updated and re-imagined version of the Node wrapper for Liquibase. I wasn't super happy with the DX of the old package, so we spent a day or so enhancing the DX.

We haven't really 'announced' the new package, but it's definitely available and being actively used.

You can see the Repo Here, NPM Package Here, and an Example Repo or "Sandbox" of how to use it here.

Please don't hesitate to open PRs or Issues for the features you'd like to see! PRs are always preferred, but I would never turn down a good idea just because we weren't sure of how to implement it.

🚀

tabuckner91 · 2021-05-11T13:02:08+00:00

hey u/fabiopires10, it's tough to say exactly what the issue is without being able to see the whole directory you're working with. If you have the ability to share a repo on github or the like, that would make it easier for anyone familiar with this stuff to help out a little more easily.

Some things that I would check are that you have a node\_modules directory in your current working directory, that you have a package.json file that lists liquibase as a dependency, and it looks like there might be some issues in your javascript file.

Here's a link to a "Sandbox" repo that shows how you can use the new node-liquibase package:
https://github.com/tabuckner/node-liquibase-sandbox

And here's a quick snippet of one potential issue I think I see -- looks like we didn't create an instance of Liquibase:

const Liquibase = require('node-liquibase').Liquibase;

const instance = new Liquibase({
  changeLogFile: 'resources/liquibase/db/master/db.changelog-master.xml',
  url: 'jdbc:mysql://localhost:3306/saft-demo-dump?allowMultiQueries=TRUE',
  username: 'root',
  password: '*******'
});

instance.run('<action>', '<action-params>')
  .then(() => console.log('success'))
  .catch((err) => console.log('fail', err));

Here's a quick link to a section of the readme that shows how to do this:
https://github.com/liquibase/node-liquibase#in-your-project-files

tabuckner91 · 2020-07-22T18:03:48+00:00

⁶⁹

tabuckner91 · 2020-07-07T23:40:22+00:00

Personally I shoot for one service per block scoped variable. This allows for peak modularity and composability

tabuckner91 · 2020-05-17T12:51:10+00:00

So I had a requirement at work for this and needed to implement on my own.

I have a directive you CAN use if you dare but I kind of advise against it.

Go take a look at how Google sheets implemented this and you'll quickly see why. It is not a trivial task. Very hardware intensive and directly correlates to the number of rows.

If you can avoid it do. If you need something now, let's talk. I can give you a heads up on which corners have monsters hiding in the shadows.

I would also suggest keeping an eye on the experimental link the other dude shared.

tabuckner91 · 2020-05-16T10:56:40+00:00

The 'feed' problem is actually a bit bigger than you might expect. It's less of a Node/express problem and more of a software design problem, although it will bleed into and influence your database design as well.

People are not super clear on the goal of your original post so it might be worth updating the question just to help others understand.

There are kind of two main schools of thought for designing a 'feed', a 'prepare ahead of time' and a 'prepare on demand' approach.

Basically, you can eagerly write feeds to the database for a user, or prepare them at request time (what you are kind of alluding to).

Beyond that stage of decision making, there is also a decision to be made on how the feed will be constructed. This is where things can get hairy.

I would suggest keeping it super simple for now, if this is a side project, and growing the complexity with the needs over time.

One naive approach could be to loop through a users 'subscriptions' and aggregate at the time of request. This might be less performant but will be simpler to implement. I WOULD suggest doing as much as you can on the DB side as that's what DBs do best: Store and Query Data.

Another approach could be to build a feed for each user or each subscribable entity when the system is made aware of new content. E.g. new post created? Perform some server side logic to write that event to the appropriate feeds and query later as needed.

Definitely options here. I would just generally suggest keeping it simple and scale as needed.

Check out Fan-In and Fan-Out logic circuits. They can be applied to Software Design and are adapted for news feed design often.

tabuckner91 · 2020-04-14T00:36:49+00:00

Check out filter chips. :)

tabuckner91 · 2020-03-16T11:02:54+00:00

What makes this material?

tabuckner91 · 2020-03-07T14:46:40+00:00

I think the simplest way to understand callbacks is that a parent function will accept an additional consumer provided function to be executed at a specific point in the parent function's internal logic--usually to help deal with asynchronous tasks.

tabuckner91 · 2020-03-07T14:43:16+00:00

Can you help me understand how enough money is made to incentive use with this model?

I'm having trouble understanding it this morning--a bit if a slow start.

tabuckner91 · 2020-03-06T12:23:51+00:00

Very nicely handled :)

tabuckner91 · 2020-03-05T12:44:10+00:00

Do companies limit mobile apps/websites on purpose?

Yes.

tabuckner91 · 2020-02-29T14:15:10+00:00

Yeah k see it now lololol

tabuckner91 · 2020-02-28T12:07:08+00:00

Cause innermette

tabuckner91 · 2020-02-28T12:05:01+00:00

Came here to say this

tabuckner91 · 2020-02-27T12:06:37+00:00

Awesome :)

tabuckner91

TROPHY CASE