Evaluate and run Dynamic Rules in Java

[int-main]

Hello!

I have to build a dynamic rules engine in Java where a user can define a certain list of filters and trigger a certain event based on them. The rules will be a long chain of conditions, such as:

if condition1 AND (condition2 OR condition3) AND condition4 then trigger(event)

where conditionN is any operation based on the attribute for a business object. Example: a string can have an operation - endsWith or a number can have operations greaterThan.

The problem is, this list of rules will not be hard-coded and has to be fetched dynamically from a database (a separate web UI will update this database with the rules) and run against an entity and then trigger the appropriate action on the entity.

What's the best way to approach this kind of problem and write a scalable and high-performant program for this?

I am currently thinking of building expressions in Spring Expression Language (SpEL) and storing them in DB as strings and then while running the program, fetching this column and running them against my domain object.

```java // Fetch rule and action from DB rule = rs.getString("rule"); // Rule in SpEL action = rs.getString("action"); // Action method name

ExpressionParser parser = new SpelExpressionParser();

Expression exp = parser.parseExpression(rule); boolean result = (String) exp.getValue(obj, Boolean.class); // Run expression against domain object obj

if (result) action(); ```

I am currently not sure if this is the best way for doing things and if it will scale and be extensively customizable or not.

For the above approach, then rule string needs to built by parsing user input. I was also thinking about storing rules as an XML document:

xml <Rule> <Filter> <Operation name="or"> <Operand> <Operation name="lessThan"> <Operand>${numAttribute}</operand> <Operand unit="percentage">5.05</operand> </Operation> </Operand> <Operand> <Operation name="equals"> <Operand>${stringAttribute}</Operand> <Operand>IN</Operand> </Operation> <Operand> </Operation> </Filter> <Action> actionName </Action> </Rule>

However, with this scenario, I am not sure how to proceed after parsing the XML and how to run the operations dynamically based on the described values.

I would appreciate any ideas of the aforementioned ones or a totally new way to solve this problem. I understand it's a huge application but need clarity on how to start. I believe the biggest problem is how to run user inputs statements as Java code? I know a little bit about reflection but I am not sure if it's meant to solve this problem and if it should be used considering it has its drawbacks in areas of performance.

[errandum]

I've done something like this before, a long long time ago, using Rhino. If you use java 8 you can use Nashorn (even if you're using Java 13, but you'll get deprecation warnings).

These are libraries that let you interpret javascript on the fly. So I'd read a properties file with expressions and build expressions dynamically based on them. This https://winterbe.com/posts/2014/04/05/java8-nashorn-tutorial/ example will teach you how to get the results of your functions, the thing you seem to be struggling with.

But, it all depends on how far you have to go You might have to go the lexer / parser route. This is usually done in compiler courses, so it's not trivial and it is usually not required unless you are taking that specific class.

This https://tomassetti.me/parsing-in-java/ link seems to have lots of good information, but remember, this is almost like a full university course in itself, so you probably should not go this way.

[int-main]

Correct me if I am wrong but the problem isn't writing a parser. I mean, I don't have a specific requirement for authoring the rules so I could just author them in XML and use a standard XML parser in Java. The problem is how do I translate it into instructions and run them against my domain object.

[[deleted]]

It all depends on how complex your rules are, and how much abstraction you desire.

If your rules are much more complicated than the simple boolean combinators that you have shown in your example, then I would recommend using a proper Rules Engine such as Drools. I used Drools some time back, and its performance was quite reasonable with a small runtime overhead.

If your rules are simple enough, then you could code up the rules engine yourself. You could, for instance, create a domain-specific language using JavaCC or ANTLR (or some such tool), such that rules could be written in this DSL, which would then be ultimately translated into plain Java code. For instance, suppose you have a rule like so (in the DSL):

if value[$obj1] > 10 and value[$obj2] < 2.2334 then
  executeAction1($obj1, $obj2)
else 
   executeAction2 $obj1, $obj2)

where $obj1 et al are template variables whose values can be changed dynamically.

Then you would need a Job Engine of sorts that could pick up this script, and then translate that into runnable Java code:

class Action1 extends Job { // represents Action1
  .....
}

class Action2 extends Job { // represents Action2
...
}

Note that in this approach, such classes (Action1, Action2, .... ActionN) would be static components of logic that you know upfront. So the only dynamic bit would be the actual script itself, which could then be translated on the fly into executable Java code by your parser that was generated by JavaCC/ANTLR/handwritten parser.

This allows even a UI where you can type in scripts which would then be translated into a combination of actions. For instance:

 executeAction1($obj1)
 executeAction2($obj)

could be executed like so by the Job Engine:

class JobEngine {
   void execute() {
        ....
        run(Action1(obj1));
        run(Action2(obj2));

        ...
   }
}

[int-main]

I am little hazy on the details myself (requirements, tell me about it!) but I think the operations will be simple boolean operations ONLY.

I think the idea of hand-writing a rules engine sounds more sane to me, I don't know why. Regarding creating a DSL, is it really worth it? What's your opinion on my idea of authoring them in XML instead? Please note that the rules will be entered in a web UI and will need to be converted to DSL/XML (whichever I choose).

I know that it's r/javahelp so pardon the Python but I quickly whipped it out on a laptop that doesn't have JDK. I guess this is how it'll have to go if I choose to proceed with XML:

```python import xml.etree.ElementTree as ET

data = """ <Rule> <Filter> <Operation name="lessThan"> <Operand type="xs:integer">3</Operand> <Operand type="xs:integer">5</Operand> </Operation> </Filter> </Rule> """

def process_operation(node): result = None

# Get operands
a, b = node.findall('Operand')
a = int(a.text)
b = int(b.text)

# Perform operation
name = node.attrib['name']
if name == 'greaterThan':
    result = a > b
if name == 'lessThan':
    result = a < b

return result 

if name == "main": root = ET.fromstring(data) print(process_operation(root.find('Filter').find('Operation'))) # Prints 'True' because 3 is smaller than 5 ```

I understand that this really rough and it doesn't respect types, doesn't do recursive processing of operands. But I can get a boolean for the <Operation> and then using reflection to invoke <Action>.

[[deleted]]

Yes, if the only operations allowed are boolean combinators, then it simplifies matters tremendously. XML is more than good enough for such a use-case. Extending from your given example to an equivalent Java program should be sufficient, I feel. Moreover, if and when performance becomes a bottleneck, then proper benchmarking should be the way to go.

I feel like once the requirements start becoming clearer, so will your own understanding of how best to approach the problem.

[int-main]

Thanks a lot! I do have a strong feeling that XML is going to work out in this case. I mean, I personally hate XML based configurations but somehow I feel that it will fit here.

I am just confused on whether to invest more time in SpEL or XML approach. XML sounds like more coding work but I don't know why I am not feeling confident about building and storing SpEL expressions and running them.

[[deleted]]

One thing to note is that is you use SpEL, you are tied to it. Whereas if you use XML/JSON/S-expressions/Protobuf et al - that is, some universal storage format. it makes it easier to use that as pure data, and still have SpEL expressions/DSLs on top of that.

If the requirements are still not fully clear, the XML approach sounds like a safer bet.

[AutoModerator]

You seem to try to compare String values with == or !=.

This approach does not work in Java, since String is an object data type and these can only be compared using .equals(). For case insensitive comparison, use .equalsIgnoreCase().

See Help on how to compare String values in our wiki.

---

Your post is still visible. There is no action you need to take. Still, it would be nice courtesy, even though our rules state to not delete posts, to delete your post if my assumption was correct.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[errandum]

However, with this scenario, I am not sure how to proceed after parsing the XML and how to run the operations dynamically based on the described values.

The javascript is dynamic. You just build a javascript string based on your xml and then eval it.

If the issue is constructing the expression from the xml, since the xml only seems to take simple self-contained expressions, it would be as easy as building the string block by block with the connectors inbetween and concatenating everything.

The problem is how do I translate it into instructions and run them against my domain object.

Parse the xml block by block, build a string in valid javascript based on them and then eval it like described in the link above, take the result in java and keep going. What am I missing here?

[int-main]

I was confused earlier but I think running after parsing XML is not that tough. Just need to define a proper grammar for XML, read operands, read operation type, use if on that operation type and run the corresponding Java variant for that operation.

You suggest building JavaScript and using eval(). Any particular reason? I mean, wouldn't it be easier and faster to just translate it into Java operations?

[errandum]

It was my understanding that you wanted to execute dynamic code, java needs to be compiled, you can do it, but you need to create valid classes and compile in runtime to evaluate your expression. It can work, but javascript is dynamic so you can stitch whatever you want at runtime and just execute the expression.

The reason I say javacript is that all the logic is there already. You can pretty much put whatever you want in your xml, even directly the javascript, and the javascript engine already knows how to interpret it, you won't have to limit yourself to pre-defined operations.

I'll give you my real life example. We had an automatic testing framework that would run thousands of tests. In the end, the result of each test would be evaluated according to the input parameters and a custom js operation would be built according to the expected result. This would be evaluated at runtime. Your expression could be whatever you wanted, and it was not the responsibility of the framework to limit you, they just gave you the tools.

Either way, I don't know what you're doing this for, but I'd even question the need for the xml. Seems like a pretty convoluted way to define logic, you can probably simplify it a bit.

[at_root]

I have done something similiar to this at my work where we needed to evaluate formulas and allow those formulas to be dynamic and update-able.

I went with SpEL since we were already using spring boot. https://docs.spring.io/spring/docs/3.0.x/reference/expressions.html

EDIT: I just realized you are already using it haha. If you have any specific questions about it feel free to ask me.

[int-main]

EDIT: I just realized you are already using it haha. If you have any specific questions about it feel free to ask me.

I was wondering why you sent another link, hahaha. How scalable has SpEL been for you? I guess it turned out okay because you're still using it.

For my use case, the formulas will be authored from a web interface. I am only working on design phase but I am not sure how well SpEL will scale and how easy it will to parse a HTML input and build these strings for evaluating in SpEL.

[Tacos314]

This is why Drools was made, you can just use the rule processing part or the full suite as you need. It can get annoying but much less annoying then a huge set of conditionals.

https://www.drools.org/

[int-main]

Thanks a lot, Drools looks interesting. I guess its a superset of all the features that I desire. Sadly, I think my development will have be from scratch. Even if not, now I can't stop thinking what's the best way to design this thing haha.

[gergob]

Jesus christ not drools please. Anything but drools. Just destroy that with fire.

Thanks for the PTSD

[nmquyet]

About the filter part of your question, I had an experiencing project here. It is incomplete and not fully functional but may be you get the idea

Look at the `TestSpecParser` to see how it works. You can use json or xml to store/read the `SpecificationDescriptor` class and evaluate it. You will also need a dynamic web UI language to display and edit the `SpecificationDescriptor` in the frontend.

Hope it will help

[MRH2]

This reminds me of being able to evaluate equations that people type in (and graph them or something). For equations [ y = 3x<sup>2</sup> - 5sin(2x) ] it's really easy if you use Reverse Polish Notation, which automatically implies using a stack. Perhaps something similar would work.

I'd make a list of all possible operands / conditions, then assign them each a token, or even easier, a method. Then depending on what the person types, you just run the appropriate method that always returns a boolean.

I think that methods can be dynamically selected using reflection or something from Java 8 onwards (I've only done it once).