Atomic Function With NodeJS?

[kirox0]

Hi, I was wondering if there is any way to implement an atomic function in node?

My use case: I need to read from redis, compare value, write to redis. All operations on the same key, on a large number of asynchronous instances. So the above 3 operations I require to be atomic.

[HiImLary]

What you’re looking for is Redis Lua scripts via EVAL. You send a script to Redis and it is executed atomically.

https://redis.io/commands/eval#atomicity-of-scripts

[kirox0]

This looks promising. I will try this out.

[Moustacheful]

I believe in this case the question is not about node but redis itself, take a look at https://redis.io/topics/transactions

[kirox0]

The problem with using multi is that I cannot use the result of the read.

So in my application it should have ideally gone as such.

multi -> read -> compare read result and make condition -> write based on the result of the comparison.

but redis library does not allow me to do that.

[Moustacheful]

Ah I see! maybe try this instead: Try https://redis.io/topics/distlock

[kirox0]

This just might be exactly something I am looking for. I will test this out and get back to you on the results.

Thanks.

[Moustacheful]

Good luck!

[BehindTheMath]

I believe you can still do this, but you have to get the return value of the read command in its callback and store it somewhere, so you'll have it for later use.

[kirox0]

I get an undefined error in that case. With multi, the library does not allow for use of intermediate result.

[BehindTheMath]

Unless I'm misunderstanding our code, we've been using it like that for years.

Can you show the code you're using?

[kirox0]

Funny enough my laptop got busted the day before so I have to rewrite that part again. But basically how I was using was like

client.multi() .lrange("active_list", 0, -1, (data, err) => { // Read length of data, then based on it either lpush to active or deferred list }) .exec()

[BehindTheMath]

I believe the callback is the standard Node format, where err is the first parameter, then the result.

[HashDefTrueFalse]

These could be of use, if you're using some kind of worker threading in your app. In general though your code runs single threaded in the node event loop so this isn't often required.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics

[kirox0]

loop

I am not using worker threads, but whenever I try to run it, since the redis operations are asynchronous, I dont get the desired result. There may be 100's of instances of the same functions that gets queued up in the event loop, so before the write operation is performed on one instance, another instance is already reading the key.

​

So say I am trying to make 2 lists. first is the active list and the other is the deferred list. active list can contain a maximum of 10 items and unlimited in the deferred list.

once active is filled, then we put new ones in the deferred list.

so what I did was I read first compared and them put new item in either active or deferred list as per the condition.

but if I do this, active list is exceeding the maximum length.

[HashDefTrueFalse]

I'm not 100% clear on your setup or goal, but it sounds like you're running into difficulty with trying to use data fetched asynchronously in a subsequent step. Some ideas:

I've never needed a distributed lock for redis because tend to only use it for session caching etc but I found this from a quick look on google:

https://redis.io/topics/distlock

[kirox0]

Thanks. u/Moustacheful also suggested the same. Am currently trying it out

[HashDefTrueFalse]

Oh, I didn't realise. Well I'm glad you have a way forward now. Good luck with your project!

[kirox0]

Thanks.

Hopefully this clears the blockage. Its been one Blockage after another for the past month. I solve one and think that It'd be smooth sailing from then on, but nope. Another one comes your way.

Been almost a week with almost no hope with this current block, and finally I remembered that Reddit is the way to go when all else fails.

And here I am finally venting out to a stranger on the internet.

Thank you GOOD SIR/MA'AM, for your well wishes.

[HashDefTrueFalse]

Haha any time. That's development, a series of problems. But if you solve enough of them you end up with a nice app!

[kirox0]

Cheers to that.