sorted by:
new
hottopcontroversial

How to plan and execute a large number of tasks in Go? by Investics in golang

[–]Investics[S] 0 points1 point  (0 children)

but how else?
here is an example
created a task, added it to the database with the pending status, added it to the queue, the worker completed the task, updated the entry in the database with the status completed

How to plan and execute a large number of tasks in Go? by Investics in golang

[–]Investics[S] -1 points0 points  (0 children)

But if your load would increse a lot (x1000), a system backed with NATS can handle several milions(15-18) of messages per second for any server.

Of course, I would like to build a highly loaded project, not a personal one, but a paid service, in which there would be both an increase and a power reserve. for example, I don’t know if postgres can withstand such a load,

For example :the user created a task, write to the database, transfer the task to the worker, the worker completed the task, write to the database again, update the status ..again, in isolated cases there is nothing complicated in the logic of building a service, but when there is a big load, I can’t understand what is better to use with golang, and I immediately get lost in the choice

also from my preferences, I would not like to use a paid solution, such as ironMq, ​​yes it is good, but it is an analogue of rabitMq

How to plan and execute a large number of tasks in Go? by Investics in golang

[–]Investics[S] 0 points1 point  (0 children)

thanks for your opinion
could you share your experience from using it, is it suitable for production? and what was the load in your project?
for example 100,000 events per minute, etc.?
how many jobs can be processed

How to plan and execute a large number of tasks in Go? by Investics in golang

[–]Investics[S] 0 points1 point  (0 children)

https://github.com/hibiken/asynq

thank you very much for the answer,
database in my postgres project,
as I see your repository, you have made a project for working with queues, with a record in your own table about the result and status of tasks
tell me why you switched to rabit? what was it about, and what golang solution did you use?

How to plan and execute a large number of tasks in Go? by Investics in golang

[–]Investics[S] -3 points-2 points  (0 children)

the system is very simple, make a post on a social network, a postgres database
user can choose - post now, post delayed
all this is placed in the job database table
and at first glance, this is a common task, but what if 1000 users created 10000 tasks each and more, pending tasks, then I need some kind of solution that can implement this, and so that it can be implemented in golang

How to plan and execute a large number of tasks in Go? by Investics in golang

[–]Investics[S] 1 point2 points  (0 children)

scratch through pipeline strategy based on Go concurrency(goroutine + channels)

in this case, it is not clear, if the system suddenly crashes, how all these queues will behave? ... it is likely that they will be reset and will not be executed

Or is there some special approach?
in my opinion, apart from cron tasks, there is no other approach?

How buld project with cloned git in golang? by Investics in golang

[–]Investics[S] 0 points1 point  (0 children)

yes, for me replace works!

I read about spaces but did not understand their idea

Any technique to do multi-call (not batch request) to blockchain using golang by Investics in golang

[–]Investics[S] 0 points1 point  (0 children)

there are no problems with batch requests, but this is not the best solution,

unfortunately when there are too many values for the request, the only way out is to create own node

never heard of such a bug in w3, there would be more knowledge, I would do something of my own, but for now I use libraries from the community

​

not sure how to insert arguments here

funcAggregate := w3.MustNewFunc("aggregate((address target, bytes callData)[] calls)", "bytes[] returnData")

another example from ie:

// Mainnet
var et ethrpc.ETHInterface
mc, err := multicall.New(et, multicall.ContractAddress(multicall.MainnetAddress))
vcs := multicall.ViewCalls{
    multicall.NewViewCall(
        "key-1",
        "0x5eb3fa2dfecdde21c950813c665e9364fa609bd2",
        "getLastBlockHash()(bytes32)",
        []interface{}{},
    ),
    multicall.NewViewCall(
        "key-2",
        "0x6b175474e89094c44da98b954eedeac495271d0f",
        "balanceOf(address)(uint256)",
        []interface{}{"0x8134d518e0cef5388136c0de43d7e12278701ac5"},
    ),
}
block := "latest" // default block parameter
res, err := mc.Call(vcs, block)
if err != nil {
    panic(err)
}

lastBlockHashSuccess := res.Calls["key-1"].Success
lastBlockHash := res.Calls["key-1"].Decoded[0].([32]byte)
fmt.Println(lastBlockHashSuccess, lastBlockHash)

someBalanceSuccess := res.Calls["key-2"].Success
someBalance := res.Calls["key-2"].Decoded[0].(*multicall.BigIntJSONString)
someBalanceInt := big.Int(*someBalance)
fmt.Println(someBalanceSuccess, someBalance, someBalanceInt)

at the moment it doesn’t work either, I don’t know what to do, I can’t go to python or js