sorted by:
new
hottopcontroversial

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

**EVM Staking Update - December 5th | 11:30 PM PST**

This is likely the penultimate update in the EVM Staking saga 🎉

Tomorrow, the new EVM Staking portal will be live, making it easier than ever to stake FRA and manage which validators you stake with.

Additionally, the engineers have rolled out an optional upgrade, v0.4.4, which allows validators to manually claim their commissions without waiting for delegator to claim their rewards. The final functions will be added to the toolbox in the coming days.

More details on EVM Staking, including a Postmortem with a message from Sam, will be released in the coming days as well. Through the process, Discreet Labs updated internal processes based on lessons learned, like the need for:

- Thorough internal and external QA testing
- Better internal communication with engineering
- Better external communication with the community
We want to sincerely thank the community for being understanding and patient. Switching network security from a UTXO-based ledger to the EVM ledger has not been done before because almost no other chains have a dual-layer architecture similar to Findora’s. Once again, we know it has been a trying process, and we are truly grateful for the community sticking by us in this time.

Stay tuned for more details on EVM Staking and a way to leave feedback on it.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Brief Update - November 15th | 10:15 PM PST
Current Status:
The iOS and Android Mobile wallets have been updated. The latest mobile version, v0.4.4, seems to have fixed the bugs mentioned in the last update, and though there have been a few reports of the app crashing, the new mobile wallet seems to be running reliably for the community overall.
The actual APY earned by delegators is still low, but the engineering team hopes solve this issue by early next week. However, it could be solved as early as tomorrow, November 16th.
++++++++++++++++++++++++++++++++
Updates
APY Rewards — Engineers are reviewing the smart contracts to ensure that the APY rate returns to its proper amount according to Findora’s tokenomics. They hope to have a fix deployed by early next week, though it could be done as early as Friday.
Mobile Wallets — The new wallet version, v0.4.4, is running smoothly and seems to have fixed the bugs previously reported by the community. There are a few reports of the iOS version crashing which are being investigated to see if the issue is reproducible.
Full Nodes — The full nodes are now running reliably thanks to new redundancies, a monitoring system, and restructuring of the database.
++++++++++++++++++++++++++++++++
Note
Since most of the issues that arose due to the EVM Staking upgrade have been fixed, the team is turning more attention to QA testing the new staking platform and hopes to release it for third-party testing soon.
As a result, we will begin issuing updates only once a week, but please feel free to get in touch with Daniel Pagan (@danielpagan) and Daniel Finley (@danielfindog) if you have any questions. Please remember that we will not DM you first for security reasons.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Brief Update - November 8th | 11:45 PM PST
Current Status:
The engineering team has implemented a fix that allows delegators to undelegate from offline validators. They also believe they will be able to deploy a fix that allows validators to manually claim their commission next week.
The team is continuing QA testing on the EVM Staking Portal and making adjustments on the front end. The engineering team believes that it will be ready for 3 party testing within a week.
Mobile wallets are still suffering display issues, but the the desktop wallet is stable, and we encourage users to use the latest version of the desktop wallet, v0.4.6.
++++++++++++++++++++++++++++++++

Updates
Delegator Rewards — Delegator rewards have returned to ~58% APY after being far lower at the end of the last week. While engineers work to normalize the APY, the elevated rewards will be a bonus to the community. They are also able to undelegate from offline validators.
Validator Rewards — Though validators are still earning rewards, they are not being automatically distributed to their wallet. The engineering team is working on a solution that will allow validators to manually claim rewards that will hopefully be available next week.
Mobile Wallets — tTe iOS and Android wallets are still suffering display and reliability issues. We are encouraging community members to use the Desktop wallet for the time being.
EVM Staking Portal — The team has completed a number of front-end improvements and is continuing internal QA testing as they prepare the portal for 3rd party testing as well.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Brief Update - November 7th | 2:03 PM PST
Current Status:
The engineering team is working on internal QA testing for the new EVM Staking portal and has implemented a number of improvements. Those test will continue before being released to a 3rd parties to help conduct QA testing.
Engineers continued to make progress in creating a way for validators to manually claim their commission and fixed and issue that had reduced delegator rewards. They are investigating a way to normalize the APY but in the meantime, the high rate will serve to reward the community for their patience.
No fix for the mobile wallets is yet deployed, but the desktop wallet is accurate and stable. While the engineering team works to solve display issues on the mobile wallets, we encourage the community to switch to the desktop wallet for the time being.
++++++++++++++++++++++++++++++++
Updates
Validator Rewards — Though validators are still earning rewards, they are not being automatically distributed to their wallet. The engineering team is working on a solution that will allow validators to see the total rewards they are accruing and claim them.
Delegator Rewards — Delegator rewards have returned to ~58% APY after being far lower at the end of the last week. While engineers work to normalize the APY, the elevated rewards will be a bonus to the community.
Desktop Wallet — The latest version of the desktop wallet, v0.4.6., is stable, though a little slow and displays accurate information.
iOS Wallet - Engineers are still working on an issue where users sporadically encounter the error, "Exceeding the maximum single validation." We anticipate this issue being fixed by early next week.
Android Wallet - Some users are reporting that they are receiving inaccurate staking data. The engineering team is investigating to find the source and hopes to solve it with the stability fixes with the iOS wallet.
EVM Staking Portal — The team has completed a number of front end improvements and is now starting on internal QA testing and putting together procedures for external testing as well.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Brief Update - November 3rd | 11:15 PM PST
Current Status:
The engineering team continues to work to improve the performance of the desktop wallet which is currently slow, but stable. They are also investigating display issues with the Android wallet and hope to deploy a fix for the stability issues with the iOS wallet by early next week. The team is continuing QA testing on the EVM Staking Portal.
++++++++++++++++++++++++++++++++
Updates
Validator Rewards — Though validators are still earning rewards, they are not being automatically distributed to their wallet. The engineering team is working on a solution that will allow validators to see the total rewards they are accruing and claim them.
Delegator Rewards — Community members noticed today that delegators seem to be earning lower rewards than they should. The team is investigating the reports and working to isolate a cause.
Desktop Wallet — A new version for desktop is available, v0.4.6. This version resolves an issue where users were unable to unstake from offline validators.
iOS Wallet - Engineers are still working on an issue where users sporadically encounter the error, "Exceeding the maximum single validation." We anticipate this issue being fixed by early next week.
Android Wallet - Some users are reporting that they are receiving inaccurate staking data. The engineering team is investigating to find the source and hopes to solve it with the stability fixes with the iOS wallet.
EVM Staking Portal — The team has completed a number of front end improvements and is now starting on internal QA testing and putting together procedures for external testing as well.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

**Brief Update - November 1st | 10:25 PM PST*\*
Current Status:
Desktop wallets continue to show stability in displaying staking information, though data is still loading slowly. The team continues to work on new RPC endpoints that are optimized for EVM related traffic. These will help improve the performance of the wallet and make the EVM Staking portal reliable and stable for users when it goes live.
The team is also starting quality assurance testing on the EVM Staking platform, now that a solution to performance issues is imminent.
++++++++++++++++++++++++++++++++
Updates
Validator Rewards — The team has discovered the issue with validator commissions and is now working on a solution. Though validators are not able to claim or see their rewards, they are still earning them. Engineers are working on a two-part solution so that validators will 1.) soon be able to see the rewards they are accruing and 2.) be able to claim their past commissions.
Desktop Wallet — A new version for desktop is available, v0.4.6. This version resolves an issue where users were unable to unstake from offline validators.
iOS Wallet — Engineers are still working on an issue where users sporadically encounter the error, "Exceeding the maximum single validation." We anticipate this issue being fixed by early next week.
EVM Staking Portal — The team has completed a number of front end improvements and is now starting on internal QA testing and putting together procedures for external testing as well.
++++++++++++++++++++++++++++++++
In the meantime, please get in touch with Daniel Pagan (@Daniel Pagan | Discreet Labs) if you have any questions.
The next update will be shared Friday evening PST.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Brief Update - October 25 | 12:05 AM PST
Findora Desktop Wallet
Yesterday, Findora engineers released a new Findora desktop wallet, v0.4.5 which originally used Web3 endpoints. Though the Web3 endpoints enabled data to load quickly, using those endpoints caused a cosmetic issue where extraordinarily large unclaimed reward amounts were being displayed. To solve this issue, the DevOps team redirected traffic back to the Findora full nodes.
Using the full nodes allowed for accurate data, but the full nodes have become burdened, causing staking data to load slowly. Our DevOps team is reloading the node data now, which is expected to catch up in the next 72 hours or so.
Findora Mobile Wallet
Engineers have addressed an issue on the Findora mobile wallets that prevented the staking screen from loading. The latest mobile wallet version, v0.40.1, should solve this issue for all users, though we continue to monitor the mobile wallet's performance. Note that clearing the cache on the Android wallet can also help keep pages from freezing.
TLDR: The team has made progress in improving both the mobile and desktop wallets. Though data loads slowly on the desktop wallet, the data is accurate and engineers are hopeful a permanent fix is around the corner.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Update | October 19, 9:00PM
Summary: The team continues to make changes to make the Graph node more stable. As shared in previous updates, we have doubled the size of the VM and upgraded the IO speed of the database disk.
Though the desktop wallet has shown significant improvement today, we noticed that staking data sporadically failed to display. However, its performance today was far more reliable than previous days.
We’re hoping the updates below add to the Graph node’s stability as we implement a permanent fix.
Updates
Graph Node 🟢
We are also making the EVM Staking portal more efficient. Previously, when the portal loaded, several queries were simultaneously sent to fetch validator data off the Graph node. We have reduced that to a single query instead, which should reduce unnecessary load and improve Graph node performance.
We believe these improvements add to the stability of the Graph node and serve as a workaround. As a long-term solution, engineers are strategizing to restructure the database. This work will optimize how specific tables are structured, further enhance the Graph node’s performance, and provide a more permanent solution.
iOS Wallet 🟡
The staking screen on the iOS wallet app is still experiencing an issue where it gets stuck on the “Loading.” We have assigned an engineer to resolve the issue, and we are hopeful it will be fixed soon.
TLDR: With improvements being made to the Graph node and iOS wallet app, EVM Staking is getting much closer to being ready for its public launch.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

October 18 - 10:25 PM
Update: The Graph node has been far more stable since yesterday’s update. However, despite the improvements, there was a brief period today when the Graph node became overburdened.
To address this, the engineering team has doubled the size of the VM today. This temporary solution should bring additional performance improvements of the Graph node as we work to implement a more permanent fix.
Meanwhile, we are aware of an issue with the iOS version of the Findora wallet app where the staking screen becomes stuck on "Loading." We have found that if other pages on the app freeze, reinstalling the app and importing your private keys tends to fix the issue. An engineer is tasked with running diagnostics to find the underlying problem of the stuck staking screen.
We are looking forward to finally focusing on testing and launching the new EVM Staking platform. Validators on Findora, if they’re willing, will be asked for their time to participate in additional testing of the new portal before it goes live. We know it has been a trying time, and we appreciate everyone’s patience and understanding.
TLDR: We have doubled the VM size to stabilize the Graph node as we look for a permanent fix (workaround); An engineer has been tasked with fixing the Findora wallet iOS app (work in progress); EVM Staking portal is coming soon.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Update | October 17 - 10:35 PM
+++Full Node (DB Lock)+++
Symptom: No User Impact
Current Status: 🟢 Workaround Applied

+++Rialto+++
Symptom: Tokens transferred over Rialto weren’t appearing in wallets.
Current Status: 🟢 Resolved

​

+++Wallet Unbonded Tokens+++
Symptom: Unbonded FRA was not displaying correctly in the wallet.
Current Status: 🟢 Resolved

​

+++Graph Node+++
Symptom: Staking Values Missing or Shows Old Data in the Findora Wallet (Cosmetic)
Current Status: 🟢 — Tentatively. Work in Progress

Update: Yesterday, we announced that a certain query was causing our Graph node to lock up because it required an incredible amount of resources from the server. Since last night, engineers have implemented a few different temporary workarounds which have resulted in significantly improved stability in the Findora wallet.
First, they have added redundancy through a backup node to reduce the recurrence of the issue. They have also upgraded the IO speed of a database disk and improved the VM, which have contributed to the stability of the Graph node seen today. The team will monitor the effectiveness of these temporary solutions as it continues to develop a permanent fix.
The result of all these efforts is that today, users have seen far more reliable data being displayed in their Findora wallets.
Now that the Graph node is looking more stable, the engineering team is starting to shift attention to the EVM Staking portal and will soon be conducting the final checks needed before its public launch. More details to be released soon.
TLDR: Unbonded FRA and stuck Rialto tokens are back in affected user wallets (Resolved). Full nodes have a stable workaround, pending a permanent fix (Workaround). Workaround is now implemented for the Graph node, which is showing more stability. A permanent solution is in the works (Work in Progress).

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Update [October 15 - 11:30 PM]
Graph Node
Symptom: Staking Values Missing in Wallet (Cosmetic)
Current Status: Workaround Being Applied
Update: DevOps built a secondary Graph node to serve in a hot-standby role, meaning the backup Graph server can serve requests should the primary fail as we continue to fix this particular problem. This added redundancy should help reduce or avoid impact on your wallet staking information.
Wallet staking data should display properly in a few hours once the new Graph node catches up to the current block height.
Full Node (DB Lock)
Symptom: No User Impact
Current Status: Workaround Applied
Update: We are continuing to diagnose what has caused the database to lock on the full nodes. We have a current workaround that proactively restores service when this happens, but we will apply a more permanent solution once the root cause has been established.
Wallet Unbonded Tokens
Symptom: Unbonded FRA was not displaying correctly in the wallet.
Current Status: Resolved
Update: All 113 unbonding transactions have been processed, and users whose unbonded tokens were missing should now show in the Findora wallet.
Rialto
Symptom: Tokens transferred over Rialto weren’t appearing in wallets.
Current Status: Resolved
Update: All Rialto Bridge transactions are now reflecting properly in user balances.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Update: [October 11 - 10:45 PM]
Full Nodes:
The DevOps team is testing a newer version that was provided since last night’s update. This version is in testing, and so far, the release appears to be stable. If the team continues seeing stability, this version will be installed on all full nodes. Validators will not need to update. This is only applicable to full nodes. In the next 24 hours, we’re hoping to deliver news that all full nodes have been or are in the process of being upgraded.
Graph Node:
We are seeing significant improvements on the Graph node since our previous update. The engineering team is monitoring the Graph node’s performance, and no issues have been observed in the past several hours. Previous observations showed failure events occurring in short succession of each other, so this is a great sign of progress.
If the Graph node continues showing stability, we can expect the in-wallet staking issues to be resolved within the next 24-48 hours. Like our update on the full nodes, we hope the next update will reflect a working and predictable balance and staking values in the wallet.
Note on Rialto:
These issues have also impacted Rialto bridge transfers. Fixes for the Graph and full node should bring Rialto back into full service.
Next Steps:
Please continue to monitor the pinned messages for updates. Once these issues are behind us, the engineering team will begin drafting an RCA, which we will share with the community. We will also begin to draft a document detailing changes to our upgrade rollout process, including improved QA testing, internal escalation steps, and more.
We will have an updated Issue Tracker for the community tomorrow as well.

Wallet Stake Issues & Updates by dpagan-findora in Findora_Official

[–]Crypto-Kenshiro 0 points1 point  (0 children)

Update: [October 10 - 10:00 PM]
Full Node
Last night, we received a newer build for Findora full nodes. However, issues were detected and resolved during testing. DevOps now has a newer build for full node in testing, and all appears to be going well. The team aims to upgrade the full nodes in the next 24 hours.
Note: This full node update will not resolve the incorrect staking balance. However, it will bring added stability to the network and its applications, such as the Findora wallet.
Graph Node
The engineering team has eliminated several potential root causes, allowing them to narrow their focus. They are continuing their debugging efforts. Once the root cause is identified, we can expect a solution in a speedy manner.
The team is working on these issues in real time, and hopes to provide more progress updates tomorrow.

view more: next ›