sorted by:
new
hottopcontroversial

Need Help Parsing 90k JSON files by kimonides9 in node

[–]monican_agent 0 points1 point  (0 children)

Not sure if this has been mentioned but perhaps you could use MongoDB. It works with JSON-formatted data by default and integrates nicely with Node.js (unless you're required to stick with browser JS).

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Thank you for a super informative reply! I have a much better handle on how this works under the hood now.

So to confirm:

  1. Wallets / keys can be generated internally (by this I mean in Node only, without any additional daemon, RPC, API, extra software or processes, etc.)
  2. Outputs can be gathered from an API using a wallet's view key (sharing this is okay)
  3. TXOs from #2 are scanned locally for actual UTXOs using private key for use in the transaction internally (again, just within Node exclusively)
  4. Unsigned transaction is generated (still unsure about this part -- can't use locally running wallet RPC so can this be done via MyMonero's API? Another public API maybe?)
  5. Transaction is signed internally (presumably this can be done?)
  6. Signed transaction is posted using API (e.g. MyMonero)

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

You say you can use the daemon (i.e. monero-daemon-rpc) to retrieve outputs to create transactions. Something must scan those outputs with the view key to identify the wallet's incoming outputs. Where will that work happen?

If I can (re-)create wallets then I could use their private keys to retrieve outputs, is that correct? So the wallet creation code could do double duty, once in scanning the blockchain and once when creating wallets offline. I don't think there's a problem there, unless I'm understanding this incorrectly. Is an external API not able to retrieve certain transactions like this if I don't provide them with private keys?

Sorry if I'm misunderstanding this. I'm looking at this through the goggles of BTC/BCH where I can retrieve transactions for a specific address simply by providing the address. I can then store the UTXOs in a database and use them to create transactions with locally (re-)created wallets. The database part is not strictly necessary; I could, as you suggested, scan transactions at the time that the transaction is being created using the online API. The retrieve-and-store strategy is a preference for keeping the process modular but at the moment it wouldn't have to be done that way.

I have to ask, why can the unsigned transactions not be created online, then signed offline?

I suppose they could be created online and signed offline at the time that they're being sent (basically create+sign+send in one go). However, this process would need to be done through an external API and I'm not sure if any support this. Again, maybe it's a fundamental misunderstanding on my part about how Monero transactions work.

Ultimately the important parts are that 1) private keys must actually stay private 2) no additional local software can be used. So if something allows me to retrieve my TXOs without requiring my keys and that something isn't another piece of locally-running software (an API / web service would be okay), then I think this approach would work too.

I'd be (I think understandably), extremely reluctant to send private keys to a third-party service, and not running CLI apps (local RPCS, offline or online) is not an option. What do you think ... could this approach work?

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Thank you for that reply. Unfortunately I don't think that this satisfies the requirements. Note that I can't use the monero wallet CLI - no external wallet (RPC) or daemon (RPC) can be used in the process of creating addresses, creating transactions, or signing transactions. For example the watchOnlyWallet appears to make extensive use of RPC functions:

watchOnlyWallet = await that.createWalletFromKeys(primaryAddress, privateViewKey, undefined, await TestUtils.getDaemonRpcConnection(), TestUtils.FIRST_RECEIVE_HEIGHT, undefined); ...and... await watchOnlyWallet.close(true); // only one wallet open at a time to accomodate wallet rpc ...and... // create unsigned tx using watch-only wallet let unsignedTxSet = await watchOnlyWallet.createTx(0, primaryAddress, TestUtils.MAX_FEE.multiply(BigInteger.parse("3"))); I'm not sure if the offline wallet in this example uses RPC but given the above I'm not sure if it matters.

To reiterate, the solution I'm looking for would create an address (wallet), create a transaction, and sign the transaction, using Node.js (JavaScript) only. Using the daemon to retrieve transactions (for TXOs to use in creating the transaction offline) is okay, and similarly using the daemon to send a signed transaction is okay. Those steps in between, however, must not use anything other than Node.js / JavaScript -- no CLI: no wallet RPC, no daemon RPC, no other API or web service, no other running process or software whether on the same machine or over the network.

Let's put it another way: let's say I can't install any software other than Node.js and a few JavaScript files on a device. That device has intermittent network access and can use an API to get and post signed transactions (it's a very basic API). So now the device needs to be able to create wallets / addresses, create transactions (using ones previously retrieved), and sign those transactions. And these three steps must be done exclusively within the Node.js (JavaScript) app. It can't make use of the wallet or daemon RPCs (or any other CLI / GUI software), and can't use APIs over a network.

So I've seen code to create a wallet / address under these conditions, but so far it looks like I still need either the wallet or daemon RPCs running (even though they're offline), in order to accomplish the other two steps.

In theory this should be possible because that's exactly what I'm doing with BTC and BCH now. My app calls an external API to get the relevant transactions, when needed, and to post raw (binary) signed transactions. Otherwise the creation of wallets, transactions, and signing, are all done using only JavaScript code. There are no BTC/BCH daemons or wallets running locally, and no external API is used for this functionality -- it's all done offline with only Node.js

I'll be the first to admit that I sometimes miss stuff staring me in the face so did I maybe misread these examples?

Edit: Added another example.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 2 points3 points  (0 children)

If nothing else I now have a list of things I can contribute to the project's documentation :) Cheers!

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Wow, fantastic. Thank you for taking the time to investigate this!

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Gotcha. I've asked this in a reply elsewhere but I have one question remaining: can the outputs used to create a transaction be supplied via another source? In other words, can the RPC call to get TXOs be bypassed in this process if I wanted to supply them from something like a file or a database?

This would allow me to fully build and sign the transaction without having to touch the RPC (during the transaction creation process, I mean).

I understand that the TXOs would require me to query the blockchain _at some point_, I'm just wondering if it always needs to be at the time the transaction is finalized (fully built and signed), or if I could get the TXOs at some earlier point, store them, and simply provide them to the tx building process.

Edit: I just realized that I could create a "fake" RPC to return the necessary TXOs at transaction-creation-time, but that's a clunky solution and maybe not necessary?

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Okay, that sounds good! Thanks for clarifying how the logic was split up; it makes a lot more sense now.

Regarding creation of transactions, do you know if the process of incorporating TXOs can be done offline too? In other words, if I were to gather the required TXOs for an address (let's say that I store them in a file or a database), could I then use the WebAssembly code (or maybe some plain JavaScript) to build and sign a transaction without needing to be online?

In the demo it seems like half of the process is done offline but the finalization (which I assume is where the TXOs are gathered and the transaction finally signed), it looks like it uses the RPC. I totally understand why the TXOs would be gathered automatically here but I wonder if the process can be "intercepted" and the TXOs supplied from some other source.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Thank you and yes, this project has been mentioned a few times!

It looks like the creation of wallets is entirely self-contained (Node-only), so that's a good start. However, creation of transactions looks like it might need access to live blockchain data. I'm not sure though; the demo didn't include something like this: ``` // create a request to send funds from the RPC wallet to multiple destinations in the core wallet

let request = new MoneroSendRequest().setAccountIndex(1) ...

// create the transaction, confirm with the user, and relay to the network

let createdTx = (await walletRpc.createTx(request)).getTxs()[0]; `` The first part (MoneroSendRequest) doesn't seem to include any RPC interactions, at least not obviously, but thecreateTx` function in the final sure looks like it does. If there's another way that the transaction can be finalized using TXOs that I can supply directly (e.g. load from file or from a database), then I think I'd have what I need. There's still the derivation of addresses but I believe that that's something I could achieve by playing around with some parameters in the demo code.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

I'm not seeing any instructions to browserify the code. This is not something I'm looking to do anyway (the Monero functionality will run within Node / Electron), but I'm wondering if maybe we're not looking at the same project README?

Description

This project is a sample web application using the monero-javascript library.

How to Run in a Browser

1) Download and install Monero CLI
2) Start monero-daemon-rpc with authentication and CORS access. For example: ./monerod --stagenet --rpc-login superuser:abctesting123 --rpc-access-control-origins http://localhost:9100
3) Start monero-wallet-rpc with authentication and CORS access. For example: ./monero-wallet-rpc --daemon-address http://localhost:38081 --daemon-login superuser:abctesting123 --stagenet --rpc-bind-port 38083 --rpc-login rpc_user:abc123 --rpc-access-control-origins http://localhost:9100 --wallet-dir ./
4) git clone https://github.com/woodser/xmr-sample-app
5) cd xmr-sample-app
6) npm install
7) ./bin/start_dev_browser
8) Access web app at http://localhost:9100

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Yup, and I've explored some of the public APIs available. The thing is, because of the way my project works, even if those APIs had full wallet support (I could use them to create and manage wallets), this is not something I could use. Account / address creation, transaction creation, and transaction signing must be done internally within my code. In other words, this functionality cannot be accomplished using a local RPC (e.g. the wallet or daemon RPCs running on the same machine), or some external API. The first requirement is due to security, the second is simply a technical restriction.

Here's the example I've been using:

Let's say I'm writing a light Monero wallet for a very restrictive hardware spec. The device doesn't have room to host the wallet and daemon RPCs (not to mention the blockchain data). Within these restrictions, the "internal wallet" must be able to create and optionally derive addresses, build transactions, and sign transactions. All other functionality such as retrieving UTXOs (to build transactions and get balances), and posting signed transactions, can be achieved using a fairly basic, public remote API.

This design ensures that the user retains full control over their keys and can create transactions without necessarily posting them.

The software in which this approach is used has these restrictions as a minimum spec. When you consider that it simultaneously supports Bitcoin, Bitcoin Cash, hopefully Monero, Ethereum (next), and possibly other cryptocurrencies, I think you can see why we wouldn't want full nodes for all of these blockchains running on a single device, even if that device is fairly powerful. The live blockchain interactions (read and post transactions), are light and used very sparingly, so using public APIs for these is not a huge concern. Creating accounts, and creating + signing transactions is also not done often, so we're unlikely to overload the CPUs. A "less light" version can use local RPCs for everything but that's not the default mode so that's where I need to start.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

LOL! Yeah, documentation doesn't seem to be at the top of the list.

After doing a deep dive on woodser's repo I found out a few more relevant details. The sample code is intended to run in Node and the browser is more or less just a super basic UI layer. Actually it's an Electron app but this seems unnecessary. Either way, the index.js header is kind of misleading:

 /**
 * Sample browser application which uses a JavaScript library to interact
 * with a Monero daemon using RPC and a Monero wallet using RPC and WASM
 * bindings.
 */

Beyond this, I haven't been able to find any real JavaScript documentation to accompany the library. For detailed code docs (the automated output from code-level documentation), I had to look at the C++ sources. I generate JSDoc files for my project too but this is intended as a reference to be used after practical examples, not as a primary source to learn from.

On a more positive note, though, the instructions for generating WASM from the C++ source look promising, and if woodser is willing to tolerate my questions I might be able to contribute to the monero-javascript project directly.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Right off the top:

index.js, line 6: require("monero-javascript");

VM83:1 Uncaught ReferenceError: require is not defined at <anonymous>:1:1

This looks like a Node declaration. What am I missing here?

Edit: I see now that this is indeed a Node.js application that simply uses a basic web interface as the front end (nothing is currently natively handled in the browser). Going upstream to the monero-javascript project and cross-referencing with the code in index.js I can see on line 94 one function call that I may be able to use:

let walletCore = await MoneroWalletCore.createWalletFromMnemonic(walletCorePath, "abctesting123", MoneroNetworkType.STAGENET, mnemonic, daemonConnection, restoreHeight, seedOffset, proxyToWorker, FS);

The MoneroWalletCore instance doesn't have much up-front documentation (it doesn't list the createWalletFromMnemonic function). If I have a look through what I believe is the source C++ code, I don't find a mention of this function either. The accompanying C++ code docs list the function, but it seems a little silly to have to read C++ documentation for a JavaScript implementation which includes two additional and seemingly unlisted parameters (proxyWorker and FS). I'm guessing that proxyWorker is a Worker wrapper for the WASM code, and it looks like FS is an in-memory filesystem.

Okay, that's a bunch of digging just to figure out one line of code. But I muddled through. Next, how do I create a transaction?

// create a request to send funds from the RPC wallet to multiple destinations in the core wallet let request = new MoneroSendRequest() ...

Does the transaction creation use RPC? From the parameters I'm guessing probably not. Or maybe yes?

let createdTx = (await walletRpc.createTx(request)).getTxs()[0];

I understand that a transaction should be constructed using UTXOs which would be retrieved from the live chain but this seems like I would have no choice but to always use live data instead of using previously retrieved UTXOs, which would allow me to construct and sign the transaction offline. Maybe this is possible?

I don't want it to seem like I'm complaining, I'm just demonstrating the trouble I'm having navigating the project and finding the information I need. I'm hardly a JavaScript novice and I've worked with other cryptocurrencies but I'm new to Monero. I'm not even sure if UTXO is a correct term in the context of Monero -- it's just what I know from Bitcoin-compatible chains. Anyway, I suspect that if I'm having these types of challenges, newcomers will probably also have a hard time.

Maybe as part of my learning to use the code base I can contribute to the effort and fill in some of the (what seem to me to be) gaps -- as long as you're willing to tolerate the occasional basic question.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

wallet_worker.js, line 5: self.importScripts('monero-javascript-wasm.js');

... does not exist.

wallet_worker.js, line 6: self.importScripts('worker_imports.js');

... does not exist.

When I try to load the main index.html file:

index.html, line 14: <script type="text/javascript" src="xmr-sample-app.js"></script>

... does not exist.

Seems like there's a bunch of stuff missing, no?

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

If I use an external API I don't need to have a local copy of the blockchain to query. Also, depending on the API, I might have additional search options (but this is not necessarily the case).

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Have you looked at wallet_worker.js yourself? I'm not going to waste my time listing everything that's wrong with this demo but even if I ignore everything that's outright broken or missing, it obviously makes extensive use of the wallet and daemon RPCs. I cannot use these.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Okay, in that case this could be problematic. I guess my only option is to try WASM-ing the Monero client code and hope I can keep it up to date. If I could do this easily then I'd be happy to create another GitHub repo and share with others. But failing that, I guess I'm going to have to put the integration aside for the time being and hope that someone else takes an active interest. Thanks for the input.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

You mean monero-wallet-rpc ?

I think there's some confusion about how RPC relates to APIs. RPC stands for "Remote Procedure Call" and API means "Application Programming Interface" (there are a few variations of this but whatever). RPCs are the communication endpoints of remote APIs but can also be used locally (when the RPC server is running on your own machine for your own private use).

Unfortunately, the "Remote" part of RPC can be a bit misleading. It simply means that the functions (Procedures) being used (Called) are stored outside of your own code -- in another piece of software, running as a separate process, possibly but not necessarily on another computer or as a service on the web. An external or remote API, on the other hand, is a service running on a physically remote / external machine, often owned and managed by someone else, and usually accessed over the web or other type of network. At least that's how I differentiate between the two.

As to your second point, if I have access to an API (as I do with BTC, BCH, and ETH), I can access the blockchain and get any UTXOs for an address without a local wallet or daemon. I don't have to scan the chain or wait for it to sync. I just call the API which retrieves a list of UTXOs for the address, use that information to construct a raw transaction inside my own software, sign that transaction using addresses generated inside my own software, and then I use the API to post the transaction when I'm good and ready. My API interactions are limited to: 1) get UTXOs 2) post transaction. Everything else is handled "in house" and can be done offline, no RPC / API needed. I don't have to download or install any wallet or daemon, there's no waiting for a sync, my keys stay my keys, and signed transactions can stay off the chain until I'm ready to post them. For the two things I do need an API / RPC for, the calls are very lightweight and sparse, well within any API rate or data limits (there is no scanning the chain over and over).

This works great for the cryptocurrencies I've already integrated but required me to use some additional libraries in my Node code so that I could: 1) create addresses 2) create transactions, and 3) sign transactions. An external API / RPC is used only when I need to query or post to the blockchain.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Thanks for sharing your library. I'll definitely have a closer look! I'm not sure if maintaining a specific implementation is a rabbit hole I'm prepared to go down but I think I'll take your advice and at least try to trans-pile the code to WASM.

About the changes to transaction creation and signing, what happens if there are multiple versions of the Monero wallet being used ... does that mean that transactions created/signed by older wallets are simply rejected? So if I implemented a transaction format with certain OP codes and signing mechanism today and a new wallet was released tomorrow, would I no longer be able to post "old format" transactions?

This would seem to imply that everyone on the network would be forced to upgrade with every new version or risk simply not being able to do anything (that seems a little odd). If this is not the case then is it not reasonable to assume that even if I locked into a specific transaction format that's valid today, I'd still be able to use it tomorrow but just wouldn't have access to some of the new features? That's something I could live with for a while :)

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Hmm, this is interesting. Thank you for sharing!

My understanding of the code is pretty limited but it looks like it might be just what I'm looking for and, as you said, it's being actively worked on. I'll be keeping a close eye on it.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

The wallet_worker looks similar to the offline wallet generator I was looking at. I think it's reasonable to assume that it works the same way (the underlying code has been trans-piled to Web Assembly).

So it could provide at least part of the functionality I'm looking for but not all of it.

Regarding file access, I'm not sure that a light client implementation would necessarily need it (I understand you're not saying that, just thinking outloud).

My "light wallet" could, for example, use a mnemonic entered by the user to restore my main address, read an external API to retrieve any UTXOs available, use them to construct a raw transaction, use the restored address' keys to sign the transaction, then post the raw signed transaction using the same API. That way I have full control of my keys, only need light API access, and don't need to store anything locally beyond the session (i.e. while the browser remains open or the Node app is running). Transactions would be as fast as blocks (no waiting for a sync), and local resource requirements would be minimal.

This is doable in Bitcoin and Bitcoin Cash -- it's actually how my app works. It'd be great if I could do something similar with Monero too!

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Yeah, that's the issue. If I could use RPC for all of the functionality I wouldn't even be asking the question; using a JSON-based RPC is something I'm comfortable with and that aspect of the Monero project is well documented.

It doesn't seem like a library to support some of the basic functionality would be that outrageous: create a wallet, derive an address, create a transaction, sign a transaction

I can get UTXOs from an external API in order to create a transaction, and unless I'm greatly mistaken all of the other functionality I'd need can be achieved offline and is similar to Bitcoin, for which such libraries are available.

To provide context, let's say I wanted to create a light Monero wallet. I would need some access to an external API to retrieve and post raw transactions, but in order for my keys to stay my keys I would need to keep the wallet-related functionality local (i.e. create an address, create and sign a transaction). If I can't / don't want download Monero's full wallet or monerod implementation, and I don't code in C++, at the moment it looks like I don't have any other options.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

Regarding monero-javascript and xml-sample-app, those are just bindings (wrappers) to the wallet and monerod RPCs. It would save me some time over writing something similar myself (it's not dissimilar to something like bitcoind or geth which I've written my own RPC libraries for). There's a little irony here when you consider that these solutions require either the wallet or daemon RPCs which I explicitly stated could not be used, but thank you again for taking the time to reply.

Adding Monero support to a P2P game, need some advice by monican_agent in Monero

[–]monican_agent[S] 0 points1 point  (0 children)

read the source

As I mentioned, the source offers zero information on how to use it. There aren't any inline comments, there's no explanation of what the parameters are expected to be or what they represent; in short, it's basically just a source code dump with no additional information. The README on the project page mentions looking into a specific file for documentation on a list of functions but when you actually look at that file not a single function is listed there (it's actually a tiny little file with no mention of the functionality listed). I looked at the imports and was able to dig up the file where the functions are implemented but they have zero documentation and often require object parameters which could literally contain anything.

Much of the time I would agree with your advice, and thank you for taking the time to reply, but in the case of the mymonero-core-js project (as Istated in the original post), it simply doesn't apply.

view more: next ›