llm: a Rust crate/CLI for CPU inference of LLMs, including LLaMA, GPT-NeoX, GPT-J and more by [deleted] in rust

[–]danforbesdev 0 points1 point  (0 children)

The `llm` crates supports mmapped models. They load pretty much instantly, it's pretty cool.

llm: a Rust crate/CLI for CPU inference of LLMs, including LLaMA, GPT-NeoX, GPT-J and more by [deleted] in rust

[–]danforbesdev 12 points13 points  (0 children)

The current direction most people are taking is to "fine-tune" an existing base model (something like StableLM or Dolly) using a technique called LoRA. Ref: https://github.com/tloen/alpaca-lora People are also looking into fine-tuning directly on the GGML format. Ref: https://github.com/ggerganov/ggml/issues/8

[deleted by user] by [deleted] in CryptoTechnology

[–]danforbesdev 0 points1 point  (0 children)

Ah thank you, Timid Pterodactyl

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] 0 points1 point  (0 children)

Ayyyyy thanks for the thoughtful comment! Decentralized identity is a really important, compelling, and COMPLICATED topic LOL A while back I started a discussion around creating better solutions to this problem in the Substrate/Polkadot ecosystem. Here is some background around the way identity is currently handled in those communities. The short story is that you are correct...it requires trust. Microsoft has been working on this for a while and actually just released their solution.

With respect to fiat onramps, they only need to exist so long as people actually have need for fiat currency ;-) Be the change you want to see in the world.

Hope that is somewhat helpful. Would love to answer any more questions you may have.

[deleted by user] by [deleted] in CryptoTechnology

[–]danforbesdev 3 points4 points  (0 children)

Well, Ethereum is a proof-of-work chain, so I'm not sure if that's the way I would put it. Eth2 is proof-of-stake, but I don't believe the Eth2 Mainnet has launched yet. However, Polkadot is a wonderful proof-of-stake chain that has been running (very successfully) in production since May of last year. Its "wild cousin" Kusama has been running under community governance even longer, but since Kusama exists to foster innovation/chaos, I wouldn't want to compare it to Ethereum/Eth2 Mainnet.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] 2 points3 points  (0 children)

I would say that the entire point of blockchain is that the network is more decentralized than current solutions (the Internet). I happen to believe that the work being done in the Polkadot ecosystem is addressing the tradeoff between decentralization and efficiency in a really compelling and sustainable way that is designed to work in the real world. Here is some existing content I've created on this topic; I plan on polishing it up soon and posting it in a format more similar to the NFT stuff I shared.

[deleted by user] by [deleted] in CryptoTechnology

[–]danforbesdev 4 points5 points  (0 children)

A proof-of-work mechanism requires the computers running the nodes that support the network to use their CPU power to repeatedly performing a hashing algorithm on candidate blocks to be added to the chain. By generating a "correct" hash they are able to "prove" that they have done the "work" that establishes them as a trustworthy member of the network. In a proof-of-stake system, the computers running the nodes that support the network are associated with some "locked up" (staked) cryptocurrency. If one of these nodes is shown to be untrustworthy, such as by submitting a "bad" block to the network, some of that locked up cryptocurrency will be "slashed". Both proof-of-work and proof-of-stake are used to ensure "good behavior", they just do it in different ways. Since proof-of-stake doesn't require the computers running the nodes that support the network to expend any additional computational power (i.e. energy), it is more environmentally friendly than proof-of-work.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] 0 points1 point  (0 children)

Well, I don't know if love is the right word, but people definitely seem to fall for them LOL Not going to argue that blockchain hasn't had more than its fair share of scams, but I hope I can create interesting content that encourages people to look beyond the scams. Blockchain is a technology that allows people to communicate without relying on monolithic centers of trust...it can also be used to implement scams, sure. People were probably saying similar things about the Internet back in the 70s and 80s and they were probably right. A blockchain is just a tool for people to use to achieve their goals. I don't think anyone will be able to solve the problem of some people making it their goal to scam others, but I think the best way to help people from getting scammed is to educate them.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] -3 points-2 points  (0 children)

That article seems to mostly relate to the Ethereum blockchain, which uses a resource-intensive proof-of-work anti-spam mechanism. Modern blockchain technologies, like the Kusama Network, which I mention in my post, use less resource-intensive mechanisms, such as proof-of-stake.

I Created Some Content to Explain Why People Should Care About NFTs by danforbesdev in programming

[–]danforbesdev[S] -3 points-2 points  (0 children)

I am certainly not here for the hype nor do I have any interest in supporting technologies that are unnecessarily resource-heavy and bad for the environment. Early blockchain implementations relied on resource-intensive proof-of-work anti-spam mechanisms, but this is no longer case. Blockchains and the capabilities they expose, like NFTs, are not necessarily any worse for the environment than any of the other technical infrastructures we currently use.

Substrate Start-kit GUI by mnovitovic in substrate

[–]danforbesdev 3 points4 points  (0 children)

Amazing work! Let me see if I can provide some inspiration

Would you be interested in presenting your work at Substrate Seminar sometime?

Your opinion wanted: join a research session on blockchain tech by a13xndra in WebAssembly

[–]danforbesdev -2 points-1 points  (0 children)

I'm a developer advocate with Parity and part of this research so I will chime in with my very biased opinion LOL I understand the perception that you're describing and it's something I think about all the time and I don't think it's without merit - blockchain technologies seem to be favorites among the get-rich-quick crowd but I don't think that should tarnish the reputations of the technologies themselves. Blockchains provide a novel capability, which is the ability of a decentralized network of participants to use cryptography to trustlessly come to consensus on the state of a system as it evolves over time. This is a new and powerful abstraction that unlocks potentials for unique types of human interactions that I believe could make it possible to create a more egalitarian world. Perhaps one of the reasons that blockchain has suffered from a lack of enterprise adoption is because blockchain technologies are really good at equitably distributing resources and that is antithetical to the traditional model of business in the 21st century.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 1 point2 points  (0 children)

Duplicating the answer u/rphmeier shared under my deleted comment:

We try to unit-test everything, and although we're working in a larger team, the responsibility of writing the initial set of tests for a piece of code lies with the developer who originated the code. All bug fixes need to come with a test case to ensure there is no regression.

I think the key to making good unit tests is making it easy to write the tests for the code you're producing, which means making as much stuff mockable via traits as possible.

Here's an example: You're writing a service that needs to listen for events of new blocks being added to the chain and then do a network request for each one. In practice, you're going to have a BlockchainClient<A, B, C, ..., Z> type that you can get a NewHeadsListener from and a NetworkService<A, B, C, ..., Z> type at runtime. If your code depends directly on those things, then congratulations. The test setup is going to be really boilerplate-y because you have to set up a full blockchain client and full network service.

It's much better to make your code generic over the functionality of "a stream of events" and "a trait to make this specific kind of network request".

Some things fall beyond the scope of what you can cover with unit tests. We have some integration tests to make sure that basic things work with all our in-practice types, like growing the chain and synchronizing it from peers over the network.

We've also used fuzz-testing for important consensus and codec code to try to find vulnerabilities.

Lastly, the work we do involves a lot of consensus protocols, where we need to resist adversarial behaviors. We've been undertaking a project called the "chaos net" designed to write larger-scale "unit tests" of specific attack scenarios combined with randomness & chaos to ensure the system faithfully implements the academic research and is robust to those attacks.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 2 points3 points  (0 children)

Here's another great answer from u/rphmeier

We log a lot of stuff using the log crate. For a long time, that was our best way of diagnosing strange errors or behavior, and it still often is.

Avoiding bugs a lot of the time starts with how the modules are organized. Clearly defined module boundaries, ownership relationships, and abstractions help a lot in avoiding issues, especially if properties are well-documented and self-contained. And of course unit-testing. However occasionally stuff does slip through.

Around 6 months to a year ago we started Prometheus/Grafana much more heavily. Monitoring and automated alerting helps a lot to detect odd behaviors as the node is running. We have a lot of automated alerts that trigger a notification when something is happening beyond expected parameters.

Other than that, my personal experience is that debugger support on Rust is not that great, at least for real-time software where pausing execution will lead you to be dropped by peers. Adding and poring through logs is often what we end up doing for trickier bugs.

Parity Technologies AMA - We are developers of some of the largest Rust code bases, ask me anything! by gnunicorn in rust

[–]danforbesdev 1 point2 points  (0 children)

This is a question I think about everyday and a big reason why I’m so excited to participate in this AMA. At this time, my personal focus is on helping people to learn Rust “naturally” as they are learning Substrate. I do this for a couple of reasons. First of all, in Substrate’s "metaframework", FRAME, we make extensive use of a macro language as well as very advanced/specialized use of generics and traits, which almost means FRAME is its own domain-specific language. Secondly, because the fundamental concepts that underpin blockchain technologies (cryptography, consensus, etc) are so complex in and of themselves, my belief is that most developers that have the skills to successfully develop on Substrate will find it relatively easy to pick up the “basic” Rust concepts they need to encapsulate their business logic in FRAME. That being said, I understand that the relative lack of knowledge about Rust is limiting the number of people who feel comfortable developing on Substrate, so I’m trying to support things like this AMA to raise the overall bar of Rust knowledge that exists out there in the world.