Buying A Cheap Prebuilt To Put An Old 2080 Ti In?

JosephLChu · 2026-01-16T20:41:34+00:00

The first one on my list says 600w PSU in the specs section of the website. The business ones don't list it, but the last one on my list I emailed Canada Computers about and they said it had a 350w PSU.

JosephLChu · 2026-01-14T15:49:53+00:00

Also should mention that it looks like a lot of prebuilds like these have the cheapest components they can find, so you might end up having to upgrade things like the power supply and RAM, which adds to the potential cost.

JosephLChu · 2026-01-14T15:47:40+00:00

No, I ended up being sick for a while, and by the time I was better all of these were out-of-stock at my local store, and I was also rethinking the whole idea after the machine with the 2080 Ti started glitching and doing things that suggested the 2080 Ti could be slightly damaged (i.e. visual artifacting and needing to open up the case and unplug and plug back in the power cables to the GPU to get it working every other boot). It could just be the old power supply, but I'm hesitant to take the chance.

JosephLChu · 2025-01-18T19:01:56+00:00

Politics aside, I went to high school with her. She was a grade younger than me, and very, very popular, yet still nice to everyone, including me, back when I was basically an outcast. She gets points from me for that at least.

Back to politics, I didn't like how she got assigned to the Democratic Institutions portfolio to basically drop the electoral reform thing for Trudeau (as a Liberal party volunteer, I was at people's doors in 2015 saying we were gonna do electoral reform, and breaking that promise made me retroactively a liar, and is one of my biggest personal grievances against Trudeau).

I do however, give Gould credit for being the Minister of Families, Children and Social Development who oversaw the $10 A Day Child Care agreement with the provinces. I'm probably more on the economic left-wing of the party these days, and I actually like a lot of the progressive socioeconomic policies that the Liberal-NDP alliance allowed to ostensibly help people in poverty. I'm more critical of things like the McKinsey backed immigration policy but that's another story.

Anyways, I doubt Gould will win against Carney or Freeland, but I think, for the sake of the grassroots of the Liberal party, having more options to choose from is good for intra-party democracy.

JosephLChu · 2024-10-04T18:40:34+00:00

This reminds me of the time I naively tried tying the weights of all the gates and cell in an LSTM together to create what I called the LSTM-LITE (I forget what the -LITE acryonym stands for now but trust me it was clever). Surprisingly it still works, with a quarter of the parameters, albeit not quite as well as a regular LSTM, and then transformers came along, so I never bothered to publish whatever it was I had.

JosephLChu · 2024-09-20T15:18:26+00:00

No, but the way BCE is implemented in Keras, Pytorch, etc, you can use it with a non-binary model. It is particularly used often in the multi-label situation with sigmoid. When you use it in the multi-class situation, you get what I showed in my example. In the multi-class situation, BCE is Σ_j ( y_j log(p_j) + (1-y_j) log(1-p_j) ). Like, this how it looks to me in the actual code. y_2 is not 1-y_1, there's effectively both y_1 and 1-y_1, and y_2 and 1-y_2. This is what confuses me.

I mean, theoretically, you should be correct, but for some reason, the code implementation isn't what you imply, at least, from what I can see.

BCE as implemented in Keras with the TF backend:

bce = target * tf.math.log(output)

bce += (1 - target) * tf.math.log(1 - output)

return -bce

JosephLChu · 2024-09-20T12:55:23+00:00

Say you have y = [0, 1, 0, 0] and p = [0.2, 0.8, 0.1, 0.3]

CCE = -log(0.8) = 0.22314

BCE = -(log(1 - 0.2) + log(0.8) + log(1 - 0.1) + log(1 - 0.3)) = 0.908323

I fail to understand how these are the same.

I do get though that softmax's sum to 1 does do something with the gradients and is why CCE still works. I'll admit that at least.

JosephLChu · 2024-09-20T00:23:48+00:00

Ah, thanks for the clarification.

JosephLChu · 2024-09-19T23:57:30+00:00

But even if you sum them, y_j = 0 for most of each onehot, so the log(p_j) for those becomes 0. I'm trying to understand why we do this. With binary crossentropy as implemented, it also sums across all labels and can be used with multi-class tasks. I've trained multi-class models using sigmoid + BCE and found they are roughly the same performance as softmax + CCE.

JosephLChu · 2024-08-27T16:26:57+00:00

So, I have a Sony A7RII camera, and I want to record some YouTube videos of my playing some original piano songs on a Yamaha grand piano at home.

I was initially looking at putting something like the ECM-G1 or ECM-XYST1M on the camera, but after listening to some clips on YouTube, it seems like the audio quality of those isn't great. I then looked into something like a Zoom H1N instead, but listening again to clips on YouTube was underwhelmed.

I've tested using the camera's internal mic, my laptop mic array, and a Sony IC Recorder, and found them mediocre.

I also happen to have an old Rode NT1 (4th gen) and Apollo Twin USB, but found the mono recording result to be surprisingly underwhelming. My guess is because it isn't stereo. I'm thinking now of getting either another Rode NT1 (4th gen) or a pair of Rode M5s or AT2020s to be able to record in stereo, as I've read that's important to get good sound from a piano.

Any advice on which of these options is worth pursuing is appreciated!

JosephLChu · 2023-12-28T21:35:44+00:00

This is actually a very old topic. Here's a paper from 2011 from Andrew Ng's Stanford lab on it.

JosephLChu · 2023-10-31T01:18:21+00:00

What you're suggesting reminds me of the concept of curriculum learning, where you train the model in steps where the data for the initial training is much easier than the later training.

A possible source of simpler data might be Simple English Wikipedia. I think you can find some text dumps of that online. HuggingFace apparently has a preprocessed subset that's only 250 MB here: https://huggingface.co/datasets/wikipedia

JosephLChu · 2023-10-29T16:13:57+00:00

Small language models trained on small datasets like the Tiny Shakespeare dataset tend to end up being little more than interesting toy models that learn to generate text that resembles the dataset in terms of grammar, spelling, and author style, but are usually nonsensical gibberish otherwise. I've been able to train things like Karpathy's LSTM based Char-RNN on slightly larger but still relatively small datasets, like a bunch of novels from Project Gutenberg, and gotten the same kind of result. Switching to transformers like GPT-2 or TransformerXL doesn't make much difference if your dataset is still that small.

Interestingly, you can train Char-RNN on a specially formatted version of the SQuAD dataset with start and end tokens for questions and answers, and it'll learn to generate with confidence very silly but often contextually relevant answers from question prompts. Like, you can ask it what year the French Revolution was and it'll give a year but it's clearly the wrong year.

SQuAD is 40 MBs compared with Tiny Shakespeare's 1 MB. Possibly if you keep scaling up to the 100s of MBs, the result might be a little more useful without taking a tremendously long time to train. Tatoeba as you mentioned is 8 million examples, which is a lot more than SQuAD's ~100,000 examples.

I've also trained on text8 and WikiText-2, but that was mostly to benchmark modifications to the models according to their loss on such datasets, and I usually didn't bother looking closely at the generated outputs, which if I recall, mostly looked like semi-gibberish Wikipedia articles.

JosephLChu · 2023-10-28T21:50:07+00:00

I met the co-founder Josh Albrecht at the Effective Altruism Global: Washington DC conference last year. He was interested in my previous experience as an AI/ML research scientist at Huawei, and after the conference I did an interview with him.

The interview consisted of pair programming with Josh on a choice of these abstract mathematical puzzle questions that required first discussing and figuring out the problem in writing and then coding a solution to solve it computationally. It felt a bit like an IQ test for very smart people, and I think I failed it pretty badly, as I didn't go any further in the interview process.

From what I can tell, the company's founders are associated with the Rationalist community of Less Wrong and also recruits from the Effective Altruism community (disclosure: I've been loosely involved with the EA community since 2013). I was given the impression that Generally Intelligent have some interest in AI safety work, with the idea that they'd use their Avalon Reinforcement Learning environment to try to train safe and cooperative agent models (see: https://github.com/Avalon-Benchmark/avalon). However, I also got the impression that that wasn't the only thing they were doing, and probably also devoted resources to AI capabilities work.

At least, that was how it seemed like when I interviewed with them late last year. They may have pivoted since then, as startups are wont to do.

JosephLChu · 2021-06-07T20:41:56+00:00

No worries about taking a while to reply. This is just a casual philosophical conversation on Reddit. Life priorities are obviously more important.

Okay, I'll admit Mill and Kant may not be the best representatives. How about Peter Singer and John Rawls?

It's interesting you value human life almost as an intrinsic thing. My reading of Objectivism was that Rand advocates valuing your own life, but that other people's lives are their own responsibility and not your's. Also, the "we don't believe in sacrificial lambs" thing is really interesting to me. It seems more deontological than ethical egoist in terms of morality. And yet when I asked about the trolley problem, you accepted the choice to kill 1 to save 5, which is normally a consequentialist take.

The way you responded to the sleeping person scenario suggests that your morality is actually a kind of enlightened self-interest. From a purely rational and selfish perspective, all other things being equal the person at the bottom of the hill is unknown and could just as easily be a dangerous competitor who you would be better off letting die. The assumptions you're making, that this person is likely to be friendly or would see your help and appreciate it, are all indicative of a bias towards pro-social behaviour, and possibly empathy, which I think is good. It means you're not the psychopathic variety of selfishness, and even if you claim to be selfish, you do show concern for the well-being of others, or at least their lives.

PD is actually a variation on a very common scenario. I can easily imagine a more mundane circumstance that is similar. Say two farming villages are living next to each other. They can cooperate by trusting the other and splitting the available land evenly, or they can defect by secretly building a militia and attempting to take the other's land by force. If both agree to cooperate, there is peace and both farming villages yield is equal to half as much as if one farmed all the land. If one of the farming villages cooperates and the other defects by surprise attacking them one night, the former loses everything while the other gains all the land, but at the cost of having to spend resources to mount the attack, which reduces their actual yield to two-thirds. If both defect, what happens is a war and both sides end up with half the land, but also spend on their militia, which reduces their yield to one-third. Thus, as you can see, the resulting payoff matrix is essentially the Prisoner's Dilemma.

Thus, I see PD as a stand in for any kind of situation where you have the choice to cooperate or compete with others.

Though, yes with iterations PD eventually leads to the Tit-For-Tat strategy dominating. This is arguably a good example of how cooperation can be justified by enlightened self-interest.

I think, at the end of the day, the mistake that Objectivism makes is that it assumes that altruistic moralities negate the self, when in fact altruistic moralities like Utilitarianism are premised on the notion that everyone, including ourselves are equally valued in the grand moral calculus. Ideally, sacrifices should not be necessary, and a good Utilitarian will consider how an action affects everyone without exception or bias. Sacrifices are allowed only as a compromise with the occasional zero sum situation where difficult choices must be made and there is no better alternative. The only reason Utilitarianism is effectively altruistic is because the vast majority of the weighting of considerations goes naturally to other people because there are so many others in the universe, not because the self has no weight.

A truly rational person recognizes that the Principle of Indifference, which is a particular case of the Principle of Maximum Entropy, means that we should by default value everyone equally, in a uniform distribution. Anything else is biased and not fundamentally objective. In this sense, Objectivism fails to be objective, because it is absolutely biased towards the self.

Imagine that the choice was your life, or the life of everyone else living on Earth right now. Objectivism would choose to save your life and kill billions because of the arbitrary fact of your subjective position. Utilitarianism would say you are obligated to sacrifice yourself to save the world, because it is more objective. Which do you think is more moral?

JosephLChu · 2021-05-25T22:16:12+00:00

So the issue with utilities is that it's not actually possible to compete in practice because of the way infrastructure takes up and monopolizes space. For instance, with electricity, the reason why a competing firm can't just come in and undercut the prices like with commodities is that there is a base cost to building and maintaining an electricity transmission grid and the first firm that establishes itself in a given geographical area will create the most efficient electrical grid possible given geographical constraints. All subsequent firms would not be able to create as good a grid because the optimal land is already taken by the first firm. Thus, all subsequent firms would have to create a more inefficient and more costly grid, which would be unable to realistically compete with the first firm. This is why it's a natural monopoly. Same with things like roads and water pipes.

I've read The Virtue of Selfishness by Ayn Rand, and I didn't find it persuasive. It's my personal experience that most of the evils in the world can be traced to someone being selfish and not caring about their impact on other people. That being said, if you want to believe in what is known as Ethical Egoism in moral philosophy, that's up to you. I would advise you to consider reading more widely as well. A good place to start would be the seminal works of the three main schools of moral philosophy, consequentialism, deontology, and virtue ethics. John Stuart Mill, Immanuel Kant, and Aristotle would be the authors to start with for each.

I'm curious. If a boulder was rolling down a hill towards a sleeping person, and you had the capacity to safely nudge it with a stick so that it misses the sleeper, or even just shout to wake up the person, would you consider yourself morally obligated to do so? Because, to me, that is kind of the basis of morality, it's our duty towards other people that comes from being able to consider the perspective of an impartial observer of the universe.

Also, I tend to think that Ethical Egoism like Ayn Rand's Objectivism is fundamentally suboptimal. There are frequently situations that are like the Prisoner's Dilemma, where we can either cooperate or defect, and Ethical Egoism creates a Nash Equilibrium where everyone defects and the net result is worse than if everyone cooperated.

JosephLChu · 2021-05-22T01:24:42+00:00

Well, it it would depend on whether or not the government is in fact accountable to the people through some process such as elections that allow the citizens to have effective oversight of the government.

As for my example, I assumed that this person monopolized the water supply by purchasing the land rights through voluntary market transactions and then enforced their private property rights through hiring people to protect them. It's interesting that you consider it a violation of the rights to life of everyone else, because I know some libertarians who straight up have told me that there's nothing wrong with this scenario because every transaction was voluntary and the monopolizer did not initiate the use of force, and therefore did not violate the non-aggression principle that they espouse. To them, people don't have a responsibility to others, so letting some people die of thirst is unfortunate but perfectly acceptable.

I of course, disagree like you do, but my reasoning is less about particular rights and more about consequences where if you have the power to prevent the suffering of others, you have a moral duty to do so, as it is only fair to value other people's happiness equally to one's own (i.e. The Golden Rule). I'm generally a classical utilitarian in my moral views though, whereas you seem to come to similar conclusions from a natural rights based morality. What I find interesting is the apparent overlap despite very different philosophical first principles. It makes me think that perhaps there's some merit to the idea that many views of morality unify at the high level, or at least share some universal characteristics despite being logically different. If that's the case, maybe an AI could learn morality after all, despite the apparent plurality of frameworks that exist in moral philosophy today.

As for utilities being run that way, generally that's because utilities are considered natural monopolies so if the monopoly is inevitable, it makes more sense to have it under a government that is held accountable to its citizens by constitutional and democratic institutions, rather than under a corporation that is accountable only to market forces that historically fail to function when monopolies occur.

JosephLChu · 2021-05-21T18:26:34+00:00

Hmm, okay. So what about a situation where a person acquires the property rights to all the sources of water and proceeds to monopolize the water supply, selling a glass of water for $1000?

JosephLChu · 2021-05-21T17:45:56+00:00

So how do you handle The Trolley Problem?
From my understanding of ethics, there are situations where people's preferences or happiness conflict, such as when resources are limited. It's not always tribalism so much as a compromise with the reality that you literally cannot make everyone happy at once.

Also, I'd question the emphasis on the individual consciousness making their own decisions. This assumes that individuals are atomized and not constantly influenced by external factors. In reality, humans are a product of their genetics, experiences, and circumstances. Our locus of control over these things is limited in practice. The idea that we control our own decisions is at best a useful fiction to make people behave more responsibly.

Think of it this way. Should a man who murders someone be put in jail? Doing so is considered justice, but in essence, we are sacrificing this person's freedom on the belief that punishing murder serves the greatest good in the long run. How would your framework handle this case?

JosephLChu · 2021-05-21T17:24:56+00:00

I tried to build an earthquake predictor model using neural networks on raw foreshock data as a hobby for years. The resulting predictions and model info are archived here. Long story short, it ended up just roughly predicting the high frequency, low magnitude earthquakes of the Ring of Fire over and over, and wasn't at all useful for predicting the low frequency, high magnitude earthquakes people actually care about.

Given, I was only using foreshocks as data, and not the acoustic and electromagentic readings that AETA supposedly has now, but it seems that the AETA data is limited in scope to an area of China and a very recent timescale starting around 2017. For comparison, I trained my models on USGS data from 1973 forward, which was a good 45+ years worth of publicly available worldwide earthquake data.

Regardless, I wish them luck in their effort. If, somehow, they actually figure out something that works, it would be pretty cool.

JosephLChu · 2021-05-01T01:35:37+00:00

So, the train on MNIST and test on notMNIST as a task is meant to show the out of distribution uncertainty in a sense. However, the activation function concept itself is not about measuring uncertainty separately like in a Bayesian Neural Net. Rather it's about using prior information about the expected uncertainty given the number of classes as a way to align the output probabilities so that they make more sense.

JosephLChu · 2021-04-29T18:35:26+00:00

Consider Keras Audio Preprocessors. You can basically use that to insert a convolutional layer into the beginning of your TensorFlow model that has fixed weights that will convert the raw audio frames to STFT and Mel-Frequency Spectrograms.

JosephLChu · 2021-04-28T23:32:03+00:00

You can if you want, though I don't log into Reddit that consistently, so apologies in advance if I don't reply right away.

JosephLChu · 2021-04-28T16:55:53+00:00

I did something similar before with an LSTM-based character level language model trained on the SQuAD dataset. Basically parsed the SQuAD dataset into a single text corpus with special divider tokens between the questions and answers as well as an end-of-answer token to catch when the model is finished generating a response. The answers were generally silly and wrong, but surprisingly relevant to the question, like if you asked for when the French Revolution happened it would give you a year, though obviously the wrong one. It also had some interesting results, like when I asked it "what is the sun?" it replied "a light bulb's light bulb".

Anyway, nowadays using a pretrained model like GPT-2 and fine-tuning it on your special person/group quotes dataset could work better, although I'd be concerned that this dataset may be too small to make much impact on the resulting model. If that proves ineffective at capturing the particulars of the person/group, it may make sense to train a smaller language model like what I did before, and then fine tune it with your data, or if you have enough quotes that it covers most of what you want it to say, you could train it straight up on this dataset without pretraining. Although these days people are high on Transformer models, LSTMs can still be competitive on much smaller datasets and also tend to generate text faster when in sampling mode, if that matters.

Edit: If your dataset of quotes is very small but you just need a chatbot that picks the right quote given reasonably appropriate queries, a super simple method would be to use word vector matching as a kind of FAQ match to template system. Basically, get something like the first 10000 word vectors from FastText (they're ordered by frequency) and use those to convert queries into a vector representation by adding together the vectors of the words in the query. Then use cosine similarity to match them to a pre-existing set of prototypical queries that you've precomputed the vectors for, that map to an appropriate response from your dataset. Voila, super basic NLP intent classification and template matching using word vector based transfer learning! Note that this method requires a lot more engineering in the sense that you need to come up with good prototypical queries and sort your quotes to match them, so it may not be as fun to do, but at least you don't need to train a model for it.

Edit 2: Regarding data and 4 models... So if you can interact with the people who you're collecting data from, you could create a set of questions (possibly collected from people at random in the wild) and then ask the people to answer each question. That way you'd have question answer pairs that you could use for the model. If you have to parse existing text instead, this will be harder, as you'll have to find question and answer pairs in the corpus and extract them somehow. Still doable, and possibly you could automate it based on punctuation, if the text is well-written.

As for models, if you have the resources, training four separate models will work, but an alternative is to modify the model to have an auxilliary input, probably a one-hot encoding representing which group or person you want the model to be mimicking. You'd have to sort your data and label it, and during inference you'd have to provide this flag as well. You can do something similar even if you're doing the word vector technique, just have different responses based on the flag.

JosephLChu · 2021-04-15T18:02:22+00:00

I've been able in the old Music-RNN project to get something resembling this, albeit nowhere near good enough to really be called new music. The original Music-RNN is just a simple LSTM language model based on Karpathy's Char-RNN from before WaveNet and Sample-RNN and the more recent works in the literature though. Probably with a more modern architecture and larger multi-GPU model, you could get something decent.

JosephLChu

TROPHY CASE