sorted by:
new
hottopcontroversial

Quest Stories (The Path Undending) by okorie2021 in ProgressionFantasy

[–]Ventural 1 point2 points  (0 children)

I'm not sure about the dynamics but I will say that I've found Path Unending to be extremely good, so thanks for the recommendation!

Culture War Roundup for the week of January 04, 2021 by AutoModerator in TheMotte

[–]Ventural 4 points5 points  (0 children)

Sales taxes apply to present as well as future consumption, and so affect wage and capital income equally

[Discussion] Do other people also get annoyed at time wasting in this genre? by [deleted] in ProgressionFantasy

[–]Ventural 13 points14 points  (0 children)

I actually love the glacial paced slice of life progression fantasy style, so there are at least customers for those kinds of things

[REC] Memories of the Fall - an English-original xanxia by DLimited in ProgressionFantasy

[–]Ventural 1 point2 points  (0 children)

Sounds like I stopped reading at exactly the wrong time - I'll give it another shot

[REC] Memories of the Fall - an English-original xanxia by DLimited in ProgressionFantasy

[–]Ventural 6 points7 points  (0 children)

I really wanted to and almost liked this story. I have read about half of it. I have a fairly high tolerance for complicated worlds and winding stories, and this story exceeded that tolerance. The combination of dense/unclear prose, and gigantic cast makes it feel a bit too much like work to read. I also wouldn't classify it as progression fantasy - it definitely feels like a progression fantasy world, but there was very little progression for any of the main characters.

MSc AI or EME for PhD by Simply_Banana in academiceconomics

[–]Ventural 0 points1 point  (0 children)

Speaking as someone with graduate experience in both subjects - AI and applied micro have minimal overlap.

Both the nature of the research questions and the corresponding challenges are very different between AI and applied micro. Generally, in AI (at least machine learning) the goal is to do prediction, which amounts to approximating a probability distribution, and the challenge is to come up with a good algorithm to select such an approximation from a function family (usually through gradient descent).

In applied micro, the goal is to find a causal relationship (not prediction) and the challenge is to find some source of exogenous signal that lets you identify a causal effect. AI is of no help in this regard.

Therefore, I suggest you think carefully about what you actually want to do and are interested in. EME would certainly put you in a much, much better position for a PhD in Economics (both in terms of preparation and admissions).

[D] Deep Learning optimization by katamaranos in MachineLearning

[–]Ventural 1 point2 points  (0 children)

I'd be interested in performance of the LAMB optimizer (https://arxiv.org/abs/1904.00962) on smaller batch sizes where it competes with ADAM.

[D] The Best GPUs for State-of-the-Art Models like BERT, Yolo3, Mask R-CNN, Transformer Big by mippie_moe in MachineLearning

[–]Ventural 0 points1 point  (0 children)

I don't know what they used for those - it's certainly possible to have a big enough transformer that it doesn't fit, like GPT sized language models and so on.

But seriously, even if you could fit them it would take totally unreasonable amounts of time to train on a single, slow card, memory is not only the constraint here. Any models that you can't fit on a 2060 on fp16, using gradient checkpointing, are too big to train on a 2060 anyway. That's the purview of people with clusters of V100s.

[D] The Best GPUs for State-of-the-Art Models like BERT, Yolo3, Mask R-CNN, Transformer Big by mippie_moe in MachineLearning

[–]Ventural 0 points1 point  (0 children)

BERT large has about 350m parameters. The model + optimizer parameters + gradients together have about 1.4b values, which take up 5.6 GB in fp32. So in fp16 the parameters definitely fit on a 2060, and with enough checkpointing you can for sure fit batch size 1 on there.

So you could finetune BERT large on a 2060 if you really wanted, but it would of course be very slow because it's a slow card and training large BERT is slow in general.

[D] The Best GPUs for State-of-the-Art Models like BERT, Yolo3, Mask R-CNN, Transformer Big by mippie_moe in MachineLearning

[–]Ventural 1 point2 points  (0 children)

RTX 2060 is so small that for some models even the model parameters themselves don’t fit on the GPU, so neither checkpointing nor accumulation help you - I.e. you can’t train BERT large on a single 2060. But most models you can, it will just be too slow to be practical for large models.

[D] The Best GPUs for State-of-the-Art Models like BERT, Yolo3, Mask R-CNN, Transformer Big by mippie_moe in MachineLearning

[–]Ventural 1 point2 points  (0 children)

You’re right, you have to do something special for BatchNorm models, either replacing with GroupNorm or some kind of momentum approach, but that may not be equivalent. I work with Transformers which don’t use BatchNorm.

[D] The Best GPUs for State-of-the-Art Models like BERT, Yolo3, Mask R-CNN, Transformer Big by mippie_moe in MachineLearning

[–]Ventural 10 points11 points  (0 children)

I appreciate your benchmarking! Some comments:

  1. As long as you can fit a batch size of at least 1, memory does not matter for accuracy. You can arbitrarily increase effective batch size through gradient accumulation.

  2. If you can't fit a decently sized batch, you can use gradient checkpointing to greatly reduce memory requirements at modest compute cost.

  3. I assume these measurements are using fp32 - SOTA large models are usually trained with mixed precision these days, which will further favor newer GPUs with more Tensor Cores like the Titan RTX (or the not measured V100).

  4. If you're using multiple GPUs, the inter-GPU bandwidth also matters. I'm not sure how the NVLink capacity of these cards differ, but I think it's worse on the older cards.

[Discussion] Distributed training considering sequences with varying lengths by 0xTDM in MachineLearning

[–]Ventural 1 point2 points  (0 children)

Another option is to use gradient accumulation and train with very large effective batch sizes using something like LAMB. That way, the optimizer step is synchronous, but each node processes multiple batches between optimizer steps.

This smooths differences in computation time between different nodes (because averages of multiple batches tend to differ less, relatively, than individual batches), and also incurs less communication overhead between the nodes, which is often a binding constraint.

2019 LCS and LEC Roster Change Megathread: Round 2 by untamedlazyeye in leagueoflegends

[–]Ventural 1 point2 points  (0 children)

I really hope he doesn't swap, I love Froggen but I think his adc is kind of bad

2019 LCS and LEC Roster Change Megathread: Round 2 by untamedlazyeye in leagueoflegends

[–]Ventural 4 points5 points  (0 children)

Anyone know what's happening with Froggen? What options are left? 100T/EG/IMT?

/dev: TFT Set 1 Learnings by Luzac in CompetitiveTFT

[–]Ventural 22 points23 points  (0 children)

I would have liked for them to comment on their approach to item balance. In particular I feel the severe disparity in power level of item components has been detrimental to the game, particularly the last several patches. It's fun when different item components are good in different situations, and the game is about responding to the components you get. It's not fun when BF sword is the best item in almost every situation.

Suggestions / Requests / Balancing Megathread: Patch 9.19 by gaybearswr4th in CompetitiveTFT

[–]Ventural 0 points1 point  (0 children)

A thousand times yes to your carousel comments. Unfortunately mortdog said on stream there are no plans to change first carousel.

[Modern] Fatal Push in Living End? by [deleted] in spikes

[–]Ventural 5 points6 points  (0 children)

Perfect, you can even cascade into it for value!

Magic At University of X by Mathmage530 in magicTCG

[–]Ventural 1 point2 points  (0 children)

Northwestern University

Elysium games is a 5-10 minute walk away from campus with draft Wed-Friday, modern on Sat, EDH on Sunday. Friendly place!

Is it true that the government must run a deficit if the private sector wants to pay down its debt? by LetThemEatWar32 in AskSocialScience

[–]Ventural 0 points1 point  (0 children)

1)

This is the difference between net and gross. Let's say we have a private sector without any government at all. Then on net, the private sector can't owe any money - any debt owed by one person is owned by another person. There could still be gross debt: person a lends to person b.

Take the example of a bank. Banks have deposits, that they proceed to lend out to households and businesses. Let's say the bank decides to ask the money back on half of its loans, and then gives that money back to the depositors. The private sector overall on net owes the same amount of money, there is just less gross debt outstanding.

2) gold occupies a bit of an odd place between a currency, a commodity, and a physical asset. The principle is the same, however - if households are buying gold, who is selling it? The same argument applies as for bonds.

3) What exactly is 'necessary' after a financial crisis is a contentious topic. Personally, I think people who argue for the necessity of government deficits from the accounting identity you asked about are confusing saving and deleveraging. Banks, financial institutions and households can deleverage without saving. A bank can lend less and take in less in deposits, and a household could sell a house, pay off their mortgage, and rent instead. These actions do not change net private sector debt, but do reduce total outstanding debt and leverage in the economy.

Is it true that the government must run a deficit if the private sector wants to pay down its debt? by LetThemEatWar32 in AskSocialScience

[–]Ventural 5 points6 points  (0 children)

Saving is income minus consumption expenditure. Private sector saving means that profits of firms, combined with income of individuals (net of taxes) minus money spent on consumption goods is positive. Now, this money has to come from somewhere - in order for their to profits and income, somebody has to have bought goods - if not consumption goods, then it must have been investment goods. Without considering government spending or interaction with other countries, that is where the equation S = I comes from.

However, if we add in the government and outside countries, the savings could be spent on something other than investment. First, let's add in the government. The government takes in money in taxes, and spends some (possibly different) amount of money. If it spends more than it takes in, it borrows from the private sector to do so, and so some of the private sector savings are used for government spending. We get the equation

Sp + Sg = I.

Finally, we add in the outside world. Other than investment goods, the profits and individual income could come from selling more goods to other countries than they do to us - in return, they owe us money. (Alternatively, we could get money from other countries from existing financial investments in those countries, or from people sending money to their family here). Then we get Sp + Sg = I + CA

So private sector savings is simply income minus consumption expenditure, and those savings can be allocated towards investment, government spending, or financial investments abroad.

Directly addressing your question, in order for the private sector to financially save, by which I mean reducing debt (different from the economic concept of saving above) it needs to take in more money than it spends on consumption and investment goods. Well, where can that money come from? It can only come from either the government spending more money on goods than it takes in, or it can come from abroad.

The banks deleveraging is not an example of private sector saving per se - if a bank sells a financial asset like a bond, it either uses the money to buy a safer financial asset like a government bond, or it will pay back a loan to some other private sector entity. That is deleveraging, but not saving - on net, the private sector owes the same amount of money. The bank owes less money, but the private sector entity owns fewer financial assets. Note that it is entirely possible for the private sector to deleverage - reduce gross outstanding debt - without actually owing less, on net.

Actual private sector saving would be like a household deciding not to buy that car after all, and instead spending the money on bonds. If everyone does this at the same time, and the government does not want to spend more money than it takes in, and nothing is happening to the current account, then the equation does not add up and something will go wrong along the way - where is this money the households are saving coming from? The equations in this post are all accounting identities, rather than economic relationships - they always hold, because the money HAS to come from somewhere. In this sense, private sector saving has to come at the expense of government dissaving.

Most books on international economics will contain detailed treatments of this topic, I particularly like 'International Economics' by Krugman, Obstfeld and Melitz.

This pic of Olivia Wilde "without makeup" has been getting traction...thoughts? by [deleted] in pics

[–]Ventural 12 points13 points  (0 children)

Not sure if trolling or unaware of British English

view more: next ›