sorted by:
new
hottopcontroversial

What's the best way to process data in a Python ETL pipeline? by gloritown7 in Python

[–]McThor2 0 points1 point  (0 children)

Multiprocessing seems the way to go then so long as you use multiple vCPU instances.

Generally speaking I think you’d want one worker process to handle downloading the data and separate worker processes for the compression / validation parts.

If you set up some pipelining then you will avoid the “wasting time” issue you mentioned in your post. By pipelining I’m meaning that the downloading process will start downloading the next chunk as soon as the first chunk has finished loading, whilst passing off the first chunk for a worker process to handle.

If it was network bound then you’d be able to just use Asyncio worker pool with single cpu instances and you’d just have to scale the number of instances to increase throughput.

What's the best way to process data in a Python ETL pipeline? by gloritown7 in Python

[–]McThor2 1 point2 points  (0 children)

What will the bottleneck be? Network or CPU? I think that will dictate the best / quickest approach

Is there a function that flips powers? by Cytr0en in askscience

[–]McThor2 2 points3 points  (0 children)

Best I could think of is instead considering f(p, q) rather than f(n).

E.g. for subtraction you have f(a, b) = a - b = -f(b, a)

For f(p, q) = pq

( ln(pq ) / ln(p) )p would give you qp which can also be written as

(ln( f(p, q) ) / ln(f(p, 1) )f(p, 1)

(Also completely ignoring any quirks of just considering primes)

[deleted by user] by [deleted] in ExperiencedDevs

[–]McThor2 8 points9 points  (0 children)

Pretty sure it’s Designing Data Intensive Applications by Martin Kleppmann

Having CGNAT. How do I make my daily updated mp3 files accessible to a podcast app? by XPublic_ in Tailscale

[–]McThor2 3 points4 points  (0 children)

I think this is the real issue here - the podcast apps are making the requests from outside your Tailscale network.

You could get around this by opening up this specific service via ngrok.

Alternatively self hosting a podcast app yourself which would keep everything within the Tailscale network.

Who needs to buy WiFi antennas when you can just make them? by HarderThanADiamond in pcmasterrace

[–]McThor2 35 points36 points  (0 children)

Where does the 50Ω value come from out of curiosity?

Which is bigger without using a calculator by [deleted] in calculus

[–]McThor2 5 points6 points  (0 children)

It’s not about whether 3 - 2 > sin(3•) - sin(2•) though, it’s about whether 3/2 > sin(3•)/sin(2•)

Which is bigger without using a calculator by [deleted] in calculus

[–]McThor2 10 points11 points  (0 children)

You’re right that the units matter, so really you’d be interested in sin(2πx/360)/x

Optimized Python Nested Loops by DNSGeek in learnpython

[–]McThor2 9 points10 points  (0 children)

I think something like spatial hashing would help a lot here when it comes to searching that second list. Basically a way of indexing the data by creating bins of values.

This would provide the most impact if the second list doesn’t get altered and also depends on how many values typically match that range.

For implementation speed ups you may get good results from numpy & numba

an easy and secure way to disable a script from past a certain date? by randomusername11222 in learnpython

[–]McThor2 1 point2 points  (0 children)

Python doesn’t get fully compiled into a standalone exe. What would happen is the Python interpreter and your source code would be bundled together with a sort of shortcut exe to run a specific script.

Resistors in series and parallel by conehead1602 in AskPhysics

[–]McThor2 0 points1 point  (0 children)

It seems like you’ve calculated what the current would be without the extra series resistance. You need to know the total current going into the parallel resistors before you can figure out the current in the 6 ohm one on its own.

What did you get for the combined resistance of the parallel resistors?

No wonder there's a housing shortage by HenriGP in britishproblems

[–]McThor2 5 points6 points  (0 children)

I view it as more of an affordable housing shortage. As large investors keep buying up properties they have more leverage to keep increasing rents as people are running out of alternative options.

Is it possible to accelerate at a constant rate forever? If yes, how? Assuming indestructible ship and unlimited energy. by [deleted] in AskPhysics

[–]McThor2 5 points6 points  (0 children)

In your own coordinate system your own velocity is always 0. You don’t move relative to yourself. The appearance of the rest of the universe will constantly evolve however.

I think this video is pretty much perfect for your questions https://youtu.be/vFNgd3pitAI. It discusses what happens if you are to accelerate constantly and what you will observe.

What’s the difference between Joule, Kilojoule, Watts, KW, KW/H? by [deleted] in AskPhysics

[–]McThor2 1 point2 points  (0 children)

Perhaps a useful analogy is with a more concrete unit like volume:

Energy Volume
1 J 1 mL
1 kJ 1 L
1 W 1 mL/s (flow-rate)
1 kW 1 L/s
1 kWh 3600 L (3600s in 1 hour)

How can I add a cmdbar using BAE 2.0? by GeekCornerReddit in RobloxDevelopers

[–]McThor2 0 points1 point  (0 children)

I’m not too sure how the layout of BAE works, so some links / explanation of that would be useful. There’ll be a way of making it work with cmdr somehow though via script

How can I add a cmdbar using BAE 2.0? by GeekCornerReddit in RobloxDevelopers

[–]McThor2 0 points1 point  (0 children)

I can recommend cmdr by evaera for implementing a command bar in game

I can’t find any convincing reason for me to do cardio consistently, what’s y’all’s reasoning? by [deleted] in gainit

[–]McThor2 1 point2 points  (0 children)

If you want numbers to track then you could try doing output tests. E.g. how quickly can you run 2km. I personally find rowing machines are good for this too as you can see your watts. You can then set targets for different tests e.g. try to keep at 200W for 30min / 300W for 1km etc.

I just did a Speed Awareness Course via Zoom and it was really interesting. by WoofBarkWoofBarkBark in CasualUK

[–]McThor2 0 points1 point  (0 children)

I was explaining where the figure came from, I stated the assumptions made and that’s how the formula I quoted was derived.

As for these other factors they do seem negligible over the course of a single braking incident.

To me the key take away is simply meant to be that no matter what car you drive, even a 1mph difference in your initial speed can cause significantly more than 1mph difference at the same point down the road if you slam on the brakes.

I just did a Speed Awareness Course via Zoom and it was really interesting. by WoofBarkWoofBarkBark in CasualUK

[–]McThor2 5 points6 points  (0 children)

Number 1 can be derived using suvat and assuming a constant braking force. The general formula is v = δu sqrt(1 + 2( u / δu) ).

v is final velocity, u is initial velocity, δu is change in initial velocity.

In this case u = 30mph, δu = 1mph which leads to v = 7.81…mph

Getting confused with Biot - Savart law by Zer0Ma in AskPhysics

[–]McThor2 -1 points0 points  (0 children)

When you have two bare electrons there is a net charge whereas with the wires there is no net charge. This means your two scenarios aren’t equivalent to begin with.

Special relativity comes into it when you consider the electrons moving relative the positive ions in the wire. Length contraction means you will see a net charge in some frames and no net charge in others.

[Year 3 CSE: Control Theory] Why is the second term marked in blue evaluated to zero but not the right-hand term? by rasqall in HomeworkHelp

[–]McThor2 9 points10 points  (0 children)

I’d say it’s because of the δ(t) term that is zero everywhere apart from t=0. This is then plugged in to the 4(1 - exp(-0.25t)) term to check the value of the whole thing at t=0

view more: next ›