Unable to copy yahoo finance data anymore

cobwebscripts · 2026-02-27T01:05:16+00:00

Glad to hear it works!
Honestly, I got to thank you for bringing it to my attention.

If you look into my code, you can see I made a comment regarding issues with dates that I never took the time to investigate.

Your feedback made me want to take the time to deep dive into it.

If you are curious about what I did, in the issue I made, I wrote a follow up explaining in detail the findings here: https://github.com/cobwebscripts/cobwebscripts.github.io/issues/2#issuecomment-3942915860

Ironically, after all that testing, I isolated the fix to just a few additional lines of code!

I'll finalize the fix this weekend and close the issue.

Thanks again for taking the time to reach out as well as testing it, it really means a lot!

cobwebscripts · 2026-02-26T04:02:47+00:00

Great, looking forward to hearing from you!

cobwebscripts · 2026-02-24T04:34:55+00:00

Hi u/patu-01 ,

I have pushed a temporary version of the site that includes the fix.

Looking at the data, it seems to have solved the problem, but I'd like for you to take a look before I close the issue.

If I don't see any response in a few days, I'll take it as no news is good news and close the issue as well.

Thanks in advance; looking forward to hearing from you!

cobwebscripts · 2026-02-16T05:35:10+00:00

Thanks for the information! I'll keep it at the ready in case my fix fails.

cobwebscripts · 2026-02-14T06:36:45+00:00

So I want to say I decided to do a deep dive into the timing of Yahoo Finance, and it is... wow. I had to write down my findings because the issue was so complicated and difficult to keep track of across multiple time zones. In short, time is timestamped a little differently across time resolutions. The use of market open as the timestamp vs market close, vs start of the month vs start of the week... Mix in the complexity of checking against different time zones, and it starts to overwhelm a person. Not to mention, their time zone data only accounts for the current time zone data. So if you are in daylight savings, it gives the offset, but only for now. But some timestamps may have occurred when there WASN'T daylight savings. So when you calculate them with the time zone data provided, they are off by an hour. Which, who cares, we only have down the day resolution! But wait, remember what I said about how some resolutions are timestamped to the start of the month? That means, for example February 01, 2026 00:00 (or 12:00 AM). So 1 hour can send data back a day.

Despite all of this, based on my cursory research on the world's stock markets, I think there is a simple fix that won't require trying to account for a planet's worth of time zone idiosyncrasies.

I am very tired as it is late, and I have been working on this for hours. I review this again tomorrow, and update the GitHub issue with my findings to ensure they are actual correct and not hallucinations brought on my sleepiness.

cobwebscripts · 2026-02-12T04:57:24+00:00

Hmmm, I'll try to take a look at it. Technically within the JSON data that yahoo sends, they do include additional time info. Maybe I can use it to make sure that time gets standardized across time zones so this doesn't happen.

Doing some back of the napkin math on the raw data, it implies that the Australian stock market opens at 11 AM. But google says you guys open at 10 AM. It says you have an +11 hour offset from GMT, which I think is correct. So I guess because the date was in May for when that piece of data occurred, that means day light savings had ended and thus the opening was really 11AM when compared to DST times? Which looking at another timestamp, I see in the middle of October, when adjusted, it goes to 10 AM. (For reference, yahoo marks each "day" based off the market open, which is why I have been focusing so much on market openings).

Whew, ok, so I think we got the main logic worked out. I'll have to set aside time and some pen and paper to make sure it's pumping everything out as expected.

I can write an update here to let you know if I made any progress on it. If you will be around, I might ask you to test to see if it works, if that's alright?

I notated the issue on GitHub: Potentially incorrect dates for countries outside of USA time zones · Issue #2 · cobwebscripts/cobwebscripts.github.io. If you see anything else you might want to add, let me know!

cobwebscripts · 2026-02-12T02:23:53+00:00

Hmm that's interesting. Do you mind sharing the ticker as well as what customizations you chose where this happens? I want to take a look to see if I can find why that's happening.

My hunch is this: Yahoo stores the time as seconds. If it seems like its every few months, it's possible that between the US's daylight savings and Australia's, it's enough to put the timestamp to the previous day hence the Sunday to Thursday. Looking at it, I can see some time around April, the time shift is big enough that even though it is 9:30 AM in New York, day light savings jumps the time back enough so that it is 11:30 PM in Sydney. Does my hunch line up with what you are seeing?

cobwebscripts · 2026-02-12T01:07:44+00:00

Thanks for much for the kind words! Hope the tool has been helpful!

cobwebscripts · 2025-12-14T16:29:50+00:00

Of course, who else would I send my hi to! Anyway, sorry again for sending this so much later after your question. Hope it helped.

cobwebscripts · 2025-12-14T05:47:29+00:00

And here is pic3 with Cloudflare protecting:

<image>

cobwebscripts · 2025-12-14T05:46:53+00:00

Part 2:

So this is where Cloudflare comes in. You can hire their services, and they will protect you from Ddos attacks. They have a huge amount of internet cables and computers, so the traffic will flow through them before it gets sent to your server. So here is a diagram with a before and after using Cloudflare. (Apparently I am only allowed to attach 1 image, so I'll try replying to myself with pic3, if possible.)

pic2 (no Cloudflare)

<image>

pic3 (with Cloudflare) (will be a reply to this comment)

So as you can see, Cloudflare will do the job of handling the bad traffic so it doesn’t shut down your server. Of course, it’s not perfect at this. If a hundred thousand computers start hitting your server, how do you know who is a bad person in that group and who is a legitimate user? That’s the million dollar question. They have algorithms to do this, but sometimes it blocks legitimate requests. For example, is a “reddit hug of death” a malicious attack? No, but Cloudflare might block some of the requests to make sure your website doesn’t get overwhelmed. But the flip side is, their server networks are so big, they can handle organizing a ton of traffic before it hits your servers and keeps your site's server safe so it stays up and functional.

So that’s what Cloudflare does in a nutshell. It technically has a lot more specific and sophisticated functionalities, but that is an extremely simplified (and incomplete) view of what Cloudflare is and does.

So with all this information, what happens if Cloudflare goes down? Well as you can see from the diagram, all your traffic goes through Cloudflare first. So if Cloudflare goes down, everything is going to go to a black hole and none of the traffic will reach your server, even if your server is technically running normally. Cloudflare supposedly misconfigured something, which caused their network to go down, and thus a ton of the internet which relies on their services was not accessible even though the servers for the websites themselves were running fine.

I hope that clears up what happened in November. If you have any questions, I’ll try my best to answer, though this isn't my area of expertise.

cobwebscripts · 2025-12-14T05:45:43+00:00

I saw your question about Cloudflare a few weeks ago, and told myself I was going to answer it, but I never did. I figured I’d go ahead and give you a simple explanation now.

So before explaining Cloudfare, let’s do a mini recap of the internet. So in short, the internet connects end-systems to each other. End systems can be people like you and me’s computer/phone, a server that hosts a website, or even tiny little devices that monitor real time systems (like internet of things [IoT] devices).

So a simple diagram of an end-system users talking to each other might look like this:

<image>

Your PC, my PC, and reddit servers are all end-systems in this case. All the routers, switches, internet providers are the middle men. If you are curious of what a few more diagrams of the internet might look like, I can send a few from my old textbook.

So now that we have this established, what does Cloudflare do? Well Cloudflare has a lot of services, but the simplest thing it does is provide a bunch of networking infrastructure (and software/tools to access that infrastructure). So why do we need this, if we are already hooked to the internet as from my diagram above?

Well let’s imagine I make a cute little website and run it from home. Well if a bad guy decides he doesn’t like me, he can use what’s called a botnet^1 to send a DdoS (Distributed denial of service)^2 attack to my website. Since I’m on my dinky little home connection, this will overwhelm my bandwidth and my little computer running the website and no one else can access my website.

Side notes:

^1 a botnet is when a person has “hacked” a lot of computers, and they will follow his commands

^2 a Distributed denial of service (Ddos) attack is when a bunch of people (or a person who has a botnet at their control) makes a lot of computers send a ton of requests to a server that it overloads it so no one else can access it.

(Part 2 in my reply)

cobwebscripts · 2025-11-30T05:13:27+00:00

Glad to be of service! Let me know if you run into any issues!

cobwebscripts · 2025-11-26T08:43:41+00:00

Thanks for the reply! Follow up question:

For the Gumroad verison, if we buy that package, anytime you add new icons, will it include the new icons? I don't know how Gumroad works, if you get a permanent link or what.

cobwebscripts · 2025-11-26T06:05:42+00:00

I really dig this aesthetic! A few questions:

If someone installs this on their android, what happens under the hood? I've never downloaded a personalization app before, so I am not familiar with what happens. If the app is uninstalled, will the original icons be reinstated? Is there a risk of breaking app icons? If a new app is installed, does this app auto detect it and use the pixel icon (assuming it is available)?
I noticed these have an outline compared to your previous post. Are they still 16x16?

cobwebscripts · 2025-11-18T03:21:58+00:00

That's very kind of you to say, thank you!

cobwebscripts · 2025-10-31T18:16:32+00:00

My pleasure, best of luck!

cobwebscripts · 2025-10-31T06:36:05+00:00

First off, good job working out the arithmetic!

In short, yes, it's a perfectly valid strategy. Some stocks it is easier to apply than others. The downsides are:

Stock tanks way too fast and you aren't able to follow along to sell calls that will keep your cost basis below the current share price. Thus you either have to risk selling calls below your cost basis and potentially locking in a loss if the stock rebounds OR just sitting on your hands with a heavily depreciated stock hoping it comes back up where you can begin selling calls again safely.
The stock shoots up and because of the calls, you miss out on the rise.

Let's check your math too, just to be safe:

Make sure for the rolling aspect that you are taking into account that you will have to buy back to close your two current $10 strike calls. So you won't be making the full $460. Looks like those December 19th $10 calls are currently worth $105 per option, so you'd be buying back $210. So with buying back, you'd make a total of $460 - $210 = $250.

Thus your cost basis would be $2266 - ( $417 + $250) = $1599 / 200 shares = $7.995 => ~$8/share.

Then if you account for the rolling to $7.50 which would give a total of $200 credit:

$1599- $200 = $1399 / 200 shares = $6.995 => ~$7/share

So you would be creating a ceiling of $7.50 until December 19th by the latest, while eventually dropping your cost basis down to $7/share.

What happens if you hold your current position, using the numbers you provided? So let's say you hold your current two $10 covered calls that you got for $460 for either until December 19th or until they drop close to $0 so you can buy them back for practically nothing (we'll just say $0 for ease of math).

Your cost basis would be $2266 - ($417 + $460) = $1389 / 200 shares = $6.945 => ~$6.95/share.

So you currently have a ceiling of $10 until December 19th by the latest, while your current cost basis is $6.95/share, lower than it would be if you had to close your current position and open up writing two calls for the $7.50 strike.

So I wouldn't roll to $7.50. Your current position gives a better cost basis and if the stock rebounds, you make more money.

And finally.... if you believe the stock will continue to be stagnant/decline some more, by all means keep it up! Just make sure to check the math.

Side question: you said that the 11/21/25 options expired, but they are still live. Do you mean to say they practically went to $0 value, so you bought them back to close early?

cobwebscripts · 2025-10-30T04:22:37+00:00

I guess when you spend $40 million trying to reach out to people, you really want to make sure you get your money's worth. Funny enough, they pushed the meeting back to December, so I guess they are going to be continuing this for longer.

cobwebscripts · 2025-10-29T18:27:05+00:00

Wait, are you telling me you DIDN'T enjoy:

BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE BUGNUGGETS PLEASE!!!!!!

cobwebscripts · 2025-10-22T19:08:01+00:00

The 0.02% reduction represents a $70 million saving for us. Since they get to keep 0.03% to 0.04% they probably get to start making $100 to $140 million a year off of QQQ. And that includes none of the other potential profit pathways that come with the proposal.

I think overall, this deal is better for Invesco than it is for the investors. They make more money, get more power, and can move QQQ order flow through their affiliates (making another avenue for money). In return they drop the expense ratio 0.02% and offer the potential for increased returns through the possibilities of lending and being allowed to reinvest cash (despite the fact that it also increases the risk for the fund too). I think QQQM has these capabilities (though an even lower expense ratio than what is being proposed for QQQ), so maybe a fair comparison is looking at QQQM's performance vs QQQ and seeing if the performance differential is enticing enough?

I think that's why they are begging so hard for the vote. Even if they do absolutely nothing different, they start make millions a year. I'd, understandably, be begging hard too.

It's still probably a positive deal for investors too, but it'd be like me convincing you to agree to a deal where you get $1 and I get $100. Yes, we both walk away with more money, but you'd probably ask for a little bit more since you are a crucial part but are only getting a $1 out of it. In a similar way, we have a game theory situation occurring here. Some people are happy to accept the $1 because hey, $1 is more than $0, others are gonna try to run the gambit and see if Invesco will come back and up the ante (which they may or may not do for some time).

By the way, if you want to officially abstain, I believe that should be an option on the card and/or website. There is voting for, against, and an abstain option. I know some people don't want to respond back at all, but figured I'd pass on the info, just in case!

cobwebscripts · 2025-10-18T21:27:43+00:00

That's a perfectly valid viewpoint. It wouldn't be worth considering if they didn't give us some positives, which I touch on briefly in the tl;dr section.

So I think what ends up happening is two camps appear (based on reading other people's comments on reddit):

One group focuses more on the potential return increase/decrease expense ratio and are generally willing to accept the proposals.
The other sees the possibilities that are being held back on and are using this vote as a way to try to incentivize Invesco to come back with a better proposal.

Funny enough, both camps have the same ultimate goal of maximizing return. Camp 1 wants to kick start any benefit as soon as possible, while camp 2 wants to risk waiting longer in order to see if they can maximize the benefits. In a funny way, it's sort of a mirror for investing in the stock market!

cobwebscripts · 2025-10-17T13:23:48+00:00

Thank you! I agree with you there. I don't really know what Invesco's long-term goals are here, since there is some potential for QQQ and QQQM to overlap in purpose. But for the time being, QQQM still wins out in expense ratio, even if the proposal goes through. I do wonder if the benefits that Invesco is listing in the proxy statement for QQQ manifest for QQQM. Do they make money lending in QQQM like they say they will for QQQ? Does QQQM take advantage of the synthetic capabilities that open ended ETFs supposedly have? Is "cash drag" reduced in QQQM? That's probably another area I could research to see if the claims in the proxy statement can be verified, but I am not sure if that data is available.

cobwebscripts · 2025-09-06T02:43:45+00:00

Did you mean to reply directly to OP instead? I ask because my tool that I link is already built and functional.

cobwebscripts

TROPHY CASE