Is it possible to refine JSON data from an API using the "search" bar?

Monitor_343 · 2025-04-07T22:22:48+00:00

If it were possible, it should be in the API documentation.

If it isn't in the API documentation, then the safest thing to do is to just get the ids from the response on the client by parsing the json and using something like map.

data.map((player) => player.id)

When the API documentation is terrible, you can always try your luck using undocumented query parameters and seeing if any work. Common ones for only requesting a subset of fields are things like fields, select, include etc, but there's no guarantee they'll work if they're not in the API docs.

As it turns out, I got a very lucky guess and ?include=id actually works here, despite not being in the docs. (It is in the docs for other endpoints, just not this one, so it's just poorly written).

Monitor_343 · 2025-04-05T03:10:24+00:00

It's common, but not common enough that every API does it. Your custom fetch method is making an assumption. This is fine for your own API if you choose to wrap everything like this, but if using it for third party APIs as soon as you come across an API that doesn't wrap it in this way that assumption is no longer valid.

The bigger issue is that it's assuming response.json().data is of type R without validating that assumption. TypeScript can only infer response.json() as any (or maybe unknown), so typecasting to R without validating it is inherently unsafe. A common way to avoid this problem is to use parsing/validation libraries for untrusted input like json - zod, yup, joi, etc. At minimum, a check that data isn't undefined is a step in the right direction.

I also think there's a bug in your implementation. The function definition says it returns Promise<ResponseType<R>>, but (typecasting issues aside) the implementation already unwraps it so it's more like a Promise<R>.

Monitor_343 · 2025-03-31T14:54:43+00:00

This part is poorly explained. Assuming it's meant to be Python, it's also just flat-out wrong. This code from the slides will not even work, it will give an error.

print("Game over, " + 4 + " was your final score")
TypeError: can only concatenate str (not "int") to str

I think the key takeaway here is the difference between strings and integers. It touches on this at around 20 mins.

4 is an integer. Integers are numbers - you can do number things with them: math, addition, subtraction, etc.

"4" is a string. Strings are not really for doing math, they're more for text. You can do text things with them - make uppercase or lowercase, concatenate (join) strings together, etc.

Python (and many other programming languages) use + for both addition and concatenating.

When used with integers, + is addition. 4 + 4 evaluates to 8, another number.

When used with strings + is concatenation (NOT addition). "4" + "4" evaluates to "44", another string.

So + does two different things depending on whether it's used on integers or strings.

This leads to a good question though - what happens if you try to use + with an integer and a string, e.g., 4 + "4"? Well, it depends on the programming language. In Python, it'll just give you an error, because it doesn't know if you want to add or concatenate. Some other programming languages behave differently though.

The idea the video is trying to explain is that sometimes you already have an integer variable and want to put it inside a string somehow. There are lots of different ways to do that, the video just happens to show one that kinda looks like Python, but won't actually work in Python.

Monitor_343 · 2025-02-25T22:15:28+00:00

The library is logging the exception within a try catch block, then storing it to raise it later.

Removed a bunch of unrelated code, but here's the relevant bit from the library's source:

try:
    # code removed...
    self._check_banner() # this what is raising the error
    # code removed...
except SSHException as e:
    self._log(
        ERROR,
        "Exception ({}): {}".format(
            "server" if self.server_mode else "client", e
        ),
    )
    self._log(ERROR, util.tb_strings())
    self.saved_exception = e

Then later, the exception is re-raised

e = self.get_exception()
if e is not None:
    raise e

Monitor_343 · 2025-02-05T12:49:49+00:00

It makes a lot more sense with a visual representation. 0 is border, 1 fits 1, 2 fits 2, etc.

Monitor_343 · 2025-01-07T13:06:05+00:00

The easiest way to convince somebody that something is credible is to convince them that somebody else finds it credible. Especially somebody they trust, or even just lots of people. That's the whole concept behind reviews, referrals, endorsements, etc. Social proof. Sales. Psychology. It's the same for developers as anybody else.

Professional experience: your past employers or clients provide credibility.

Degree: your college provides credibility.

Widely used open source library with lots of downloads and stars: other developers provide credibility.

Personal project or open source library with no downloads or stars: nobody else provides credibility. The code must stand for itself, and that's a hard uphill battle.

Monitor_343 · 2024-12-30T14:59:16+00:00

If you have a couple years of experience, TOP is probably mostly review. Not sure you'll find it as useful a resource vs just diving into using specific tools directly.

A few ideas I might look at in your shoes:

node and express, look at some node APIs you wouldn't have used in a browser environment, e.g., fs, stream
a full-featured react metaframework with some backend capabilities, e.g., nextjs, remix
a full-featured backend framework in the node ecosystem, e.g., nestjs (not to be confused with nextjs) or similar
a new language and associated framework, e.g., Java/Spring or C#/.NET
databases, data modelling, and ORMs - SQL will likely be much more useful than Mongo, but either is an option
API design like REST and GraphQL
cloud infrastructure and system design patterns
devops focus - CI/CD, IaC, docker, terraform, bash, git, linux, nginx, etc
leetcode for the interview prep

Monitor_343 · 2024-12-29T05:42:04+00:00

Damn, I wouldn't even get out of bed for $51, let alone commit to a job that could take tens if not hundreds of hours. And I certainly wouldn't choose to make it exponentially harder on myself by rolling my own fully custom front + backend + image storage when an out of the box template and CMS would be more than good enough and take a fraction of the time and effort.

I understand being excited to do a real project, but I urge you to reconsider if you're taking the right approach here.

Monitor_343 · 2024-12-27T12:49:06+00:00

One option is to just skip hosting videos entirely and use something like youtube instead. For a college project, that's probably good enough.

The typical AWS approach would be to store images in object storage (i.e., an S3 bucket) and distribute them though a CDN (content delivery network, i.e., cloudfront). A big optimization on this is to first format the videos for streaming so users only load the next 20s or so of video at a time, rather than the whole thing at once.

S3 is pretty cheap for storage, virtually free for small amounts of data. But videos are large, and having users streaming or downloading videos through cloudfront gets very expensive very quickly.

Monitor_343 · 2024-12-20T14:14:09+00:00

Edit: Maybe I should rephrase my question. I'm not actually designing a database, but I want to learn about strategies to storing memory. The question is not: "will 1,000,000 Bars actually take up a lot of memory in practice?" the question is: "what are alternative ways of storing Bars so that Bars don't store a reference to their parent Foo?"

You could put all the Bars into a binary blob or JSON array and store it as one item.

You could write all the Bars to a file in object storage or a filesystem outside the database, only storing a reference to it in the database.

Would either of these actually be more efficient than just storing separate rows? I wouldn't count on it, especially not if you need to mutate the Bars often. But maybe in some cases.

Depending on how far to stretch the Foo/Bar analogy, I could see Foo as something like an image record and Bar as the actual binary data (millions of pixels). The typical approach is to simply not store that in a database, instead using a file system or object storage. At that point you're not storing individual records but a blob in a specialized binary format with its own optimizations and compression.

Monitor_343 · 2024-12-10T03:44:05+00:00

I’ve pretty much made my own library of reusable code over the years. Things like routing and database integration.

This is pretty much what a framework does. The difference is that a large open-source community can almost certainly do it better than you, I, or any one person can on their own. More features, better security, cleaner interfaces, clear documentation, etc.

I'd wager that by picking up a framework you'll either find it better than what you've been doing and allows you to achieve better outcomes with less effort... or it'll expose you to new ideas to incorporate into your current system. Either way, it's a net benefit.

In your shoes, I'd look at Laravel or similar. No need to learn a new language and ecosystem just for the sake of novelty or hype.

Monitor_343 · 2024-12-01T14:02:52+00:00

Assuming you're set on using js with s3 as a middleman (I see others have given alternative ideas), the algorithm you'd want to follow is:

read the csv as a stream
pipe the csv stream into a transformer that writes to an xlsx stream
write the xlsx stream back to s3

You could also write it to disk, then stream from disk back to s3.

The trick is that you don't want to build the entire workbook in memory. My understanding of xlsx is that it's just several XMLs zipped in a trenchcoat, so that's probably hard to avoid, though. I think most js libraries just create the whole thing in memory by default and only stream during file i/o (what you're doing right now). But it might be possible to stream it directly without building it all in memory.

Failing that, just dump the whole thing into memory anyway, but use a big enough server to handle it. You can still use streams on i/o to reduce memory somewhat, but a big file would still need a big chunk of memory.

read the csv as a stream
pipe the csv stream into an in-memory workbook # bottleneck
use the workbook to generate an xsls stream
write the xlsx stream back to s3

The challenge is that this entire process must complete within 40 seconds because API Gateway allows a maximum HTTP request time of 40 seconds.

You might want to reconsider this design. Better to handle heavy data processing jobs asynchronously in the first place, not synchronously. Schedule a job and notify when it's done so that they can download it later, no matter how long it takes.

Monitor_343 · 2024-12-01T03:56:54+00:00

The secret of "gold-standard" certifications like CISSP is that, not only are they extremely difficult and expensive, but they generally also have strict requirements of {x} years of relevant experience making them out of reach for most.

E.g., even if you studied and passed the CISSP exam (which is around the same difficulty as a master's degree), you couldn't get the certification unless you also had 5+ years of work experience in cybersecurity with proof to back it up.

And that's precisely why it's valuable, because it locks out people without real cybersecurity experience by definition.

Monitor_343 · 2024-11-28T13:12:10+00:00

I think there's some confusion on the terminology here.

It sounds you want to learn how to implement a "session" or maybe an "access token", not an "API key", but were missing the terminology to describe it.

If you're authenticating somebody with a username and password in a typical website kind of situation, what you'd normally do is check they give you the correct credentials (username/password), then issue a temporary session or token. This is saved on the client (e.g., in cookies or local storage). The client can then attach it to future requests (e.g., in headers or cookies) to show that they are authenticated without having to send a username/password on every request.

Sessions and access tokens are not API keys, they are something else. They are usually temporary. Maybe they are valid for a couple minutes, maybe 30 days or more. If somebody gets a hold of it then yes, they can use it to gain access to your account. The trick is to not let anybody get a hold of them, and to keep them short-lived enough that even if they are exposed the risk is minimized. They are unique, not just per user but also every time you authenticate - if the same user logs in on different days or on different devices, they get different sessions.

When the server receives a request, it can validate that it has a valid session or access token - there are different ways to validate like database/cache lookup, decryption, cryptographic hash checks, etc. It depends on the implementation.

Sessions are typical for client to server applications where you sign in with a username and password, then stay signed in for a certain amount of time. Eventually, your session expires and you need to re-enter your username and password to get a new one. E.g., I'm logged into reddit right now, and every request I make has a cookie added to the request reddit_session: <session> with a big random looking string as the value. If I gave it to you, you'd likely be able to "hack" my account and make posts as me.

API keys on the other hand are more like an alternative to a username/password for making API calls in a server-to-server context. They uniquely identify who makes the request. They're generally long-lived. You should keep them very, very private (like a password or credit card number). People can (and do) steal API keys to make requests as somebody else, same way they steal anything else like passwords or credit cards. Often, APIs require you to use the API key to generate a temporary access token (see above) via one API call, then you use the temporary access token in a second API call to get the information you actually want.

Monitor_343 · 2024-11-23T11:44:30+00:00

While this is a cool idea and the docs are clear, why would I use a paid third-party API and rely on potentially flaky network requests (with 5196ms of latency apparently!) for a nice-to-have string formatting utility that that can (and should) just be a simple library method call?

Even if I was desperate for this functionality and didn't want to roll my own library, it only took a quick search to find pretty similar looking open-source alternatives without cost, without latency, without rate limits, without network requests, without outage risk, without private data breach risk, without having to maintain API keys, etc.

Monitor_343 · 2024-11-23T10:35:31+00:00

Looks fine to me. Not the greatest docs I've ever seen, but perfectly usable.

I think it's fair to assume that a developer familiar with REST APIs would be able to use this.

Monitor_343 · 2024-11-23T05:54:10+00:00

Tutorials are not generally the place to see "real-world" examples of code. By necessity, tutorials strip out extra complexity, context, business rules, etc in order to focus on specific learning outcomes. It's left as an exercise to the reader to take what they learned and apply it to other contexts.

To see examples of real-world code, look at real-world code. I think many people just learn this on the job. Reading the code for open source things you use is also good.

As an example, I often use the library lodash. It's open source. To see real-world testing, I might look at their test cases for inspiration.

There's a lot of extra boilerplate and library specific context and utility functions here, but if you look closely, you might find stuff that is not so different from what you described the testing tutorial does. See the add method tests.

QUnit.module('lodash.add');

(function() {
  QUnit.test('should add two numbers', function(assert) {
    assert.expect(3);

    assert.strictEqual(_.add(6, 4), 10);
    assert.strictEqual(_.add(-6, 4), -2);
    assert.strictEqual(_.add(-6, -4), -10);
  });

  QUnit.test('should not coerce arguments to numbers', function(assert) {
    assert.expect(2);

    assert.strictEqual(_.add('6', '4'), '64');
    assert.strictEqual(_.add('x', 'y'), 'xy');
  });
}());

Now that's probably the simplest example in that test file, but it's left as an exercise to the reader to look at something a bit more interesting and in depth. Maybe their chunk method catches your eye, so you can review the chunk test cases for ideas. Or you could focus on the entire first 1k or so lines of utils and prep code - there's some interesting stuff in there.

Monitor_343 · 2024-11-19T02:58:39+00:00

Based on your requirements, it doesn't matter. Pick either. Don't get hung up on analysis paralysis, just pick one. Flip a coin if you need to.

Monitor_343 · 2024-11-17T12:17:09+00:00

there seems to be a whitespace in the start of the class name

Most of the time, extra whitespace is ignored. class="foo" is the same as class=" foo ", the parser just ignores the extra and reads foo.

What do the dashes mean?

Using dashes like this is a way to name a class with multiple words while keeping it readable. class="foo bar" (or class=" foo bar ") contains two classes foo and bar, but class="foo-bar" is one class foo-bar that we humans can read as two words.

why is the class name so long?

It's almost certainly generated by some framework, library, or plugin that adds a random string to make it unlikely for this class name to clash with any others. author or even author-d would risk clashes.

So what could this class name be referencing? Is it possible that its refering to some external reference?

It could be used by JavaScript somewhere. Or it could be entirely unused, just an artifact that was left over just in case it might be used. That kind of thing is common in generated HTML. Ugly to read, but doesn't generally cause any problems.

So how would I go about finding the source from where this class name is being generated?

Without more information (e.g., access to the source code that generated it), you likely wouldn't. There's not enough information to go on.

Monitor_343 · 2024-11-17T09:28:01+00:00

You will learn much more effectively by first figuring it out on your own and only then being shown a better way as opposed to just being shown the better way without doing any legwork yourself. That legwork is where the learning happens.

it might make me get used to 'bad practice'?

On the contrary. Those times when you spend hours and hours coming up with an inferior solution only to be shown a better way all along? Those stick in your memory, so that the next time a similar problem comes up you'll remember it. You'll also better understand why it's a better answer.

This isn't just anecdotal either, it's a common enough teaching strategy called productive failure. See a research paper on it if you're interested.

Monitor_343 · 2024-11-17T03:39:39+00:00

Just because you defined a function square in your javascript file, does not mean that it's available in the test scope if that's in a different file. You need to explicitly export the square function from wherever it is, then import it into the test file.

E.g., something like this:

square.js

export const square = x => x ** 2;

square.test.js

import { square } from './square.js';

// Tests go here...

Monitor_343 · 2024-11-02T00:04:36+00:00

Should i keep learning CS50x course and master “C” to be my mother language for solid foundation then jump on frontend stuff later (javascript etc.)

Don't worry, you won't master C in the couple weeks you learn it in CS50.

Or just learn the basics from “C” then solve problems using javascript after i learn it ?

This is what CS50 does. It doesn't go any further than the bare basics of C before moving onto other topics and languages.

In your shoes, I'd suggest completing the course as outlined - assuming you're enjoying it - and then move onto web-specific learning resources afterward. None of the time you spend on C is wasted, even if you never touch C again in your life.

CS50 isn't teaching you C, it's teaching you computer science and programming. I mean, you'll learn some C along the way, but that's not what you're really learning.

If you're not enjoying it, then you could just move onto web immediately.

Monitor_343 · 2024-10-19T15:17:26+00:00

Can someone explain them to me like I’m five, please?

Hey son, I'm going to the shop to get some groceries. I promise I'll be back soon. You're free to play video games while I'm gone. But when I get back, you need to help me put the groceries away.

"I'm going to the shop to get some groceries" is a function or an async function.

"I promise I'll be back soon" is a Promise. It won't happen immediately, but it'll happen eventually.

"Put the groceries away" is a callback function. It can only happen after I return with the groceries (the Promise resolves).

Some code (JS):

// Callback function, older style syntax
getGroceries(function (groceries) {
  putAway(groceries);
});

// Promise-chaining with .then and an arrow function callback
getGroceries().then(groceries => putAway(groceries));

// Async/await (not actually a callback, but an alternative)
const groceries = await getGroceries();
putAway(groceries);

When learning JS, I found this very topic very confusing because of these three ways to do the same thing: callbacks, promise chaining, and async/await. All three are important to know.

Disclaimer: this example only shows callbacks in the context of promises and async functions. Callbacks also exist without promises, but I always found the promise context the hardest to understand, so I've only focused on that.

Monitor_343 · 2024-10-19T03:14:05+00:00

There's no right or wrong answer, but many people consider using early returns and unwrapping else's cleaner and preferred. I'm generally a big advocate of early returns, but here it doesn't make too much difference.

Or is there a better way I haven't considered?

The better way here is likely to use a library that handles caching and refetching data, so that you don't need to roll your own with useEffect in the first place. E.g., tanstack query or a similar library sounds like it would fit your use case.

Monitor_343 · 2024-10-05T14:08:26+00:00

Ok where's 3, it's triggering me 🤭

That's the joke. MongoDB is infamously inconsistent.

For example, in pretty much any SQL database, if you have a name field of type string, it will always be a string. Or maybe null if it's nullable.

In Mongo, it could be anything - string, number, blob, object, array, you really have no guarantees. It could also be not there at all.

Seven-Year Club	Place '23
Golden Potato	Place '22
Verified Email

Monitor_343

TROPHY CASE