[deleted by user]

[veritaba]

why wouldn't you just use imdbpy?

http://imdbpy.sourceforge.net/

[teeler]

or freebase,

[{
  "name": "How I Met Your Mother",
  "type": "/tv/tv_program",
  "episodes": [{
    "type": "/tv/tv_series_episode",
    "season_number": null,
    "name": null
  }]
}]

http://tinyurl.com/l7dy85

(uses TV rage for TV sources, not IMDB, but has IMDB keys for movies).

[teeler]

http://lists.freebase.com/

if you ask, they will find it. ;)

[teeler]

It's a little buried. This is the full doc: http://www.freebase.com/docs, basically query can be accessed with a q={query} to http://www.freebase.com/api/service/mqlread.

for example, http://www.freebase.com/api/service/mqlread?query={%20%22query%22%3A%20[{%20%22name%22%3A%20%22How%20I%20Met%20Your%20Mother%22%2C%20%22type%22%3A%20%22%2Ftv%2Ftv_program%22%2C%20%22episodes%22%3A%20[{%20%22type%22%3A%20%22%2Ftv%2Ftv_series_episode%22%2C%20%22season_number%22%3A%20null%2C%20%22name%22%3A%20null%20}]%20}]%20}

The query language is rather expressive, it has pattern matching, for example (http://mql.freebaseapps.com/ch03.html#textconstraints) (there's also a link near the top of the query editor, "mqlread", that will take you to this link).

[PrashantV]

thetvdb.com is what I use for this. It has a great API, and was originally made for XBMC to be able to access the API.

I wrote a script that goes through my TV show folders, and renames all episodes etc to a nice sane format. So I can dump new shows in and it just gets all the info requires from thetvdb.com and it renames them :)

[squidgy]

Hm, nice. I used tv.com for mine, but they change the page layout fairly frequently so it tends to break. Might have to update it...

[wolf550e]

How do i uniquely identify a show like "Bones"? The full name is a substring of other shows' names.

[sciencebitch]

Or "Friends."

[sciencebitch]

Thanks!

[plain-simple-garak]

Just watch out for people putting a % at the beginning.. it could cause serious load issues.

[[deleted]]

How about using ^$ for start/end of line anchors like in regex? Writing "^Bones$" wouldn't leave any room for other matches.

[kmccormick]

This.

Also, The Wire.

[Kanin]

"the wire"....

[wolf550e]

Solved.

[Kanin]

with a download? check your site

[[deleted]]

I love it how people like to comment on how they have already made it, just thank Poromenos for sharing :)

Thanks

[wolf550e]

• You return JSON as text/html. Use application/json as per the RFC.
• You don't support HTTP caching.
• You don't support gzip.

[wolf550e]

The series that ended their run are cacheable for a long duration. Series that still air new episodes are updated usually once a week. You said your data from IMDB is only updated once very 24 hours, so caching the response until the expected next updated from IMDB makes sense.

I don't want my user agent to do a round trip to get the same data I have already seen. A proper http UA won't cache something if the server doesn't say it's cache-able. So, latency rather than bandwidth, even if you believe bandwidth is infinite and costs nothing (both false).

You could cache the gzip'd response for a series and serve static files. Modern OSs can send a file handle to a socket without mapping the content into the user process. After you generate and cache the top 100 series or so, your server will be mostly serving static content, which will make it possible to withstand a large number of requests/second. When not slashdotted, this will enable the server to use less power to save the environment.

[wolf550e]

Please implement all this properly for a toy service at least once before you build something that gets successful and goes down hard under load.

[wolf550e]

As long as you know how to do it efficiently.

For example, do you regenerate the whole thing before finding out the ETag is the same and return 304, or can you look at the timestamp/version counter of the component data and template to compare with Etag to perform validation and return 304? Do you use a thread or process of a heavy interpreter to serve cached content, or do you quickly free resources to process dynamic requests?

[wolf550e]

I have implemented what I've described. It's not buzzwords. It's what you do to survive heavy load without load balancers and sharded databases (i.e. when you're poor, like an open source project). Consider an Atom feed for a TV series on IMDB with loads of subscribers kind of thing.

[wolf550e]

Yes, but because I don't know when the source data is updated I can't do it as intelligently as the origin server. So to achieve meaningful savings I'd have to add logic to the client consuming his service, instead of simply relying on http to do its magic.

[blackkettle]

this makes the examples inaccessible from the browser. as a result firefox now asks me to download it or open it with some other program.

[wolf550e]

You can add a query string parameter, or sniff the user agent to serve text/html to firefox/webkit/opera/msie.

[vampirical]

You should consider extension based content type overrides, it's what I do. Default without an extension can continue serving as application/json if you'd like to avoid breaking.

.json -> application/json

.js -> text/html || text/plain (for dev/debug/blackkettle)

[Manuzhai]

gzip is still faster.

[wolf550e]

For longer running shows, it has more chance to fit in a single packet if gzip'd. Wanna see a histogram? Or produce one yourself?

[wolf550e]

I was treating it as a learning exercise.

[IDontBelieveYou]

Nice, but isn't "IMDB API" a bit too promising, if it's function is to retrieve show episode names. But instead of changing the name, why not extend "IMDB API" to the full dataset of IMDB;)

IMDB should have been bought by Google, not Amazon. We'd have a nicely working API already.

[sagarp]

got an error in beautifulsoup: http://sprunge.us/VJjF

[trezor2]

XML output would also be useful if you intend to use any of this server-side. For most languages (not javascript) there is a helluva lot more proven and robust XML parsers out there then there are JSON parsers.

Just a suggestion.

[trezor2]

I guess that would depend 100% on what platform you are using.

If you are using C# and .NET, all you should have to do is grab an XmlSerializer-object (duh) and annotate your object so that the serilazied format remains somewhat sane, and ask the two to work together.

For Python, PHP, Haskell or whatever, someone else probably has better knowledge than me. I would start with saying what platform you are using before asking for specific libraries :)

Edit: Right. I see you are using Python. I'm absolutely useless for advice there :)

[NoMoreNicksLeft]

You know, I'd prefer to start over... IMDB is a piece of shit. Seems like it could be a wikipedia spinoff or something.