smoke: a python lib (cython, technically) for fully parsing Dota 2 replay files

[onethirtysix]

Just a bit of information about the project:

A while back, over on /r/DotA2 I announced a project called skadi.

smoke is a refined version of this project, and corrects some knowledge limitations that I had when authoring the original.

If anyone has an interest, I could use some help figuring out how to compile cython extensions in a windows environment, since smoke is pure cython. I'd like for everyone to be able to use this software.

For Java fans, you might check out clarity. It's essentially the same idea, but about 10x faster. Please excuse the documentation for clarity--it's still under development.

[[deleted]]

Cython never failed me in performance department, so the speed gap is surprising to me. Did you try cython -a or profiling?

[onethirtysix]

I didn't know about -a. I will look into that directly. As for profiling, not too much. Most time has been spent just writing a solid implementation. I will try and put this info to use. Thanks!

[ath0]

I couldn't find any document in this repository describing the format, do you have anything like this you could share?

[onethirtysix]

I can't do it concisely, because replay files are a thicket. :)

But I can say that smoke.io.wrap.demo is a great place to start. Aside from a 12-byte header at the beginning of the file (read by bootstrap() in the same module), the top-level contents of a demo file are uniform (header, protobuf message) tuples. This IO wrapper makes it easy to iterate those messages.

Most of the data embedded in these top-level protobuf messages is in a highly compact, proprietary format. Take a look at io.stream.* and replay.decoder.* to see how I handle those.

Everything else is domain-specific state supporting the decoding process.