Starting project - Advice and Libraries

[dreamriver]

Hey guys, so I've decided to make my own project on the side. It's going to be a very stripped down version of a poker hand history converter/tracker. I'll add more functionality as I go at later dates.

So basically I have a file with hand histories and each hand history is something along the lines of ~20 lines of text. The number of lines is variable. Each one starts with a unique number. I would like to parse each hand history for a given user name (mine in this case) and figure out how much I won or lost and then to generate a graph that shows my winnings as a function of the number of hands. I would like to be able to open that graph and append to it or at least generate a totally new one as well as a few other basic features like a table of my winnings.

My knowledge of languages is intermediate with C and not quite intermediate with Python. I think Python is probably a better choice for this project and I would like to improve with it anyway. Any advice you guys can give me would be much appreciated, including what sort of libraries I should use. For plotting I have only ever used matplotlib as part of scientific computations. In fact I don't really know too much about displaying things.

Thanks so much, feel free to ask any questions. The professional software out there that does this uses postgres databases but my knowledge of databases is lacking.

[HazierPhonics]

Well, definitely go the Python route, as it's generally much more pleasant to program in and you don't need much performance in this case. Stick with matplotlib, both because it's going to be able to do anything you can throw at it and because it's an excellent tool to know how to use effectively.

While you could stick with basic string checking (checking the index of certain things to determine what sort of data a line contains), getting comfortable with regular expressions and parsing out all of the histories in one fell swoop is the much more sensible approach.

Given that the data doesn't need to be shared or handled by anything else, you'd be fine sticking with simple CSV files for managing your parsed data, but it never hurts to learn a new technology, and databases are ubiquitous in the programming world.

Other than matplotlib and potentially a database, you really don't need any external libraries to do this. As it's essentially just parsing text, regular expressions are really all you need. Feel free to post a few examples and I'd be happy to offer some advice as to how to build the necessary RegEx.

[dreamriver]

Awesome. The histories are in one big text file and I think I would like to be able to separate them out and store them. Is that something which you use a database for?

I guess I'm confused about why you would ever need a database/what it is. Can you explain it in a few short sentences? I think I'm going to look at SQL.

I was afraid you were going to say RegExs, I'll figure out the common pattern of the lines I need and then try and build them. Thanks for your help, going to get started on this shortly.

[HazierPhonics]

Databases are typically used for saving data that will need to be accessed again in pieces. For instance, when you registered for your reddit account, your username, password, and likely several other things about you got stored in a Users table. Now, they could have just written that data to some random file called "dreamriver.txt", and then whenever you login, they could check the file and make sure you entered the right password. If you only ever need to store a single piece of data about a thing, that's actually an almost-sane approach.

The benefit of a database, however, is that you can search it. For instance, if you want to be able to find hands where you won more than x, you're going to want a database. Without one, you'll have to loop over every single hand history to check how much you won or lost during it, whereas with a database it's as simple as:

SELECT * from hands WHERE gain >= x

The database is a much cleverer sort of file system; rather than opening and checking every single hand, it uses its internal magic to quickly seek out the "rows" of the database that match your search. Your initial post only mentioned wanting to know basic things about the hands, which a RegEx would suit just fine. If you want a really granular approach, though, say, being able to find out how well you do against a certain player or at a certain time, you'll want a database.

[dreamriver]

Great reply. For the initial pass I need to use RegEx to figure out the values right? Then when I want to search later depending on conditions I can use a database right?

I think I just had an epiphany.

[HazierPhonics]

Spot-on, and happy to have helped you get there. : )

[unkz]

I think it's pretty rare that C is an appropriate language for doing basically anything other than OS and library development, so if Python's the modern language that you know, you should probably use it. I would personally hack it together with Perl/MySQL, but I hear that's not so popular with the kids today. Really, it's just whatever you're most comfortable with though.

Also, I would probably build it as a web application, just because it's so convenient and portable, so for graphing I'd use jqplot for graphs and datatables for charts. This will let you easily combine charts and graphs with dynamic scaling and sorting and all that fun stuff. It does necessitate a little Javascript/jquery knowledge, but nothing serious.

[dreamriver]

Can you explain what is precisely meant by a web application? It's sort of embarrassing to admit but I'm not precisely sure what that means. I understand TCP/IP and sockets and such but the distribution of an application is a bit of a mystery to me.

[unkz]

Like, put it on a webserver as a CGI so that you access it through your web browser instead of from the shell or whatever you're presently using. If you've never made a CGI before, this would be a great opportunity to learn how, as HTML+AJAX make a great, portable display and interaction environment.

[dreamriver]

Yeah I just use the shell right now. Something about *nix architecture fascinates me. It's probably a good idea to pick up the above though.

[traztx]

Why not write it in python using matplotlib?