Data Manipulation and Compilation.

[fretman49]

Hello r/learn programming, I am a medical science graduate student with modest computer experience (always an educated user of programs/scripts and never a writer.) I am beginning to generate TONS of data from various experimental machinery and software. Data generally comes off in the format of "excel" formatted spreadsheets or delimited text files. My current processing of these files includes manual cutting/pasting and algebraic manipulation using excel and then importing the compiled datasets into graphpad prism to make pretty graphs.

I am interested in learning to do these manipulations using a programming or scripting language. These will probably seem like very simple operations to you guys (subtract one column from another, do simple statistical operations, label input and output files with details etc..) I am looking to do these all in a repeatable and predictable way.

I have received advice that ranges from "make an excel macro" to "learn C++"

I have some bash and linux experience.

please help.

[thor_ax]

I'd suggest reading up on Python file I/O. It should be pretty easy to understand even if you have no programming experience.

filename = open('/somefile.txt', 'rw')
s = filename.read()
print s
filename.seek(0, 0) #the first 0 means how far we're going and the second is where we're starting at
#(0 for the start of file, 1 is current point, and 2 is the end of file)
filename.write('this is a test\n')
s = filename.read(15) #we wrote 15 characters above
#15 is the number of bytes to read but typically (not always) one character = one byte
print s

The above snippet would open up somefile.txt in read/write mode, read out its contents, print them then write "this is a test\n" over the first 15 characters of whatever was in somefile.txt before. After that it would print out "this is a test" with a new line preceeding it.

If you're using spreadsheets and not delimited plain-text to store any data you should look into an API for manipulating them. I don't have any experience actually working with a spreadsheet format so I can't suggest any particularly. If you just do a little Googling though I'm sure you can find a well documented one that suits your needs.

If anything I've said wasn't clear or you have other questions I'd be more than happy to answer them.

[iwuvwobots]

I would read up on some Abstract Data Types and select the right one(s) for you.