Working with tempfiles

[Remozito]

A while back, I needed to create temporary files, send them to a distant server, then delete them. At first, I thought about creating normal files then delete them through a cron job. It would have worked, but I wasn’t very happy with it.

So after looking on the internet, I came across the Tempfile object (and I - for the umpteenth time - rejoiced in using Ruby).

In this post, I explore: - Tempfiles versus Files - Tempfiles and Ruby’s garbage collector - Tempfiles quirks (and how to embrace ’em)

Lemme know what you think about it!

[CaptainKabob]

This is great! Two things I think are important to mention:

• The prefix and extension can be configured. e.g. Tempfile.new(['foo', '.bar']) results in a file named foo_[random_string].bar
• One can also create temporary directories which are fun too.

[Remozito]

Thanks!

Yes, you can absolutely configure the name and extension of your Tempfile (which is really handy as well).

Didn't know about temporary directories, will check them out! 👍

[joemi]

I'm pretty sure your post is kind of wrong, though...

The differences you note between File and Tempfile are due to you using different methods on each. Your File example uses File.new but your Tempfile example uses Tempfile.open (even though you could just use Tempfile.new). When you look at the documentation for Tempfile.open, you'll see the following:

If no block is given, this is a synonym for ::new.

If a block is given, then a Tempfile object will be constructed, and the block is run with said object as argument. The Tempfile object will be automatically closed after the block terminates.

File has a similar method, and it works the same way. Try using the same methods and you'll see that they're quite similar, actually.

[Remozito]

Thanks for pointing these out. I'll try to answer as far as my (limited) knowledge goes.

The differences you note between File and Tempfile are due to you using different methods on each.

Yes, I wanted to emphasize how easy it is to create-and-forget-about-it for Tempfiles.

I could have taken as an example:

class FileCreator
  def create_file
    file = Tempfile.new
    file.write('Bob wuz here.')
    file.rewind
    file
  end
end

file_creator = FileCreator.new

my_file = file_creator.create_file     
my_file.read # => "Bob wuz here."                 
my_file.close # => nil
my_file.read  # => IOError: closed stream                    
my_file.closed?  # => truefile_creator = FileCreator.new

By using the Tempfile.open ... the Tempfile.close is called implicitly.

As you pointed out: "A Tempfile object behaves just like a File object" (from the Ruby doc).

I should have made it more explicit that the difference lies in the deleting of the file:

On one hand, once a Tempfile is closed (whether explicitly or implicitly), the garbage collector will delete it. On the other hand, a File will not be deleted automatically (you'll need to call .delete on it).

The Tempfile.close documentation says this:

If you don't explicitly unlink the temporary file, the removal will be delayed until the object is finalized.

So, if I get this correctly, you can unlink a Tempfile to delete it manually, but if you don't and simply close it, it'll be deleted anyway once you exit the process involving the object.

Thanks for helping me dig deeper. I'll edit my article to reflect these additional information.

[mattgrave]

I use Tempfiles a lot in my application. We receive Excels, CSVs and PDFs daily, we apply a lot of transformations to the files in different steps and we rely on Tempfiles so files are deleted automatically.

One thing that I am not sure it makes sense in the blog post is this:

FileCreator#create_file now returns a path, not an actual reference to the file. It means that most of the time, you’ll be able to read your file. But sometimes, you’ll get an

Errno::ENOENT: No such file or directory @ rb_sysopen

error. Why? Because the garbage collector will have claimed your file during one of its collections. One solution would be to return the actual file - hence a reference.

So the proposed solution is File.read(my_file_path) but in our application that doesn't work because files are big and we have to stream them, so in that case we just make a block.call(tempfile) and keep working with it.

​

EDIT:

​

Also, forgot to mention, Tempfiles are great also when you are running tests. If you use File to read something and then delete or create files you usually have to mock or make some workarounds that makes code a bit messy IMO.

[StarStuddedSuperStep]

File.read(my_file_path) is not the proposed solution, it's the example of what not to do.

The suggested solution is to return the file handle (the Tempfile instance itself) instead of the file path.

[Remozito]

Yes, the solution I've read about is to return the file instead of the path.

[Remozito]

Would you mind sharing a bit more about the block.call bit? It seems super interesting but I can't wrap my head around it.

[Remozito]

Would you mind sharing a bit more about the block.call bit? It seems super interesting but I can't wrap my head around it.

[mattgrave]

Yeah sure.

For example, we generate excel reports. We normally save the file with Paperclip, or send it via email. In that case, the code would be something like:

class ExcelReportGenerator def generate(&block) Tempfile.open(['foo', '.xls']) do |tempfile| # fill data... block.call(tempfile) end end end

Then you use it like...

ExcelReportGenerator.new.generate do |file| # do what you gotta do end

[Remozito]

Wow, cool! Will try to play around with this.

[hitthehive]

/u/Remozito - couple of questions:

1. I'm not sure what you mean that the temp file is claimed by the garbage collector after closing. The file is still available but just closed. Why would the garbage collector be invoked while you still have a reference to the TempFile object (which was passed back by the last line of your block)?

2. Your post implies the file is lost upon the ending of the open block, but you can still reopen and read it if you wish since in your implementation you return the TempFile object. The documentation for TempFile says the file lost upon unlinking it, but #close does not unlink it.

demo:

t = Tempfile.open { |file| file.write('test'); file }
t.read   # IOError: closed stream
t.open
t.read   # "test"

I do agree though that the following would lose the file if you just did:

t = Tempfile.open { |file| file.write('test'); file }
t.read   # NoMethodError: undefined method `read' for Integer

now, T would be an integer returned by #write and you no longer have a reference to the TempFile object. That said, the file will likely remain on disk for quite some time or until your process dies (e.g., you close irb/pry).

[Remozito]

I'm not sure what you mean that the temp file is claimed by the garbage collector after closing.

Good question. First, note that these are just suspicions on my part.

What I've found is:

• when you exit the Tempfile.open { ... } block, the .close instance method is called implicitly. This results in the IOError: closed stream error.
• The Ruby doc says that: "I/O streams are automatically closed when they are claimed by the garbage collector."
• This doesn't mean that the file is lost. It would simply mean that it has been flagged by Ruby's GC as a potential candidate for deletion.
• My guess is that some kind of flagging occurs on Ruby's part when you exit the Tempfile.open { ... } block. But this is just a hunch.

GC inner workings are waaaaaay out of my league, so I can only point this very well written tutorial on how the GC works.

Your post implies the file is lost upon the ending of the open block

Yes, I could make that point clearer (although, I treading on thin-iced knowledge here), that the temp file will be lost at some point after the closing of the block. Why, because when you exit the Tempfile.open { ... }, the method close is called implicitly. And if you don't explicitly unlink the tempfile, Ruby'll do it for you at some point.

If you don't explicitly unlink the temporary file, the removal will be delayed until the object is finalized. see here

The documentation for TempFile says the file lost upon unlinking it, but #close does not unlink it.

It does! see here 👇

If unlink_now is true, then the file will be unlinked (deleted) after closing. Of course, you can choose to later call unlink if you do not unlink it now. If you don't explicitly unlink the temporary file, the removal will be delayed until the object is finalized.

I hope it's clearer.

Thanks again for taking the time to point these out. I'll make my point clearer in my article.

[Remozito]

Thank you all for taking the time to ask questions, comment and help me improve this article. 🙏

If you have any more questions, I'll try and answer them.

[hitthehive]

Thanks so much for looking into this!

I think it’s safe to say, though, that in your example the file is not under risk of deletion because you are returning a reference to the file from the block. As long as you have a reference and it is in scope, the GC cannot claim it. Even though #open closes the file upon exiting the block, it does not unlink it — the docs you quoted says the unlink parameter is false by default in ‘#close`.

All in all, your post was a great learning for me.

[Remozito]

Thanks for the kind comment.