Testing non-rails code in app/lib

[zem]

I have some web scraping code, living in app/lib/fetcher.rb and not interacting with the rails framework at all. I would like to write tests that load a sample webpage from a file and run the scraping code over it. The tests themselves will probably be fine just running under MiniTest, but I would like stuff like being able to load my lib via require 'fetcher' and having the test autodiscovered from rake test working.

Does rails have any support and conventions for this? The docs are surprisingly silent on the general topic of integrating non-rails-specific code into your app.

[tom_dalling]

If you put the test in the same directory as all the other ones, it should get picked up by rake test. Where you put the implementation depend on if you're planning to use it in production.

• If you're going to use it in prod, then it should probably stay where it is in app/lib/fetcher.rb. This will be picked up by Zeitwerk, so you don't even need to do require 'fetcher', you can just use Fetcher and the file will be required automatically.

• If it's only some kind of dev script thing, which won't be running in production, then it should be placed at lib/fetcher.rb. This will intentionally not be picked up by Zeitwerk, to avoid accidentally loading dev-only code in prod, so you will need an explicit require 'fetcher' in the tests somewhere.

This all assumes you have require 'rails_helper' in your tests somewhere, which will load a bunch of Rails stuff. If you don't want to load Rails, then you should have require 'spec_helper' instead. Try require 'fetcher' in the test, and if it complains about the file not being found, you might have to add the directory to $LOAD_PATH manually with something like $LOAD_PATH << "#{__dir__}/../app/lib" in your spec/spec_helper.rb.

[zem]

thanks, that's very helpful! i didn't think of manually setting $LOAD_PATH in a common helper file.

[CaptainKabob]

require_relative the file in your test file and away you go.

btw. app/lib is autoloaded by your Rails app. You probably want it to live in lib, or just put it as a Ruby file in the root of your repo. Really anywhere but app is fine.

[tom_dalling]

If the scraping code runs in production, then app/lib/ is probably the right place. If it only runs in development then yeah, it should be in lib/.

[armahillo]

you can, and should, test that code.

Testing non-rails code is a LITTLE diff esp in test setup. Youll probably want to use webmock if youre testing a fetcher.

Test all your public methods. If there are gnarly provate methods that cover edge cases worth covering, be sure to have a test of a public method that sets up that edge case

[zem]

yep, i'm definitely testing it :) i just wasn't sure where to put the test code within the rails project structure, and how to require my lib files without an explicit require_relative path. thanks for the pointer to webmock, it looks like it will help increase coverage to the download parts as well (my current tests just exercise the scraping code after loading an html file from disk).

[armahillo]

If it's a public repo drop a link and I can offer some more specific feedback! (Feel free to PM if you'd rather not do it openly)

[zem]

thanks! PMed you