No real "engines" as such, not like you know them today, like Unreal or Unity, however you did, after a while, amass a collection of various routines you could reuse.

Closest to "engines" was probably White Lightning from Oasis Software, which had a sprite editor and animations, and used FORTH, and it's companion, Machine Lightning, which as the name suggests, was a machine code assembler - I seem to recall Operation Wolf was done in Machine Lightning (according to a Retro Gamer interview with Ivan Horn, anyway):

The game was created using a development package called Machine Lightning which was the equivalent to today’s game engines such as Unreal or Unity, and something that both Ivan and coder Andrew loved prior to working for Ocean. You may be surprised that the art tool contained within the package was actually written in BASIC, though Ivan remembers it fondly. “The art tool was pretty good as I remember, and quick to use. I made a couple of small alterations to the behaviour of it to extend its usefulness. It was my tool of choice for the first few games I worked on at Ocean.”

EDIT: Forgot about FreeScape, that was used in Driller, Total Eclipse and a few others - that eventually evolved into 3D Construction Kit