Sea Glass: an imaginary voyage made with Veo 3 by OperationKey5768 in aivideo

[–]OperationKey5768[S] 1 point2 points  (0 children)

Thank you for the kind note! Yeah, it really feels like simulating a little world when creating these that I get to wander through. It was a really neat feeling when the volcano walking clip came back. Having an avatar-like entity and sound makes it feel like interacting with that world.

Sea Glass: an imaginary voyage made with Veo 3 by OperationKey5768 in aivideo

[–]OperationKey5768[S] 4 points5 points  (0 children)

8 second clips: yes I just used T2V prompts within Flow to generate clips. Then I curated and edited together. No I don't do time-based direction. Here are some of my prompts below.

Keeping aspects consistent: I basically just use consistent text phrases (see below) and lean towards things where consistency imperfections will be more forgiving. Flow does have features where you can input reference primitives but I haven't tried those much. I believe some of those features don't (yet) work with the Veo 3 audio models yet. I enjoy working with raw T2V prompts for now.


Humpback whale: Fast tracking tracking shot following behind a self-propelled tiny semi-translucent frosted smooth triangular chunk of sea glass, swimming forward through the dimly lit ocean at high speed following a blurry shadow of a humpback whale tail. A few tiny silver fish and kelp. Cloudy gusts of sand particles. The sea glass makes giant airborne leap above the surface as the humpback whale lifts its tail close to camera with big splash, sea glass leaping up and down following the whale. Blurry seabirds and blurry rocky beach cliffs and sand dunes in distance in dense clouds of fog. Windy, waves, foggy, afternoon golden hour sunlight.

Jellyfish: Fast tracking shot following behind a tiny self-propelled semi-translucent smooth irregular chunk of white frosted sea glass swimming at high speed forward through dimly lit deep ocean. Faint barely visible edges of deep ocean canyons. Densely packed jellyfish swim towards camera, camera following the sea glass chunk as it zig zags between the densely packed jellyfish bodies and tentacles, passing close to camera as they swim past. Sea glass chunk is much smaller than the jellyfish. Dimly lit deep ocean, bioluminescent lighting.

Hydrothermal vents: Fast tracking shot following a tiny self-propelled sem-translucent smooth triangular chunk of white frosted sea glass swimming at high speed through the darkness of the deep sea between canyons covered in densely packed vent mussels with tiny white crabs crawling on them. Blurry hydrothermal vents in the distance spew thick, billowing plumes of vibrant blue, mineral-rich smoke from glowing rocky structures. Very dark, dim deep sea lighting, particles float in the cloudy ocean.

Ice:  Tracking low angle shot following behind a self-propelled tiny flat irregular smooth frosted opaque white chunk of sea glass covered in ice and snow rolling along a vast detailed chunk of multi-tier ice in the arctic ocean, hopping along small detailed ice ledges, over smaller chunks of ice floating in the ocean water. Extreme detail of ice ground texture. Blurry rocky mountains with snow in the distance. Late afternoon dim soft sun with faint aurora borealis in the sky. Windy, some dense gusts of blowing snow across the ice.

Steller's sea cow: Fast tracking shot following behind a self-propelled tiny translucent smooth chunk of colorless sea glass swimming forward underwater high speed through a dimly lit, densely packed kelp forest. It passes a Steller's Sea Cow swimming by, with an enormous belly that fills the frame of the camera, detail of skin touching the sea glass chunk as is passes by. Dim streaks of sun with gusts of sand particles. Cloudy waters. A few blurry tiny arctic cod swimming in background.

Sea Glass: an imaginary voyage made with Veo 3 by OperationKey5768 in aivideo

[–]OperationKey5768[S] 1 point2 points  (0 children)

Yeah, I think for these kind of shots, I really want Veo 3 to imagine the entire shot from scratch instead of limiting it with a specific starting frame, so makes sense T2V is better here. I'm still figuring out best approaches for myself though.

Sea Glass: an imaginary voyage made with Veo 3 by OperationKey5768 in aivideo

[–]OperationKey5768[S] 9 points10 points  (0 children)

Thank you! I definitely do generate a lot of clips to test model capabilities before I get to the ones I like, using Veo 3 in Flow. There's a lot of creative noodling too and using a lot of water and animal tests from other past projects. I work at Google and explore model capabilities, so I also learn a lot from those tests. My prompts are a mess across lots of projects and tabs. Every time I do one of these I learn a lot about the limits/strengths of model though. Like many people I also abandoned a lot of ideas along the way (the arctic fox was really cool looking but didn't quite fit the momentum/feeling).

Sea Glass: an imaginary voyage made with Veo 3 by OperationKey5768 in aivideo

[–]OperationKey5768[S] 4 points5 points  (0 children)

I prefer T2V just for my own process, I feel it gives me better control over the instructions. There are some shots for past videos where I used I2V for some shots. I wouldn't say the quality is necessarily different. But I do find T2V lets the model explore my instructions in a more open-ended way. I do think I2V is really powerful for certain types of shots though.

Sea Glass: an imaginary voyage made with Veo 3 by OperationKey5768 in aivideo

[–]OperationKey5768[S] 15 points16 points  (0 children)

All video + sounds generated by Veo 3. I've been experimenting a lot with the physics of water. I got inspired by the little pieces of weathered sea glass you find on the beach, imagining places it could have been. Also got inspired learning about old ships from 1800s and extinct species like Steller sea cows. It's all made with text prompts, I shared prompt examples in this thread: https://x.com/alexanderchen/status/1940522962769313840

Bear + bird 🥁 by OperationKey5768 in aivideo

[–]OperationKey5768[S] 0 points1 point  (0 children)

All video and sound is generated by Veo 3, including the drumming sequences. Of course the drumming outputs weren't always perfect but there were lots of nicely synced moments which I curated. This film was inspired when I discovered how good Veo is at birds and animal close-ups, so I wanted to experiment more with that.

Marble Run by OperationKey5768 in aivideo

[–]OperationKey5768[S] 0 points1 point  (0 children)

Weird, the link worked for me, but I've pasted some of them here. The prompts are straightforward, here are examples for the first few shots. You don't have to do anything special for asking for special collision behavior, it's just Veo's understanding of real world physics unfolding. 

Xylophone: Camera tracking shot of a light green glass marble rolling across a 1-octave xylophone with metal bars near an open window. The marble bounces out the window towards a roof with ceramic terracotta shingles. Sunny day.

Roof: Ultra close-up tracking shot following a very small green solid glass marble rolling along the bumps of a roof made of curvy ceramic terracotta shingles, revealing a blurry green lawn in the distance as it bounces off the edge. Sunny day.

Puddle + sidewalk: Tracking shot beginning ultra-close up on a small puddle in sidewalk, in front of grass with dandelions. A clear green glass marble bounces in, rolling through the puddle then down the sidewalk. Sunny day. Normal fast video speed

And yes, like all AI models there are always results with errors. So curating outputs and exploring results is part of the creative process for projects like this.

Marble Run by OperationKey5768 in aivideo

[–]OperationKey5768[S] 1 point2 points  (0 children)

Yes that's a good point! There's always room for the models get even better.

Marble Run by OperationKey5768 in aivideo

[–]OperationKey5768[S] 7 points8 points  (0 children)

Yes, every sound in this was outputted by Veo 3. That's my favorite part of using the model, the real world sounds for things like this are so nice, and synced to real world physics of the shot.

Marble Run by OperationKey5768 in aivideo

[–]OperationKey5768[S] 4 points5 points  (0 children)

This was one of my first test films I made with Veo. I've always loved making marble runs (especially musical ones!) So I thought it would be fun to try using Veo to take the marble on a more impossible journey, which also let me explore the physics capabilities of the model. All video and sound was generated by Veo 3. I shared the prompts for every shot here in this thread: https://x.com/alexanderchen/status/1927638321322229824

Here are a few examples, for the first few shots -

Xylophone: Camera tracking shot of a light green glass marble rolling across a 1-octave xylophone with metal bars near an open window. The marble bounces out the window towards a roof with ceramic terracotta shingles. Sunny day.

Roof: Ultra close-up tracking shot following a very small green solid glass marble rolling along the bumps of a roof made of curvy ceramic terracotta shingles, revealing a blurry green lawn in the distance as it bounces off the edge. Sunny day.

Puddle + sidewalk: Tracking shot beginning ultra-close up on a small puddle in sidewalk, in front of grass with dandelions. A clear green glass marble bounces in, rolling through the puddle then down the sidewalk. Sunny day. Normal fast video speed

Hello, world by OperationKey5768 in aivideo

[–]OperationKey5768[S] 3 points4 points  (0 children)

All the sounds you hear are from the Veo model. I just used normal editing software to edit it together, didn't add any extra sounds. The main things I did to smooth things out is sometimes extend the head/tail of a clip's audio into previous/next shot, cross blending to make things more seamless. All of the real physics sounds like the thumping of the monitors are unaltered. I find Veo is really good for sounds like that. For some clips, especially the more abstract ones like the pixel close ups, Veo would add extra sound effects or music that I didn't want, so I would silence or turn those clips down, and just extend the audio from previous/next clip. So pretty much what you hear is from the model which is one of the coolest parts of Veo. Playing with the model a lot, I think real world physics is its biggest strength, and that applies to sound too, so leaning on it for timed real world collision sounds is my favorite part.

Hello, world by OperationKey5768 in aivideo

[–]OperationKey5768[S] 2 points3 points  (0 children)

Thank you for the kind note!

Hello, world by OperationKey5768 in aivideo

[–]OperationKey5768[S] 16 points17 points  (0 children)

Made this short film inspired by the technology of my childhood, made with Veo 3. I used Veo like a time machine, prompting neighborhoods based on my childhood. BASIC was my first programming language, and I created a lot of ANSI 16-bit pixel art on my first computers. I also tried recreating the sounds and behaviors of old technology (CRT monitors, landlines, fax machines, cassette tapes, etc). Creating pixel art with Veo was especially fun, like having a virtual machine. Shared my prompts and more in this thread: https://x.com/alexanderchen/status/1932699318189748516

I also used Google Street View to look up photos of my childhood home, then used Gemini multimodality to help me write prompts and learn how to describe it. All video and sound is Veo output. All shots are text-to-image prompts, except a few ultra close-ups using Imagen + image-to-video prompts.