Demo Page

This demonstration page presents the generations from 50 randomly selected prompts from the AudioCaps test set.

We present four audio sources: the consistency model fine-tuned with CLAP, the consistency model without CLAP-fine-tuning, the diffusion baseline model, and the ground truth.

The diffusion baseline queries the neural network 400 times per audio clip, while the consistency models query a same-sized network only one time.

Since the models are not trained on speech data, we do not expect them to produce meaningful speeches.

Prompt 0

Whistling followed by a child giggling and then Moe whistling.

Prompt 1

Some clanking and banging and a man speaking.

Prompt 2

A man speaking on a microphone as a crowd of people laugh followed by dinner plates clacking.

Prompt 3

Steam hissing followed by a train whistle blowing and a group of people talking in the background.

Prompt 4

A vehicle revving and accelerating as tires skid and squeak on a road.

Prompt 5

Steam escapes with a hissing noise.

Prompt 6

A man speaking continuously.

Prompt 7

Knocking sounds as race cars pass by.

Prompt 8

A man talking followed by plastic clacking then a power tool drilling.

Prompt 9

Humming of an engine with a woman speaking over a loudspeaker.

Prompt 10

A telephone ringing with loud echo.

Prompt 11

Released air hissing followed by a popping explosion then a metal ding persists as a person is laughing and a man is talking.

Prompt 12

Constant hissing with mean having conversation.

Prompt 13

A missile launching followed by an explosion and metal screeching as a motor hums in the background.

Prompt 14

An adult female speaks as a cat meows three times, and an electronic device plays in the background.

Prompt 15

Food and oil sizzling.

Prompt 16

Some light tapping on a computer keyboard and a baby crying.

Prompt 17

An electronic beep followed by a man talking.

Prompt 18

Sanding and filing then a man speaks.

Prompt 19

An aircraft engine humming followed by plastic clanking then an aircraft engine slowing down.

Prompt 20

Footsteps and scuffing occur, after which a door grinds, squeaks and clicks, an adult male speaks, and the door grinds, squeaks and clicks shut with a thump.

Prompt 21

A train horn blowing multiple times as a train runs on railroad tracks while a man and a young kid talk in the background alongside birds cooing in the distance.

Prompt 22

Strong gusts of wind are followed by cheers and shouts from several people plus the chatter of girl.

Prompt 23

Compressed air and steam releasing with a man faintly talking in the background.

Prompt 24

A man talking followed by a goat baaing then a metal gate sliding shut as ducks quack and wind blows into a microphone.

Prompt 25

A cat is meowing.

Prompt 26

A toilet is flushing followed by a cat meowing.

Prompt 27

A person speaks with distant humming and nearby clinking.

Prompt 28

A dog whimpering followed by laughing and barking.

Prompt 29

A vehicle driving by with tires briefly skidding and accelerating then slowing down.

Prompt 30

A horn and then an engine revving.

Prompt 31

Several people cheer and scream and speak as water flows hard.

Prompt 32

A person whistles to music.

Prompt 33

Laughing and speech in a slowed speed.

Prompt 34

A man speaking as insects are buzzing and wind is blowing into a microphone.

Prompt 35

Wind followed by splashing of water.

Prompt 36

A person whistling.

Prompt 37

Wood being scraped along with mechanical sounds.

Prompt 38

A woman speeches.

Prompt 39

A cat is meowing in a quiet environment.

Prompt 40

Wind blowing and a siren rings.

Prompt 41

Static and beeping.

Prompt 42

Musical whistling with wind blowing.

Prompt 43

An idle motorbike engine running.

Prompt 44

A jackhammer drilling and vibrating continuously.

Prompt 45

A train is passing by and sound its whistle.

Prompt 46

A motorboat engine running as water splashes and a man shouts followed by birds chirping in the background.

Prompt 47

A high frequency motor hums loudly and splashes water.

Prompt 48

A series of sharp, squeaky snoring noises.

Prompt 49

A bus horn honking as wind is blowing into a microphone before a bus drives by.