To unlock Metal Sonic you must beat the game with Sonic, Knuckles, Silver, and Shadow. (In no particular order)

To unlock new cups (as well as obtain Chaos Emerald cards) for Cup Circuit mode, you have to complete Easy, Medium, and Hard challenges in specific zones in Challenge Mode as follows:

Unlockables

Blue Chaos Cup Complete all challenges for Death Yard Zone

Cyan Chaos Cup Complete all challenges for Colosseum Highway Zone

Green Chaos Cup Complete all challenges for Forest Falls Zone
Grey Chaos Cup Complete every challenge in the game

Purple Chaos Cup Complete all challenges for Meteor Base Zone

Red Chaos Cup Complete all challenges for Sky Park Zone

Yellow Chaos Cup Complete all challenges for Crystal Mountain Zone

In order to unlock certain skins you must complete story mode with certain characters.

Knuckles Skin Complete story mode with Knuckles.

Metal Sonic Skin Unlock Metal Sonic.

Shadow Skin Complete story mode with Shadow.
Silver Skin Complete story mode with Silver.
Sonic Skin Complete story mode with Sonic.