Whitehall and Parliament Street were sealed off by police this afternoon as armed police and forensics officers rushed to the scene at 2.22pm.
The suspect was found with knives in his bag following a stop and search as part of an ongoing operation, a Scotland Yard spokesman said.
He was seen co-operating with police after they put handcuffs on him while kneeling on the ground on a traffic island as he was questioned for about 20 minutes by police.
The man then allowed himself to be walked into an unmarked police van which took him to a South London police station.
Scotland Yard said he was already known to counter-terror police.
Police said there is no more threat to the area, however detectives are investigating the incident.
Some buildings in the area were also closed shortly after 2pm.
The incident comes just over a month after Khalid Masood mowed down pedestrians on Westminster Bridge in a rented car before fatally stabbing PC Keith Palmer in the grounds of the Parliamentary estate.