The Conservative MP arrested on suspicion of rape and other offences was bailed, as Liz Truss said she was worried that there were again appalling accusations against a parliamentarian.
The MP was arrested on Tuesday after being accused of rape and sexual assault offences spanning seven years between 2002 and 2009.
Scotland Yard said the man in his 50 s was also indicted on charges of indecent assault, abuse of position of trust and misconduct in public office.
The MP has not been suspended by the Conservative party but the chief whip, Chris Heaton-Harris, asked him to stay away from parliament.
The foreign secretary said she did not know the identity of the MP, but defended the decision of the whips, saying the party needed to wait until the end of the police investigation.
She added: I am very concerned about the reports I have heard. This is a matter for the police.
The police are currently working on the case but it is worrying that we are seeing these appalling accusations about a parliamentarian. The arrest comes at a time when the Conservatives have faced several scandals relating to sexual abuse and misbehaviour.
A date was announced for two byelections on Tuesday triggered by the resignation of Conservative MPs. Imran Ahmad Khan, the former MP for Wakefield, was convicted of sexually assaulting a 15-year-old boy, while former Tiverton MP Neil Parish admitted to watching pornography in the House of Commons chamber.
Both contests take place on June 23rd with Labour hoping to win in Wakefield and the Liberal Democrats positioning themselves as the challengers in Tiverton. After Owen Paterson resigned over paid lobbying, the Lib Dems pulled off a shock victory in North Shropshire last year.
Another Conservative MP, David Warburton, representing Somerton and Frome, has had the whip suspended while under investigation for sexual harassment by the Commons independent complaints and grievance scheme.
Details of the MP's arrest emerged during what the government had hoped would be a crime week with announcements on toughening up policing. On Tuesday morning, the prime minister told his cabinet that crime, crime, crime is what we want to focus on.